2024-09-19T09:07:35.322 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T09:07:35.349 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T09:07:35.455 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911905 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/direct_io}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911905' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFGQccRaf8pR5l6ammMdCphO4gxOrV94uBlyLaq1CD1Lp7RUnkXFvmqm7U3hLWWkl8QFL9zSyoWi5Gsx7/O23ko= smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGiKtYcwkNEEjhQimZDq/7O22plDfTWG1gYyFtAJktImG7pBFRvIlrn9OHcPbi2XjWQjhLSxt4DTVlgOB6J4L18= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA4U5sNVA44UiE37/rSAa3TiYKVcbE2JaOWGxsODkqtbFWzD2KqkcphTlIqNMR0xqTgEoBLtbr9NgZK2emg5ffY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T09:07:35.456 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T09:07:35.456 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T09:07:35.457 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T09:07:35.458 INFO:teuthology.task.internal:Checking packages... 2024-09-19T09:07:35.479 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T09:07:35.480 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T09:07:35.480 INFO:teuthology.packaging:ref: None 2024-09-19T09:07:35.480 INFO:teuthology.packaging:tag: None 2024-09-19T09:07:35.480 INFO:teuthology.packaging:branch: squid 2024-09-19T09:07:35.480 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:07:35.480 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T09:07:35.701 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T09:07:35.703 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T09:07:35.710 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T09:07:35.710 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T09:07:35.717 INFO:teuthology.task.internal:Saving configuration 2024-09-19T09:07:35.737 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T09:07:35.744 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T09:07:35.771 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911905', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 09:02:31.014071', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFGQccRaf8pR5l6ammMdCphO4gxOrV94uBlyLaq1CD1Lp7RUnkXFvmqm7U3hLWWkl8QFL9zSyoWi5Gsx7/O23ko='} 2024-09-19T09:07:35.795 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911905', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 09:02:31.016663', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGiKtYcwkNEEjhQimZDq/7O22plDfTWG1gYyFtAJktImG7pBFRvIlrn9OHcPbi2XjWQjhLSxt4DTVlgOB6J4L18='} 2024-09-19T09:07:35.817 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911905', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 09:02:31.015444', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA4U5sNVA44UiE37/rSAa3TiYKVcbE2JaOWGxsODkqtbFWzD2KqkcphTlIqNMR0xqTgEoBLtbr9NgZK2emg5ffY='} 2024-09-19T09:07:35.818 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T09:07:35.824 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T09:07:35.824 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T09:07:35.824 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T09:07:35.824 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T09:07:35.914 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7feca5d27be0>, signals=[15]) 2024-09-19T09:07:35.914 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T09:07:35.920 INFO:teuthology.task.internal:Opening connections... 2024-09-19T09:07:35.921 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T09:07:35.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:07:36.000 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-09-19T09:07:36.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:07:36.078 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-09-19T09:07:36.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:07:36.155 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T09:07:36.162 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-09-19T09:07:36.181 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-09-19T09:07:36.181 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-09-19T09:07:36.239 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-09-19T09:07:36.240 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-09-19T09:07:36.240 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-09-19T09:07:36.240 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-09-19T09:07:36.240 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-09-19T09:07:36.240 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-09-19T09:07:36.240 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T09:07:36.240 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-09-19T09:07:36.240 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-09-19T09:07:36.240 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T09:07:36.241 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-09-19T09:07:36.241 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T09:07:36.241 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T09:07:36.241 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T09:07:36.242 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-09-19T09:07:36.264 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-09-19T09:07:36.283 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-09-19T09:07:36.284 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-09-19T09:07:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-09-19T09:07:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-09-19T09:07:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-09-19T09:07:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-09-19T09:07:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-09-19T09:07:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-09-19T09:07:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T09:07:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-09-19T09:07:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-09-19T09:07:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T09:07:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-09-19T09:07:36.343 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T09:07:36.343 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T09:07:36.343 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T09:07:36.343 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-09-19T09:07:36.365 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-09-19T09:07:36.383 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-09-19T09:07:36.383 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-09-19T09:07:36.442 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-09-19T09:07:36.442 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-09-19T09:07:36.442 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-09-19T09:07:36.442 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-09-19T09:07:36.442 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-09-19T09:07:36.442 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-09-19T09:07:36.442 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T09:07:36.442 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-09-19T09:07:36.442 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-09-19T09:07:36.443 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T09:07:36.443 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-09-19T09:07:36.443 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T09:07:36.443 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T09:07:36.443 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T09:07:36.443 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-09-19T09:07:36.464 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T09:07:36.474 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T09:07:36.480 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T09:07:36.481 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-09-19T09:07:36.483 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-09-19T09:07:36.486 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-09-19T09:07:36.502 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T09:07:36.509 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T09:07:36.509 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-09-19T09:07:36.541 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-09-19T09:07:36.544 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-09-19T09:07:36.656 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T09:07:36.729 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T09:07:36.744 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T09:07:36.744 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-19T09:07:36.744 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-19T09:07:36.745 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T09:07:36.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:07:36.765 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-19T09:07:36.822 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-503.el9.x86_64 2024-09-19T09:07:36.823 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-503.el9.x86_64 2024-09-19T09:07:36.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-09-19T09:07:39.224 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-19T09:07:43.358 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 20 MB 00:03 2024-09-19T09:07:51.792 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.5 MB 00:04 2024-09-19T09:07:53.243 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 122 kB/s | 19 kB 00:00 2024-09-19T09:07:54.380 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-19T09:08:00.036 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T09:08:02.026 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T09:08:02.026 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T09:08:02.124 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-19T09:08:02.134 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-19T09:08:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-09-19T09:08:02.134 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-19T09:08:02.134 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-19T09:08:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T09:08:02.134 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-19T09:08:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T09:08:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T09:08:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T09:08:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:08:02.135 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-19T09:08:02.135 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-19T09:08:02.135 INFO:teuthology.orchestra.run.smithi016.stdout:Install 4 Packages 2024-09-19T09:08:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:08:02.135 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 79 M 2024-09-19T09:08:02.136 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 126 M 2024-09-19T09:08:02.136 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-19T09:08:02.771 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 59 kB/s | 36 kB 00:00 2024-09-19T09:08:03.688 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-19T09:08:04.346 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-19T09:08:05.122 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 12 MB/s | 29 MB 00:02 2024-09-19T09:08:05.122 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-19T09:08:05.122 INFO:teuthology.orchestra.run.smithi016.stdout:Total 26 MB/s | 79 MB 00:02 2024-09-19T09:08:05.841 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-19T09:08:06.245 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-19T09:08:06.245 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-19T09:08:07.232 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-19T09:08:07.233 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-19T09:08:09.786 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-19T09:08:10.450 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T09:08:10.540 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T09:08:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T09:08:13.427 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T09:08:18.554 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T09:08:18.594 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T09:08:23.742 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T09:09:03.157 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T09:09:03.199 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T09:09:03.568 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T09:09:03.568 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T09:09:03.568 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T09:09:03.568 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T09:09:04.370 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T09:09:04.370 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:09:04.370 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-19T09:09:04.370 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T09:09:04.370 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T09:09:04.370 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T09:09:04.370 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T09:09:04.370 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:09:04.370 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-19T09:09:04.712 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T09:09:04.813 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T09:09:04.814 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T09:09:04.814 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T09:09:04.814 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T09:09:04.814 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T09:09:04.814 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-19T09:09:04.814 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T09:09:04.814 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T09:09:04.873 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T09:09:04.873 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T09:09:04.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-09-19T09:09:04.905 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-09-19T09:09:04.905 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-09-19T09:09:04.905 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-09-19T09:09:04.905 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 82.el9 2024-09-19T09:09:04.905 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-09-19T09:09:04.905 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T09:09:04.905 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-09-19T09:09:04.905 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8274670 2024-09-19T09:09:04.905 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-09-19T09:09:04.905 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T09:09:04.905 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T09:09:04.906 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T09:09:04.906 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T09:09:04.906 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-09-19T09:09:04.906 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-09-19T09:09:04.906 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T09:09:04.906 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-09-19T09:09:04.906 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-09-19T09:09:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:09:04.906 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T09:09:04.906 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T09:09:04.906 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T09:09:04.906 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-09-19T09:09:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:09:04.907 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T09:09:04.907 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T09:09:04.908 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 5.14.0-508.el9.x86_64 2024-09-19T09:09:04.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T09:09:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-19T09:09:06.950 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T09:09:06.982 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-19T09:09:06.984 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-19T09:09:06.999 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.VpYUdY2AZ3 2024-09-19T09:09:06.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VpYUdY2AZ3 2024-09-19T09:09:07.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.VpYUdY2AZ3 2024-09-19T09:09:07.206 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.VpYUdY2AZ3 is 6KB 2024-09-19T09:09:07.254 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.VpYUdY2AZ3 2024-09-19T09:09:07.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T09:09:07.332 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T09:09:07.333 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T09:09:07.333 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T09:09:07.333 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T09:09:07.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T09:09:07.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-09-19T09:09:37.490 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T09:09:37.491 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-19T09:09:37.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:09:55.901 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-09-19T09:10:04.902 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-19T09:10:04.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:11:04.907 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T09:11:16.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-19T09:11:16.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:11:17.216 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-19T09:11:17.595 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-19T09:11:17.595 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-19T09:11:17.595 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-19T09:11:17.612 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-508.el9.x86_64 2024-09-19T09:11:17.612 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T09:11:17.612 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T09:11:17.612 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T09:11:18.613 DEBUG:teuthology.parallel:result is None 2024-09-19T09:11:18.613 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T09:11:18.621 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T09:11:18.622 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T09:11:18.624 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T09:11:18.627 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T09:11:18.646 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T09:11:18.651 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T09:11:18.657 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T09:11:18.657 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T09:11:18.684 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T09:11:18.686 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T09:11:18.805 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T09:11:18.811 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T09:11:18.812 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T09:11:18.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:11:18.829 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T09:11:18.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:11:18.847 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T09:11:18.850 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T09:11:18.872 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T09:11:18.907 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T09:11:18.928 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T09:11:18.937 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T09:11:18.954 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T09:11:18.955 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T09:11:18.970 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T09:11:18.972 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T09:11:18.979 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T09:11:18.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T09:11:18.982 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T09:11:18.985 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T09:11:19.012 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T09:11:19.022 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T09:11:19.022 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T09:11:19.053 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T09:11:19.055 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T09:11:19.072 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T09:11:19.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T09:11:19.215 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T09:11:19.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T09:11:19.303 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:11:19.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T09:11:19.370 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T09:11:19.416 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T09:11:19.469 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T09:11:19.559 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T09:11:19.599 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-19T09:11:19.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T09:11:19.665 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T09:11:19.711 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T09:11:19.772 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T09:11:19.816 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T09:11:19.858 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-19T09:11:19.858 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T09:11:19.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-09-19T09:11:19.928 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-09-19T09:11:19.930 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-09-19T09:11:19.964 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:11:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:11:20.005 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:11:20.421 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T09:11:20.429 INFO:teuthology.task.internal:Starting timer... 2024-09-19T09:11:20.429 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T09:11:20.439 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T09:11:20.447 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T09:11:20.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-09-19T09:11:20.507 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-09-19T09:11:20.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-09-19T09:11:20.590 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-09-19T09:11:20.592 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-09-19T09:11:20.638 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-09-19T09:11:20.639 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T09:11:20.640 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-09-19T09:11:20.660 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-09-19T09:11:20.660 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-09-19T09:11:20.691 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-09-19T09:11:20.692 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-09-19T09:11:20.713 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-09-19T09:11:20.713 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive'} 2024-09-19T09:11:20.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:11:20.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:11:20.742 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:11:20.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:11:20.773 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:11:20.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:11:20.798 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T09:11:20.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-09-19T09:11:20.838 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-09-19T09:11:20.875 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-09-19T09:11:20.911 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T09:11:20.921 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T09:11:20.931 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T09:11:20.932 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi097.front.sepia.ceph.com,smithi142.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T09:15:18.778 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com')] 2024-09-19T09:15:18.779 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-19T09:15:18.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:15:18.856 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-19T09:15:18.933 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-19T09:15:18.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-19T09:15:18.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:15:19.036 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-09-19T09:15:19.122 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-19T09:15:19.123 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-19T09:15:19.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T09:15:19.202 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-09-19T09:15:19.291 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-19T09:15:19.292 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T09:15:19.302 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T09:15:19.302 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T09:15:19.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:15:19.306 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T09:15:19.306 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:15:19.308 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T09:15:19.308 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:15:19.338 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T09:15:19.339 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T09:15:19.354 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T09:15:19.355 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T09:15:19.364 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T09:15:19.380 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T09:15:19.382 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-09-19T09:15:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-09-19T09:15:19.394 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-09-19T09:15:19.394 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-09-19T09:15:19.407 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-09-19T09:15:19.408 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T09:15:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T09:15:19.418 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-09-19T09:15:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T09:15:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T09:15:19.433 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T09:15:19.448 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T09:15:19.482 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:15:19.486 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:15:19.486 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-09-19T09:15:19.486 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:15:19.486 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:15:19.486 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:15:19.487 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:15:19.487 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:15:19.488 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:15:19.489 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-19T09:15:19.489 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:15:19.489 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:15:19.489 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:15:19.489 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:15:19.505 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:15:19.508 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:15:19.508 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-09-19T09:15:19.508 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:15:19.509 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:15:19.509 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:15:19.509 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:15:19.510 INFO:teuthology.run_tasks:Running task install... 2024-09-19T09:15:19.518 DEBUG:teuthology.task.install:project ceph 2024-09-19T09:15:19.518 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T09:15:19.518 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T09:15:19.518 INFO:teuthology.task.install:Using flavor: default 2024-09-19T09:15:19.524 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T09:15:19.524 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T09:15:19.524 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T09:15:19.525 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:15:19.527 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T09:15:19.527 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:15:19.528 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T09:15:19.528 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:15:19.670 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T09:15:19.670 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T09:15:19.684 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T09:15:19.684 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T09:15:19.689 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T09:15:19.689 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T09:15:19.783 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T09:15:19.783 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:15:19.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T09:15:19.792 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T09:15:19.792 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-19T09:15:19.792 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T09:15:19.800 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T09:15:19.801 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-19T09:15:19.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T09:15:19.811 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T09:15:19.811 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T09:15:19.818 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T09:15:19.818 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T09:15:19.826 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T09:15:19.826 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T09:15:19.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T09:15:19.884 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T09:15:19.892 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T09:15:19.957 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T09:15:19.959 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T09:15:19.967 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T09:15:20.023 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-09-19T09:15:20.023 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-09-19T09:15:20.024 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-09-19T09:15:20.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-09-19T09:15:20.031 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-09-19T09:15:20.033 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-09-19T09:15:20.250 INFO:teuthology.orchestra.run.smithi142.stdout:45 files removed 2024-09-19T09:15:20.254 INFO:teuthology.orchestra.run.smithi097.stdout:45 files removed 2024-09-19T09:15:20.258 INFO:teuthology.orchestra.run.smithi016.stdout:45 files removed 2024-09-19T09:15:20.278 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T09:15:20.281 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T09:15:20.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T09:15:20.812 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 325 kB/s | 84 kB 00:00 2024-09-19T09:15:20.833 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 300 kB/s | 84 kB 00:00 2024-09-19T09:15:20.853 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-19T09:15:21.096 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T09:15:21.118 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-19T09:15:21.137 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T09:15:21.372 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T09:15:21.393 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T09:15:21.413 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T09:15:22.010 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-19T09:15:22.156 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.3 MB 00:00 2024-09-19T09:15:23.321 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.3 MB 00:01 2024-09-19T09:15:25.047 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-19T09:15:25.232 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-19T09:15:27.553 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-09-19T09:15:29.441 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-19T09:15:30.442 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.5 MB 00:01 2024-09-19T09:15:31.067 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 19 kB 00:00 2024-09-19T09:15:31.976 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 112 kB/s | 19 kB 00:00 2024-09-19T09:15:31.988 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-19T09:15:31.995 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-19T09:15:32.755 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-19T09:15:34.287 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-19T09:15:35.060 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-19T09:15:37.428 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-19T09:15:38.412 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T09:15:39.555 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T09:15:40.038 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T09:15:40.038 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T09:15:40.038 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T09:15:40.038 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T09:15:40.038 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T09:15:40.038 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T09:15:40.038 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T09:15:40.039 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T09:15:40.039 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-19T09:15:40.039 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-09-19T09:15:40.039 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-09-19T09:15:40.039 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-09-19T09:15:40.039 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-19T09:15:40.039 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T09:15:40.039 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T09:15:40.039 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T09:15:40.039 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T09:15:40.039 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T09:15:40.039 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T09:15:40.039 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T09:15:40.040 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T09:15:40.040 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T09:15:40.040 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T09:15:40.040 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T09:15:40.040 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T09:15:40.040 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T09:15:40.040 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T09:15:40.040 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T09:15:40.040 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T09:15:40.040 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T09:15:40.040 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T09:15:40.040 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T09:15:40.040 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T09:15:40.041 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T09:15:40.041 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T09:15:40.041 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T09:15:40.041 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T09:15:40.041 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T09:15:40.041 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T09:15:40.041 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T09:15:40.041 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T09:15:40.041 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T09:15:40.041 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T09:15:40.041 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T09:15:40.041 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T09:15:40.042 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T09:15:40.042 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T09:15:40.042 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T09:15:40.042 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T09:15:40.042 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T09:15:40.042 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-09-19T09:15:40.042 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T09:15:40.042 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T09:15:40.042 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T09:15:40.042 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T09:15:40.042 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T09:15:40.043 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T09:15:40.043 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T09:15:40.043 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T09:15:40.043 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-09-19T09:15:40.043 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T09:15:40.043 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T09:15:40.043 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T09:15:40.043 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T09:15:40.043 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T09:15:40.043 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T09:15:40.043 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T09:15:40.043 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T09:15:40.044 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T09:15:40.044 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T09:15:40.044 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T09:15:40.044 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T09:15:40.044 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T09:15:40.044 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T09:15:40.044 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T09:15:40.044 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T09:15:40.044 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T09:15:40.044 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T09:15:40.044 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T09:15:40.044 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T09:15:40.045 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T09:15:40.045 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T09:15:40.045 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T09:15:40.045 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T09:15:40.045 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T09:15:40.045 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T09:15:40.045 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T09:15:40.045 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T09:15:40.045 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T09:15:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T09:15:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T09:15:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T09:15:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T09:15:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T09:15:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T09:15:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T09:15:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T09:15:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T09:15:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T09:15:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T09:15:40.046 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T09:15:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T09:15:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T09:15:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T09:15:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T09:15:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T09:15:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T09:15:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T09:15:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T09:15:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T09:15:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T09:15:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T09:15:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T09:15:40.047 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T09:15:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T09:15:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T09:15:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T09:15:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T09:15:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T09:15:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T09:15:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T09:15:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T09:15:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T09:15:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T09:15:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T09:15:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T09:15:40.048 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T09:15:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T09:15:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T09:15:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T09:15:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T09:15:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T09:15:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T09:15:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T09:15:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T09:15:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T09:15:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T09:15:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T09:15:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T09:15:40.049 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T09:15:40.050 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T09:15:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T09:15:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T09:15:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T09:15:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T09:15:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T09:15:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T09:15:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T09:15:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T09:15:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T09:15:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T09:15:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T09:15:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T09:15:40.051 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T09:15:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T09:15:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T09:15:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T09:15:40.052 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-09-19T09:15:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T09:15:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:15:40.052 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-19T09:15:40.052 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-09-19T09:15:40.052 INFO:teuthology.orchestra.run.smithi097.stdout:Install 148 Packages 2024-09-19T09:15:40.052 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 8 Packages 2024-09-19T09:15:40.052 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:15:40.052 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 226 M 2024-09-19T09:15:40.052 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-19T09:15:40.329 INFO:teuthology.orchestra.run.smithi097.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 48 kB/s | 6.6 kB 00:00 2024-09-19T09:15:40.433 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-19T09:15:40.555 INFO:teuthology.orchestra.run.smithi097.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.9 MB/s | 896 kB 00:00 2024-09-19T09:15:40.647 INFO:teuthology.orchestra.run.smithi097.stdout:(3/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-19T09:15:40.672 INFO:teuthology.orchestra.run.smithi097.stdout:(4/156): ceph-immutable-object-cache-19.1.1-361 1.2 MB/s | 145 kB 00:00 2024-09-19T09:15:40.706 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T09:15:40.706 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T09:15:40.718 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T09:15:40.719 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T09:15:40.720 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T09:15:40.721 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T09:15:40.722 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T09:15:40.725 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T09:15:40.726 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T09:15:40.793 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-19T09:15:40.805 INFO:teuthology.orchestra.run.smithi097.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 15 MB/s | 2.4 MB 00:00 2024-09-19T09:15:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-19T09:15:40.819 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-09-19T09:15:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-19T09:15:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-19T09:15:40.819 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T09:15:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T09:15:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T09:15:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T09:15:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T09:15:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T09:15:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T09:15:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T09:15:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T09:15:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T09:15:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T09:15:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T09:15:40.820 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T09:15:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T09:15:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T09:15:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T09:15:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T09:15:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T09:15:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T09:15:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T09:15:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T09:15:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T09:15:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T09:15:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T09:15:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T09:15:40.821 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T09:15:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T09:15:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T09:15:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T09:15:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T09:15:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T09:15:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T09:15:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T09:15:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T09:15:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T09:15:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T09:15:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T09:15:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-09-19T09:15:40.822 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T09:15:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T09:15:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T09:15:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T09:15:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T09:15:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T09:15:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T09:15:40.823 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T09:15:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-19T09:15:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T09:15:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T09:15:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T09:15:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T09:15:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T09:15:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T09:15:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T09:15:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T09:15:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T09:15:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T09:15:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T09:15:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T09:15:40.824 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T09:15:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T09:15:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T09:15:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T09:15:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T09:15:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T09:15:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T09:15:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T09:15:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T09:15:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T09:15:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T09:15:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T09:15:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T09:15:40.825 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T09:15:40.826 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T09:15:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T09:15:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T09:15:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T09:15:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T09:15:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T09:15:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T09:15:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T09:15:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T09:15:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T09:15:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T09:15:40.827 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T09:15:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T09:15:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T09:15:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T09:15:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T09:15:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T09:15:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T09:15:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T09:15:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T09:15:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T09:15:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T09:15:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T09:15:40.828 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T09:15:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T09:15:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T09:15:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T09:15:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T09:15:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T09:15:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T09:15:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T09:15:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T09:15:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T09:15:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T09:15:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T09:15:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T09:15:40.829 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T09:15:40.830 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T09:15:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T09:15:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T09:15:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T09:15:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T09:15:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T09:15:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T09:15:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T09:15:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T09:15:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T09:15:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T09:15:40.831 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T09:15:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T09:15:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T09:15:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T09:15:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T09:15:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T09:15:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T09:15:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T09:15:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T09:15:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T09:15:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-09-19T09:15:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T09:15:40.832 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:15:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-19T09:15:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-19T09:15:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:Install 148 Packages 2024-09-19T09:15:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 8 Packages 2024-09-19T09:15:40.833 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:15:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 226 M 2024-09-19T09:15:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-19T09:15:40.922 INFO:teuthology.orchestra.run.smithi097.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 6.1 MB/s | 1.5 MB 00:00 2024-09-19T09:15:41.073 INFO:teuthology.orchestra.run.smithi097.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 18 MB/s | 4.7 MB 00:00 2024-09-19T09:15:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 19 MB/s | 21 MB 00:01 2024-09-19T09:15:41.365 INFO:teuthology.orchestra.run.smithi097.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 477 kB/s | 25 kB 00:00 2024-09-19T09:15:41.371 INFO:teuthology.orchestra.run.smithi016.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 50 kB/s | 6.6 kB 00:00 2024-09-19T09:15:41.699 INFO:teuthology.orchestra.run.smithi097.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 17 MB/s | 11 MB 00:00 2024-09-19T09:15:41.757 INFO:teuthology.orchestra.run.smithi097.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.8 MB/s | 228 kB 00:00 2024-09-19T09:15:41.824 INFO:teuthology.orchestra.run.smithi097.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 407 kB/s | 27 kB 00:00 2024-09-19T09:15:41.916 INFO:teuthology.orchestra.run.smithi097.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 346 kB/s | 32 kB 00:00 2024-09-19T09:15:42.016 INFO:teuthology.orchestra.run.smithi097.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 7.4 MB/s | 751 kB 00:00 2024-09-19T09:15:42.180 INFO:teuthology.orchestra.run.smithi016.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 1.1 MB/s | 896 kB 00:00 2024-09-19T09:15:42.192 INFO:teuthology.orchestra.run.smithi097.stdout:(15/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 13 MB/s | 17 MB 00:01 2024-09-19T09:15:42.217 INFO:teuthology.orchestra.run.smithi097.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 816 kB/s | 163 kB 00:00 2024-09-19T09:15:42.242 INFO:teuthology.orchestra.run.smithi097.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T09:15:42.263 INFO:teuthology.orchestra.run.smithi016.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.7 MB/s | 145 kB 00:00 2024-09-19T09:15:42.267 INFO:teuthology.orchestra.run.smithi097.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 9.8 MB/s | 506 kB 00:00 2024-09-19T09:15:42.410 INFO:teuthology.orchestra.run.smithi097.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 318 kB/s | 45 kB 00:00 2024-09-19T09:15:42.513 INFO:teuthology.orchestra.run.smithi097.stdout:(20/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 20 MB/s | 5.3 MB 00:00 2024-09-19T09:15:42.538 INFO:teuthology.orchestra.run.smithi097.stdout:(21/156): python3-ceph-common-19.1.1-361.g2f599 1.0 MB/s | 136 kB 00:00 2024-09-19T09:15:42.563 INFO:teuthology.orchestra.run.smithi097.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 3.2 MB/s | 161 kB 00:00 2024-09-19T09:15:42.568 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T09:15:42.569 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T09:15:42.580 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T09:15:42.580 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T09:15:42.581 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T09:15:42.582 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T09:15:42.584 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T09:15:42.586 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T09:15:42.587 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T09:15:42.589 INFO:teuthology.orchestra.run.smithi097.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 6.2 MB/s | 320 kB 00:00 2024-09-19T09:15:42.614 INFO:teuthology.orchestra.run.smithi097.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.8 MB/s | 300 kB 00:00 2024-09-19T09:15:42.639 INFO:teuthology.orchestra.run.smithi097.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T09:15:42.648 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-09-19T09:15:42.665 INFO:teuthology.orchestra.run.smithi097.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T09:15:42.684 INFO:teuthology.orchestra.run.smithi142.stdout:====================================================================================== 2024-09-19T09:15:42.684 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-09-19T09:15:42.684 INFO:teuthology.orchestra.run.smithi142.stdout:====================================================================================== 2024-09-19T09:15:42.684 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-09-19T09:15:42.685 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T09:15:42.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T09:15:42.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T09:15:42.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T09:15:42.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T09:15:42.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T09:15:42.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T09:15:42.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T09:15:42.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T09:15:42.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T09:15:42.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T09:15:42.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T09:15:42.685 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T09:15:42.686 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T09:15:42.686 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T09:15:42.686 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T09:15:42.686 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T09:15:42.686 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T09:15:42.686 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T09:15:42.686 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T09:15:42.686 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T09:15:42.686 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T09:15:42.686 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T09:15:42.686 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T09:15:42.686 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T09:15:42.687 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T09:15:42.687 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T09:15:42.687 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T09:15:42.687 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T09:15:42.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T09:15:42.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T09:15:42.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T09:15:42.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T09:15:42.687 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T09:15:42.687 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T09:15:42.687 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T09:15:42.687 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T09:15:42.688 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-09-19T09:15:42.688 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T09:15:42.688 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T09:15:42.688 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T09:15:42.688 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T09:15:42.688 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T09:15:42.688 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T09:15:42.688 INFO:teuthology.orchestra.run.smithi142.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T09:15:42.688 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T09:15:42.688 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-09-19T09:15:42.688 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T09:15:42.688 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T09:15:42.688 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T09:15:42.689 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T09:15:42.689 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T09:15:42.689 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T09:15:42.689 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T09:15:42.689 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T09:15:42.689 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T09:15:42.689 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T09:15:42.689 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T09:15:42.689 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T09:15:42.689 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T09:15:42.689 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T09:15:42.689 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T09:15:42.690 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T09:15:42.690 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T09:15:42.690 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T09:15:42.690 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T09:15:42.690 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T09:15:42.691 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T09:15:42.691 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T09:15:42.691 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T09:15:42.691 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T09:15:42.691 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T09:15:42.691 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T09:15:42.691 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T09:15:42.691 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T09:15:42.691 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T09:15:42.691 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T09:15:42.691 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T09:15:42.691 INFO:teuthology.orchestra.run.smithi142.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T09:15:42.692 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T09:15:42.692 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T09:15:42.692 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T09:15:42.692 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T09:15:42.692 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T09:15:42.692 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T09:15:42.692 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T09:15:42.692 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T09:15:42.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T09:15:42.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T09:15:42.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T09:15:42.692 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T09:15:42.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T09:15:42.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T09:15:42.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T09:15:42.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T09:15:42.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T09:15:42.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T09:15:42.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T09:15:42.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T09:15:42.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T09:15:42.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T09:15:42.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T09:15:42.693 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T09:15:42.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T09:15:42.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T09:15:42.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T09:15:42.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T09:15:42.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T09:15:42.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T09:15:42.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T09:15:42.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T09:15:42.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T09:15:42.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T09:15:42.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T09:15:42.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T09:15:42.694 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T09:15:42.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T09:15:42.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T09:15:42.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T09:15:42.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T09:15:42.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T09:15:42.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T09:15:42.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T09:15:42.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T09:15:42.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T09:15:42.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T09:15:42.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T09:15:42.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T09:15:42.695 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T09:15:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T09:15:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T09:15:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T09:15:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T09:15:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T09:15:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T09:15:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T09:15:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T09:15:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T09:15:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T09:15:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T09:15:42.696 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T09:15:42.697 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T09:15:42.697 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T09:15:42.697 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T09:15:42.697 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T09:15:42.697 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T09:15:42.697 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T09:15:42.697 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T09:15:42.697 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T09:15:42.697 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T09:15:42.697 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T09:15:42.697 INFO:teuthology.orchestra.run.smithi142.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T09:15:42.698 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T09:15:42.698 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T09:15:42.698 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T09:15:42.698 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T09:15:42.698 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T09:15:42.698 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-09-19T09:15:42.699 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T09:15:42.699 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:15:42.699 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-09-19T09:15:42.699 INFO:teuthology.orchestra.run.smithi142.stdout:====================================================================================== 2024-09-19T09:15:42.699 INFO:teuthology.orchestra.run.smithi142.stdout:Install 148 Packages 2024-09-19T09:15:42.699 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 8 Packages 2024-09-19T09:15:42.699 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:15:42.700 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 226 M 2024-09-19T09:15:42.700 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-09-19T09:15:42.732 INFO:teuthology.orchestra.run.smithi097.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-19T09:15:42.791 INFO:teuthology.orchestra.run.smithi097.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 21 MB/s | 3.1 MB 00:00 2024-09-19T09:15:42.816 INFO:teuthology.orchestra.run.smithi097.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 381 kB/s | 32 kB 00:00 2024-09-19T09:15:42.858 INFO:teuthology.orchestra.run.smithi097.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.1 MB/s | 146 kB 00:00 2024-09-19T09:15:43.108 INFO:teuthology.orchestra.run.smithi097.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 12 MB/s | 3.6 MB 00:00 2024-09-19T09:15:43.192 INFO:teuthology.orchestra.run.smithi097.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.9 MB/s | 249 kB 00:00 2024-09-19T09:15:43.267 INFO:teuthology.orchestra.run.smithi097.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 660 kB/s | 49 kB 00:00 2024-09-19T09:15:43.376 INFO:teuthology.orchestra.run.smithi097.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 14 MB/s | 7.4 MB 00:00 2024-09-19T09:15:43.392 INFO:teuthology.orchestra.run.smithi097.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 131 kB/s | 16 kB 00:00 2024-09-19T09:15:43.426 INFO:teuthology.orchestra.run.smithi142.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 42 kB/s | 6.6 kB 00:00 2024-09-19T09:15:43.426 INFO:teuthology.orchestra.run.smithi097.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.6 MB/s | 285 kB 00:00 2024-09-19T09:15:43.468 INFO:teuthology.orchestra.run.smithi097.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 9.9 MB/s | 765 kB 00:00 2024-09-19T09:15:43.485 INFO:teuthology.orchestra.run.smithi097.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 311 kB/s | 18 kB 00:00 2024-09-19T09:15:43.547 INFO:teuthology.orchestra.run.smithi016.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 1.9 MB/s | 2.4 MB 00:01 2024-09-19T09:15:43.685 INFO:teuthology.orchestra.run.smithi142.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.4 MB/s | 896 kB 00:00 2024-09-19T09:15:43.793 INFO:teuthology.orchestra.run.smithi142.stdout:(3/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-19T09:15:43.818 INFO:teuthology.orchestra.run.smithi142.stdout:(4/156): ceph-immutable-object-cache-19.1.1-361 1.1 MB/s | 145 kB 00:00 2024-09-19T09:15:43.944 INFO:teuthology.orchestra.run.smithi097.stdout:(39/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 19 MB/s | 49 MB 00:02 2024-09-19T09:15:43.952 INFO:teuthology.orchestra.run.smithi142.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 15 MB/s | 2.4 MB 00:00 2024-09-19T09:15:44.011 INFO:teuthology.orchestra.run.smithi142.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 7.9 MB/s | 1.5 MB 00:00 2024-09-19T09:15:44.052 INFO:teuthology.orchestra.run.smithi097.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 137 kB/s | 80 kB 00:00 2024-09-19T09:15:44.144 INFO:teuthology.orchestra.run.smithi097.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-19T09:15:44.170 INFO:teuthology.orchestra.run.smithi097.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 849 kB/s | 191 kB 00:00 2024-09-19T09:15:44.220 INFO:teuthology.orchestra.run.smithi097.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-19T09:15:44.244 INFO:teuthology.orchestra.run.smithi142.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-19T09:15:44.245 INFO:teuthology.orchestra.run.smithi097.stdout:(44/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 801 kB 00:00 2024-09-19T09:15:44.313 INFO:teuthology.orchestra.run.smithi097.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-19T09:15:44.363 INFO:teuthology.orchestra.run.smithi097.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-19T09:15:44.405 INFO:teuthology.orchestra.run.smithi097.stdout:(47/156): python3-pycparser-2.20-6.el9.noarch.r 735 kB/s | 135 kB 00:00 2024-09-19T09:15:44.438 INFO:teuthology.orchestra.run.smithi097.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-09-19T09:15:44.580 INFO:teuthology.orchestra.run.smithi097.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 754 kB/s | 107 kB 00:00 2024-09-19T09:15:44.614 INFO:teuthology.orchestra.run.smithi016.stdout:(5/156): ceph-common-19.1.1-361.g2f599a46.el9.x 6.3 MB/s | 21 MB 00:03 2024-09-19T09:15:44.614 INFO:teuthology.orchestra.run.smithi097.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 689 kB 00:00 2024-09-19T09:15:44.636 INFO:teuthology.orchestra.run.smithi142.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 16 MB/s | 21 MB 00:01 2024-09-19T09:15:44.639 INFO:teuthology.orchestra.run.smithi097.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-19T09:15:44.664 INFO:teuthology.orchestra.run.smithi016.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 1.4 MB/s | 1.5 MB 00:01 2024-09-19T09:15:44.681 INFO:teuthology.orchestra.run.smithi097.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-09-19T09:15:44.686 INFO:teuthology.orchestra.run.smithi142.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 507 kB/s | 25 kB 00:00 2024-09-19T09:15:44.707 INFO:teuthology.orchestra.run.smithi097.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-09-19T09:15:44.731 INFO:teuthology.orchestra.run.smithi016.stdout:(7/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 1.5 MB/s | 5.4 MB 00:03 2024-09-19T09:15:44.732 INFO:teuthology.orchestra.run.smithi097.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-19T09:15:44.757 INFO:teuthology.orchestra.run.smithi097.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-19T09:15:44.782 INFO:teuthology.orchestra.run.smithi097.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-19T09:15:44.808 INFO:teuthology.orchestra.run.smithi097.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-19T09:15:44.833 INFO:teuthology.orchestra.run.smithi097.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-09-19T09:15:44.892 INFO:teuthology.orchestra.run.smithi097.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-09-19T09:15:44.917 INFO:teuthology.orchestra.run.smithi097.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 662 kB 00:00 2024-09-19T09:15:44.920 INFO:teuthology.orchestra.run.smithi142.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 16 MB/s | 11 MB 00:00 2024-09-19T09:15:44.942 INFO:teuthology.orchestra.run.smithi097.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 490 kB/s | 24 kB 00:00 2024-09-19T09:15:44.967 INFO:teuthology.orchestra.run.smithi097.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-19T09:15:44.978 INFO:teuthology.orchestra.run.smithi142.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.8 MB/s | 228 kB 00:00 2024-09-19T09:15:45.022 INFO:teuthology.orchestra.run.smithi097.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.6 MB/s | 292 kB 00:00 2024-09-19T09:15:45.052 INFO:teuthology.orchestra.run.smithi097.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.6 MB/s | 581 kB 00:00 2024-09-19T09:15:45.054 INFO:teuthology.orchestra.run.smithi142.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 361 kB/s | 27 kB 00:00 2024-09-19T09:15:45.077 INFO:teuthology.orchestra.run.smithi097.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 711 kB/s | 38 kB 00:00 2024-09-19T09:15:45.102 INFO:teuthology.orchestra.run.smithi097.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-19T09:15:45.129 INFO:teuthology.orchestra.run.smithi142.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 424 kB/s | 32 kB 00:00 2024-09-19T09:15:45.186 INFO:teuthology.orchestra.run.smithi097.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-19T09:15:45.190 INFO:teuthology.orchestra.run.smithi016.stdout:(8/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 8.2 MB/s | 4.7 MB 00:00 2024-09-19T09:15:45.213 INFO:teuthology.orchestra.run.smithi142.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 8.8 MB/s | 751 kB 00:00 2024-09-19T09:15:45.228 INFO:teuthology.orchestra.run.smithi097.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-19T09:15:45.257 INFO:teuthology.orchestra.run.smithi016.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 379 kB/s | 25 kB 00:00 2024-09-19T09:15:45.270 INFO:teuthology.orchestra.run.smithi097.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-19T09:15:45.279 INFO:teuthology.orchestra.run.smithi142.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.4 MB/s | 163 kB 00:00 2024-09-19T09:15:45.295 INFO:teuthology.orchestra.run.smithi097.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-19T09:15:45.354 INFO:teuthology.orchestra.run.smithi142.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 1.7 MB/s | 127 kB 00:00 2024-09-19T09:15:45.362 INFO:teuthology.orchestra.run.smithi097.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 16 MB/s | 4.9 MB 00:00 2024-09-19T09:15:45.412 INFO:teuthology.orchestra.run.smithi097.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-09-19T09:15:45.438 INFO:teuthology.orchestra.run.smithi097.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-19T09:15:45.439 INFO:teuthology.orchestra.run.smithi142.stdout:(17/156): libradosstriper1-19.1.1-361.g2f599a46 5.9 MB/s | 506 kB 00:00 2024-09-19T09:15:45.530 INFO:teuthology.orchestra.run.smithi097.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-19T09:15:45.546 INFO:teuthology.orchestra.run.smithi097.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 565 kB/s | 61 kB 00:00 2024-09-19T09:15:45.580 INFO:teuthology.orchestra.run.smithi097.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-19T09:15:45.605 INFO:teuthology.orchestra.run.smithi142.stdout:(18/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 17 MB 00:01 2024-09-19T09:15:45.605 INFO:teuthology.orchestra.run.smithi097.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-19T09:15:45.622 INFO:teuthology.orchestra.run.smithi097.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-09-19T09:15:45.639 INFO:teuthology.orchestra.run.smithi097.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 573 kB/s | 19 kB 00:00 2024-09-19T09:15:45.664 INFO:teuthology.orchestra.run.smithi142.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 776 kB/s | 45 kB 00:00 2024-09-19T09:15:45.673 INFO:teuthology.orchestra.run.smithi097.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-19T09:15:45.690 INFO:teuthology.orchestra.run.smithi097.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-09-19T09:15:45.747 INFO:teuthology.orchestra.run.smithi142.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.6 MB/s | 136 kB 00:00 2024-09-19T09:15:45.765 INFO:teuthology.orchestra.run.smithi097.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-19T09:15:45.790 INFO:teuthology.orchestra.run.smithi097.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-19T09:15:45.816 INFO:teuthology.orchestra.run.smithi097.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-19T09:15:45.824 INFO:teuthology.orchestra.run.smithi142.stdout:(21/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-19T09:15:45.841 INFO:teuthology.orchestra.run.smithi097.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T09:15:45.848 INFO:teuthology.orchestra.run.smithi142.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.6 MB/s | 161 kB 00:00 2024-09-19T09:15:45.866 INFO:teuthology.orchestra.run.smithi097.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-19T09:15:45.882 INFO:teuthology.orchestra.run.smithi142.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 5.3 MB/s | 320 kB 00:00 2024-09-19T09:15:45.891 INFO:teuthology.orchestra.run.smithi097.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-19T09:15:45.907 INFO:teuthology.orchestra.run.smithi142.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.0 MB/s | 300 kB 00:00 2024-09-19T09:15:45.917 INFO:teuthology.orchestra.run.smithi097.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-19T09:15:45.942 INFO:teuthology.orchestra.run.smithi097.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-19T09:15:45.949 INFO:teuthology.orchestra.run.smithi142.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.5 MB/s | 99 kB 00:00 2024-09-19T09:15:45.968 INFO:teuthology.orchestra.run.smithi097.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T09:15:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-19T09:15:45.993 INFO:teuthology.orchestra.run.smithi097.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T09:15:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.0 MB/s | 171 kB 00:00 2024-09-19T09:15:46.085 INFO:teuthology.orchestra.run.smithi097.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-19T09:15:46.110 INFO:teuthology.orchestra.run.smithi097.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 356 kB/s | 42 kB 00:00 2024-09-19T09:15:46.135 INFO:teuthology.orchestra.run.smithi097.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-09-19T09:15:46.141 INFO:teuthology.orchestra.run.smithi142.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 381 kB/s | 32 kB 00:00 2024-09-19T09:15:46.160 INFO:teuthology.orchestra.run.smithi097.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-19T09:15:46.186 INFO:teuthology.orchestra.run.smithi097.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-19T09:15:46.208 INFO:teuthology.orchestra.run.smithi142.stdout:(29/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-19T09:15:46.211 INFO:teuthology.orchestra.run.smithi097.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-19T09:15:46.225 INFO:teuthology.orchestra.run.smithi142.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.7 MB/s | 146 kB 00:00 2024-09-19T09:15:46.253 INFO:teuthology.orchestra.run.smithi097.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-19T09:15:46.279 INFO:teuthology.orchestra.run.smithi097.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 863 kB/s | 58 kB 00:00 2024-09-19T09:15:46.304 INFO:teuthology.orchestra.run.smithi097.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T09:15:46.329 INFO:teuthology.orchestra.run.smithi097.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T09:15:46.354 INFO:teuthology.orchestra.run.smithi097.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-19T09:15:46.430 INFO:teuthology.orchestra.run.smithi097.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-19T09:15:46.509 INFO:teuthology.orchestra.run.smithi142.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 12 MB/s | 3.6 MB 00:00 2024-09-19T09:15:46.597 INFO:teuthology.orchestra.run.smithi097.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-19T09:15:46.655 INFO:teuthology.orchestra.run.smithi097.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.4 MB/s | 548 kB 00:00 2024-09-19T09:15:46.689 INFO:teuthology.orchestra.run.smithi097.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-09-19T09:15:46.714 INFO:teuthology.orchestra.run.smithi097.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.6 MB/s | 308 kB 00:00 2024-09-19T09:15:46.731 INFO:teuthology.orchestra.run.smithi097.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-09-19T09:15:46.757 INFO:teuthology.orchestra.run.smithi097.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.7 MB/s | 49 kB 00:00 2024-09-19T09:15:46.791 INFO:teuthology.orchestra.run.smithi097.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.1 MB/s | 67 kB 00:00 2024-09-19T09:15:46.807 INFO:teuthology.orchestra.run.smithi097.stdout:(111/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 281 kB/s | 26 kB 00:00 2024-09-19T09:15:46.832 INFO:teuthology.orchestra.run.smithi097.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-09-19T09:15:46.891 INFO:teuthology.orchestra.run.smithi097.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-19T09:15:46.908 INFO:teuthology.orchestra.run.smithi097.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-19T09:15:46.918 INFO:teuthology.orchestra.run.smithi142.stdout:(32/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 22 MB/s | 49 MB 00:02 2024-09-19T09:15:46.933 INFO:teuthology.orchestra.run.smithi097.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T09:15:46.943 INFO:teuthology.orchestra.run.smithi142.stdout:(33/156): ceph-mgr-modules-core-19.1.1-361.g2f5 575 kB/s | 249 kB 00:00 2024-09-19T09:15:46.950 INFO:teuthology.orchestra.run.smithi097.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 855 kB/s | 14 kB 00:00 2024-09-19T09:15:46.968 INFO:teuthology.orchestra.run.smithi142.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 991 kB/s | 49 kB 00:00 2024-09-19T09:15:46.975 INFO:teuthology.orchestra.run.smithi097.stdout:(117/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-09-19T09:15:46.997 INFO:teuthology.orchestra.run.smithi142.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 328 kB/s | 16 kB 00:00 2024-09-19T09:15:47.009 INFO:teuthology.orchestra.run.smithi097.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-19T09:15:47.019 INFO:teuthology.orchestra.run.smithi142.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.5 MB/s | 285 kB 00:00 2024-09-19T09:15:47.051 INFO:teuthology.orchestra.run.smithi097.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-19T09:15:47.069 INFO:teuthology.orchestra.run.smithi142.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 366 kB/s | 18 kB 00:00 2024-09-19T09:15:47.076 INFO:teuthology.orchestra.run.smithi097.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-19T09:15:47.095 INFO:teuthology.orchestra.run.smithi142.stdout:(38/156): cephadm-19.1.1-361.g2f599a46.el9.noar 7.4 MB/s | 765 kB 00:00 2024-09-19T09:15:47.106 INFO:teuthology.orchestra.run.smithi097.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T09:15:47.135 INFO:teuthology.orchestra.run.smithi097.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-19T09:15:47.186 INFO:teuthology.orchestra.run.smithi142.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 7.7 MB/s | 7.4 MB 00:00 2024-09-19T09:15:47.194 INFO:teuthology.orchestra.run.smithi097.stdout:(123/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.2 MB/s | 4.4 MB 00:00 2024-09-19T09:15:47.219 INFO:teuthology.orchestra.run.smithi097.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-09-19T09:15:47.244 INFO:teuthology.orchestra.run.smithi097.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 935 kB/s | 23 kB 00:00 2024-09-19T09:15:47.269 INFO:teuthology.orchestra.run.smithi097.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-19T09:15:47.295 INFO:teuthology.orchestra.run.smithi097.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-09-19T09:15:47.321 INFO:teuthology.orchestra.run.smithi097.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-19T09:15:47.347 INFO:teuthology.orchestra.run.smithi097.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T09:15:47.371 INFO:teuthology.orchestra.run.smithi097.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T09:15:47.396 INFO:teuthology.orchestra.run.smithi097.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T09:15:47.430 INFO:teuthology.orchestra.run.smithi097.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-19T09:15:47.455 INFO:teuthology.orchestra.run.smithi097.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-19T09:15:47.481 INFO:teuthology.orchestra.run.smithi097.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-19T09:15:47.514 INFO:teuthology.orchestra.run.smithi097.stdout:(135/156): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-09-19T09:15:47.539 INFO:teuthology.orchestra.run.smithi097.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 483 kB/s | 28 kB 00:00 2024-09-19T09:15:47.549 INFO:teuthology.orchestra.run.smithi016.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 3.8 MB/s | 11 MB 00:02 2024-09-19T09:15:47.565 INFO:teuthology.orchestra.run.smithi097.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-19T09:15:47.615 INFO:teuthology.orchestra.run.smithi097.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.8 MB/s | 2.0 MB 00:00 2024-09-19T09:15:47.615 INFO:teuthology.orchestra.run.smithi016.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.4 MB/s | 228 kB 00:00 2024-09-19T09:15:47.640 INFO:teuthology.orchestra.run.smithi097.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-09-19T09:15:47.657 INFO:teuthology.orchestra.run.smithi016.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 651 kB/s | 27 kB 00:00 2024-09-19T09:15:47.666 INFO:teuthology.orchestra.run.smithi097.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-19T09:15:47.691 INFO:teuthology.orchestra.run.smithi097.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-19T09:15:47.708 INFO:teuthology.orchestra.run.smithi016.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 634 kB/s | 32 kB 00:00 2024-09-19T09:15:47.716 INFO:teuthology.orchestra.run.smithi097.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-19T09:15:47.737 INFO:teuthology.orchestra.run.smithi142.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 120 kB/s | 80 kB 00:00 2024-09-19T09:15:47.742 INFO:teuthology.orchestra.run.smithi097.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T09:15:47.758 INFO:teuthology.orchestra.run.smithi097.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-19T09:15:47.784 INFO:teuthology.orchestra.run.smithi097.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-19T09:15:47.809 INFO:teuthology.orchestra.run.smithi097.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T09:15:47.835 INFO:teuthology.orchestra.run.smithi097.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-19T09:15:47.850 INFO:teuthology.orchestra.run.smithi016.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 5.2 MB/s | 751 kB 00:00 2024-09-19T09:15:47.885 INFO:teuthology.orchestra.run.smithi097.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T09:15:47.912 INFO:teuthology.orchestra.run.smithi142.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 264 kB/s | 191 kB 00:00 2024-09-19T09:15:47.917 INFO:teuthology.orchestra.run.smithi016.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.4 MB/s | 163 kB 00:00 2024-09-19T09:15:47.967 INFO:teuthology.orchestra.run.smithi016.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T09:15:48.012 INFO:teuthology.orchestra.run.smithi142.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 874 kB/s | 801 kB 00:00 2024-09-19T09:15:48.068 INFO:teuthology.orchestra.run.smithi097.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-19T09:15:48.101 INFO:teuthology.orchestra.run.smithi016.stdout:(17/156): libradosstriper1-19.1.1-361.g2f599a46 3.7 MB/s | 506 kB 00:00 2024-09-19T09:15:48.310 INFO:teuthology.orchestra.run.smithi097.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 5.6 MB/s | 2.3 MB 00:00 2024-09-19T09:15:48.352 INFO:teuthology.orchestra.run.smithi097.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 987 kB/s | 41 kB 00:00 2024-09-19T09:15:48.431 INFO:teuthology.orchestra.run.smithi097.stdout:(152/156): NetworkManager-libnm-1.51.0-1.el9.x8 5.0 MB/s | 1.8 MB 00:00 2024-09-19T09:15:48.453 INFO:teuthology.orchestra.run.smithi097.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.4 MB/s | 249 kB 00:00 2024-09-19T09:15:48.479 INFO:teuthology.orchestra.run.smithi142.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 341 kB/s | 253 kB 00:00 2024-09-19T09:15:48.653 INFO:teuthology.orchestra.run.smithi097.stdout:(154/156): librbd1-19.1.1-361.g2f599a46.el9.x86 3.9 MB/s | 3.2 MB 00:00 2024-09-19T09:15:48.679 INFO:teuthology.orchestra.run.smithi142.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 160 kB/s | 106 kB 00:00 2024-09-19T09:15:48.684 INFO:teuthology.orchestra.run.smithi016.stdout:(18/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 14 MB/s | 49 MB 00:03 2024-09-19T09:15:48.732 INFO:teuthology.orchestra.run.smithi097.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.4 MB/s | 1.4 MB 00:00 2024-09-19T09:15:48.860 INFO:teuthology.orchestra.run.smithi016.stdout:(19/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 4.0 MB/s | 17 MB 00:04 2024-09-19T09:15:48.871 INFO:teuthology.orchestra.run.smithi142.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-19T09:15:48.885 INFO:teuthology.orchestra.run.smithi016.stdout:(20/156): python3-ceph-argparse-19.1.1-361.g2f5 226 kB/s | 45 kB 00:00 2024-09-19T09:15:48.927 INFO:teuthology.orchestra.run.smithi016.stdout:(21/156): python3-ceph-common-19.1.1-361.g2f599 2.0 MB/s | 136 kB 00:00 2024-09-19T09:15:48.952 INFO:teuthology.orchestra.run.smithi016.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.4 MB/s | 161 kB 00:00 2024-09-19T09:15:48.953 INFO:teuthology.orchestra.run.smithi097.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.8 MB/s | 2.4 MB 00:00 2024-09-19T09:15:48.960 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-19T09:15:48.960 INFO:teuthology.orchestra.run.smithi097.stdout:Total 24 MB/s | 226 MB 00:09 2024-09-19T09:15:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 3.7 MB/s | 320 kB 00:00 2024-09-19T09:15:49.036 INFO:teuthology.orchestra.run.smithi016.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.5 MB/s | 300 kB 00:00 2024-09-19T09:15:49.061 INFO:teuthology.orchestra.run.smithi016.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T09:15:49.087 INFO:teuthology.orchestra.run.smithi016.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T09:15:49.137 INFO:teuthology.orchestra.run.smithi016.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-19T09:15:49.163 INFO:teuthology.orchestra.run.smithi142.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 198 kB/s | 135 kB 00:00 2024-09-19T09:15:49.187 INFO:teuthology.orchestra.run.smithi016.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 638 kB/s | 32 kB 00:00 2024-09-19T09:15:49.238 INFO:teuthology.orchestra.run.smithi016.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T09:15:49.305 INFO:teuthology.orchestra.run.smithi016.stdout:(30/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 4.4 MB/s | 5.3 MB 00:01 2024-09-19T09:15:49.371 INFO:teuthology.orchestra.run.smithi142.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 180 kB/s | 125 kB 00:00 2024-09-19T09:15:49.430 INFO:teuthology.orchestra.run.smithi142.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-09-19T09:15:49.480 INFO:teuthology.orchestra.run.smithi142.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-19T09:15:49.505 INFO:teuthology.orchestra.run.smithi016.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 13 MB/s | 3.6 MB 00:00 2024-09-19T09:15:49.531 INFO:teuthology.orchestra.run.smithi142.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-19T09:15:49.572 INFO:teuthology.orchestra.run.smithi016.stdout:(32/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 6.1 MB/s | 3.1 MB 00:00 2024-09-19T09:15:49.589 INFO:teuthology.orchestra.run.smithi142.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-09-19T09:15:49.597 INFO:teuthology.orchestra.run.smithi016.stdout:(33/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.7 MB/s | 249 kB 00:00 2024-09-19T09:15:49.623 INFO:teuthology.orchestra.run.smithi016.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 980 kB/s | 49 kB 00:00 2024-09-19T09:15:49.639 INFO:teuthology.orchestra.run.smithi142.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-09-19T09:15:49.648 INFO:teuthology.orchestra.run.smithi016.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 325 kB/s | 16 kB 00:00 2024-09-19T09:15:49.665 INFO:teuthology.orchestra.run.smithi142.stdout:(53/156): python3-urllib3-1.26.5-6.el9.noarch.r 271 kB/s | 215 kB 00:00 2024-09-19T09:15:49.698 INFO:teuthology.orchestra.run.smithi016.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.7 MB/s | 285 kB 00:00 2024-09-19T09:15:49.715 INFO:teuthology.orchestra.run.smithi142.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-09-19T09:15:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 440 kB/s | 18 kB 00:00 2024-09-19T09:15:49.757 INFO:teuthology.orchestra.run.smithi142.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-09-19T09:15:49.766 INFO:teuthology.orchestra.run.smithi016.stdout:(38/156): cephadm-19.1.1-361.g2f599a46.el9.noar 6.4 MB/s | 765 kB 00:00 2024-09-19T09:15:49.791 INFO:teuthology.orchestra.run.smithi142.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 145 kB/s | 18 kB 00:00 2024-09-19T09:15:49.825 INFO:teuthology.orchestra.run.smithi142.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-19T09:15:49.866 INFO:teuthology.orchestra.run.smithi142.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-09-19T09:15:49.883 INFO:teuthology.orchestra.run.smithi016.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-09-19T09:15:49.900 INFO:teuthology.orchestra.run.smithi142.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-19T09:15:49.942 INFO:teuthology.orchestra.run.smithi142.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 590 kB/s | 24 kB 00:00 2024-09-19T09:15:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-19T09:15:50.009 INFO:teuthology.orchestra.run.smithi142.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 713 kB/s | 47 kB 00:00 2024-09-19T09:15:50.034 INFO:teuthology.orchestra.run.smithi142.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 3.4 MB/s | 581 kB 00:00 2024-09-19T09:15:50.058 INFO:teuthology.orchestra.run.smithi016.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-19T09:15:50.084 INFO:teuthology.orchestra.run.smithi016.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-19T09:15:50.092 INFO:teuthology.orchestra.run.smithi142.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-19T09:15:50.134 INFO:teuthology.orchestra.run.smithi016.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-19T09:15:50.151 INFO:teuthology.orchestra.run.smithi142.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 660 kB/s | 38 kB 00:00 2024-09-19T09:15:50.188 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-19T09:15:50.209 INFO:teuthology.orchestra.run.smithi016.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-19T09:15:50.243 INFO:teuthology.orchestra.run.smithi142.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 292 kB 00:00 2024-09-19T09:15:50.268 INFO:teuthology.orchestra.run.smithi016.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-19T09:15:50.302 INFO:teuthology.orchestra.run.smithi142.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.5 MB/s | 3.0 MB 00:00 2024-09-19T09:15:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-19T09:15:50.336 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-19T09:15:50.351 INFO:teuthology.orchestra.run.smithi016.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.2 MB 00:00 2024-09-19T09:15:50.435 INFO:teuthology.orchestra.run.smithi016.stdout:(47/156): ceph-mgr-diskprediction-local-19.1.1- 6.5 MB/s | 7.4 MB 00:01 2024-09-19T09:15:50.461 INFO:teuthology.orchestra.run.smithi016.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.1 MB/s | 215 kB 00:00 2024-09-19T09:15:50.594 INFO:teuthology.orchestra.run.smithi016.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 802 kB/s | 107 kB 00:00 2024-09-19T09:15:50.644 INFO:teuthology.orchestra.run.smithi016.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-19T09:15:50.670 INFO:teuthology.orchestra.run.smithi016.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 289 kB/s | 22 kB 00:00 2024-09-19T09:15:50.703 INFO:teuthology.orchestra.run.smithi016.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-09-19T09:15:50.729 INFO:teuthology.orchestra.run.smithi016.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-19T09:15:50.762 INFO:teuthology.orchestra.run.smithi016.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-19T09:15:50.788 INFO:teuthology.orchestra.run.smithi016.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-09-19T09:15:50.813 INFO:teuthology.orchestra.run.smithi016.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-19T09:15:50.846 INFO:teuthology.orchestra.run.smithi016.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-19T09:15:50.872 INFO:teuthology.orchestra.run.smithi016.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-09-19T09:15:50.914 INFO:teuthology.orchestra.run.smithi016.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-19T09:15:50.964 INFO:teuthology.orchestra.run.smithi016.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-19T09:15:50.990 INFO:teuthology.orchestra.run.smithi016.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 324 kB/s | 24 kB 00:00 2024-09-19T09:15:51.015 INFO:teuthology.orchestra.run.smithi016.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 940 kB/s | 47 kB 00:00 2024-09-19T09:15:51.040 INFO:teuthology.orchestra.run.smithi016.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-19T09:15:51.066 INFO:teuthology.orchestra.run.smithi016.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.9 MB/s | 581 kB 00:00 2024-09-19T09:15:51.091 INFO:teuthology.orchestra.run.smithi016.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-19T09:15:51.110 INFO:teuthology.orchestra.run.smithi142.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-09-19T09:15:51.116 INFO:teuthology.orchestra.run.smithi016.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-19T09:15:51.191 INFO:teuthology.orchestra.run.smithi016.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-19T09:15:51.267 INFO:teuthology.orchestra.run.smithi016.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-19T09:15:51.301 INFO:teuthology.orchestra.run.smithi016.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.2 MB/s | 910 kB 00:00 2024-09-19T09:15:51.326 INFO:teuthology.orchestra.run.smithi016.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-19T09:15:51.384 INFO:teuthology.orchestra.run.smithi016.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-19T09:15:51.460 INFO:teuthology.orchestra.run.smithi016.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-09-19T09:15:51.485 INFO:teuthology.orchestra.run.smithi016.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 875 kB/s | 87 kB 00:00 2024-09-19T09:15:51.527 INFO:teuthology.orchestra.run.smithi016.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-19T09:15:51.585 INFO:teuthology.orchestra.run.smithi142.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 1.9 MB/s | 910 kB 00:00 2024-09-19T09:15:51.611 INFO:teuthology.orchestra.run.smithi016.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-09-19T09:15:51.636 INFO:teuthology.orchestra.run.smithi016.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-19T09:15:51.669 INFO:teuthology.orchestra.run.smithi016.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-19T09:15:51.694 INFO:teuthology.orchestra.run.smithi142.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-19T09:15:51.695 INFO:teuthology.orchestra.run.smithi016.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-09-19T09:15:51.720 INFO:teuthology.orchestra.run.smithi016.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-19T09:15:51.745 INFO:teuthology.orchestra.run.smithi016.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-19T09:15:51.771 INFO:teuthology.orchestra.run.smithi016.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-19T09:15:51.838 INFO:teuthology.orchestra.run.smithi016.stdout:(82/156): openblas-openmp-0.3.26-2.el9.x86_64.r 6.4 MB/s | 4.9 MB 00:00 2024-09-19T09:15:51.844 INFO:teuthology.orchestra.run.smithi142.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.8 MB/s | 4.4 MB 00:01 2024-09-19T09:15:51.864 INFO:teuthology.orchestra.run.smithi016.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-19T09:15:51.913 INFO:teuthology.orchestra.run.smithi016.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-09-19T09:15:51.939 INFO:teuthology.orchestra.run.smithi016.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-09-19T09:15:51.964 INFO:teuthology.orchestra.run.smithi016.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-19T09:15:51.989 INFO:teuthology.orchestra.run.smithi016.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-19T09:15:52.053 INFO:teuthology.orchestra.run.smithi142.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.6 MB/s | 4.9 MB 00:01 2024-09-19T09:15:52.057 INFO:teuthology.orchestra.run.smithi016.stdout:(88/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-19T09:15:52.082 INFO:teuthology.orchestra.run.smithi016.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-19T09:15:52.107 INFO:teuthology.orchestra.run.smithi016.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T09:15:52.133 INFO:teuthology.orchestra.run.smithi016.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T09:15:52.135 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-19T09:15:52.135 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-19T09:15:52.136 INFO:teuthology.orchestra.run.smithi142.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-19T09:15:52.158 INFO:teuthology.orchestra.run.smithi016.stdout:(92/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-09-19T09:15:52.183 INFO:teuthology.orchestra.run.smithi016.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-09-19T09:15:52.208 INFO:teuthology.orchestra.run.smithi016.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 879 kB/s | 44 kB 00:00 2024-09-19T09:15:52.250 INFO:teuthology.orchestra.run.smithi016.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-19T09:15:52.276 INFO:teuthology.orchestra.run.smithi016.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 536 kB/s | 36 kB 00:00 2024-09-19T09:15:52.309 INFO:teuthology.orchestra.run.smithi016.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-09-19T09:15:52.343 INFO:teuthology.orchestra.run.smithi016.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-19T09:15:52.368 INFO:teuthology.orchestra.run.smithi016.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-09-19T09:15:52.393 INFO:teuthology.orchestra.run.smithi016.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T09:15:52.395 INFO:teuthology.orchestra.run.smithi142.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.8 MB/s | 1.0 MB 00:00 2024-09-19T09:15:52.418 INFO:teuthology.orchestra.run.smithi016.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T09:15:52.444 INFO:teuthology.orchestra.run.smithi016.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-19T09:15:52.453 INFO:teuthology.orchestra.run.smithi142.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-19T09:15:52.528 INFO:teuthology.orchestra.run.smithi016.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-19T09:15:52.578 INFO:teuthology.orchestra.run.smithi016.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T09:15:52.595 INFO:teuthology.orchestra.run.smithi142.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-19T09:15:52.737 INFO:teuthology.orchestra.run.smithi016.stdout:(105/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-19T09:15:52.746 INFO:teuthology.orchestra.run.smithi142.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-09-19T09:15:52.778 INFO:teuthology.orchestra.run.smithi016.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-09-19T09:15:52.796 INFO:teuthology.orchestra.run.smithi142.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 515 kB/s | 26 kB 00:00 2024-09-19T09:15:52.812 INFO:teuthology.orchestra.run.smithi016.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 1.3 MB/s | 308 kB 00:00 2024-09-19T09:15:52.837 INFO:teuthology.orchestra.run.smithi016.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-09-19T09:15:52.846 INFO:teuthology.orchestra.run.smithi142.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-19T09:15:52.863 INFO:teuthology.orchestra.run.smithi016.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 536 kB/s | 26 kB 00:00 2024-09-19T09:15:52.948 INFO:teuthology.orchestra.run.smithi016.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 441 kB/s | 49 kB 00:00 2024-09-19T09:15:52.954 INFO:teuthology.orchestra.run.smithi142.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-19T09:15:52.972 INFO:teuthology.orchestra.run.smithi016.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-09-19T09:15:52.997 INFO:teuthology.orchestra.run.smithi016.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-19T09:15:53.005 INFO:teuthology.orchestra.run.smithi142.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-19T09:15:53.039 INFO:teuthology.orchestra.run.smithi016.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-19T09:15:53.064 INFO:teuthology.orchestra.run.smithi016.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T09:15:53.090 INFO:teuthology.orchestra.run.smithi016.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-19T09:15:53.115 INFO:teuthology.orchestra.run.smithi016.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 636 kB/s | 32 kB 00:00 2024-09-19T09:15:53.140 INFO:teuthology.orchestra.run.smithi016.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-19T09:15:53.166 INFO:teuthology.orchestra.run.smithi016.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-19T09:15:53.207 INFO:teuthology.orchestra.run.smithi016.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-19T09:15:53.233 INFO:teuthology.orchestra.run.smithi016.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-19T09:15:53.258 INFO:teuthology.orchestra.run.smithi016.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T09:15:53.292 INFO:teuthology.orchestra.run.smithi016.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-19T09:15:53.317 INFO:teuthology.orchestra.run.smithi016.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-19T09:15:53.343 INFO:teuthology.orchestra.run.smithi016.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-09-19T09:15:53.368 INFO:teuthology.orchestra.run.smithi016.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-19T09:15:53.393 INFO:teuthology.orchestra.run.smithi016.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 782 kB/s | 19 kB 00:00 2024-09-19T09:15:53.419 INFO:teuthology.orchestra.run.smithi016.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-19T09:15:53.477 INFO:teuthology.orchestra.run.smithi016.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-19T09:15:53.503 INFO:teuthology.orchestra.run.smithi016.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T09:15:53.528 INFO:teuthology.orchestra.run.smithi016.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T09:15:53.553 INFO:teuthology.orchestra.run.smithi016.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T09:15:53.587 INFO:teuthology.orchestra.run.smithi016.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-19T09:15:53.612 INFO:teuthology.orchestra.run.smithi016.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 665 kB/s | 16 kB 00:00 2024-09-19T09:15:53.637 INFO:teuthology.orchestra.run.smithi016.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-19T09:15:53.663 INFO:teuthology.orchestra.run.smithi016.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-19T09:15:53.688 INFO:teuthology.orchestra.run.smithi016.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-19T09:15:53.763 INFO:teuthology.orchestra.run.smithi016.stdout:(137/156): libarrow-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 4.4 MB 00:00 2024-09-19T09:15:53.789 INFO:teuthology.orchestra.run.smithi016.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-09-19T09:15:53.814 INFO:teuthology.orchestra.run.smithi016.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-19T09:15:53.839 INFO:teuthology.orchestra.run.smithi016.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-19T09:15:53.864 INFO:teuthology.orchestra.run.smithi016.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-19T09:15:53.889 INFO:teuthology.orchestra.run.smithi016.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 937 kB/s | 188 kB 00:00 2024-09-19T09:15:53.915 INFO:teuthology.orchestra.run.smithi016.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T09:15:53.940 INFO:teuthology.orchestra.run.smithi016.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 808 kB/s | 20 kB 00:00 2024-09-19T09:15:54.178 INFO:teuthology.orchestra.run.smithi016.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-19T09:15:54.178 INFO:teuthology.orchestra.run.smithi016.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.5 MB/s | 191 kB 00:00 2024-09-19T09:15:54.178 INFO:teuthology.orchestra.run.smithi016.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.2 MB/s | 427 kB 00:00 2024-09-19T09:15:54.179 INFO:teuthology.orchestra.run.smithi016.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T09:15:54.329 INFO:teuthology.orchestra.run.smithi016.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-19T09:15:54.504 INFO:teuthology.orchestra.run.smithi016.stdout:(150/156): librados2-19.1.1-361.g2f599a46.el9.x 6.7 MB/s | 3.4 MB 00:00 2024-09-19T09:15:54.654 INFO:teuthology.orchestra.run.smithi016.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 273 kB/s | 41 kB 00:00 2024-09-19T09:15:54.704 INFO:teuthology.orchestra.run.smithi016.stdout:(152/156): NetworkManager-libnm-1.51.0-1.el9.x8 4.8 MB/s | 1.8 MB 00:00 2024-09-19T09:15:54.713 INFO:teuthology.orchestra.run.smithi142.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 1.9 MB/s | 5.8 MB 00:03 2024-09-19T09:15:54.780 INFO:teuthology.orchestra.run.smithi016.stdout:(153/156): NetworkManager-1.51.0-1.el9.x86_64.r 3.3 MB/s | 2.3 MB 00:00 2024-09-19T09:15:54.805 INFO:teuthology.orchestra.run.smithi016.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 1.6 MB/s | 249 kB 00:00 2024-09-19T09:15:54.864 INFO:teuthology.orchestra.run.smithi142.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-09-19T09:15:54.997 INFO:teuthology.orchestra.run.smithi142.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-09-19T09:15:55.081 INFO:teuthology.orchestra.run.smithi142.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 928 kB/s | 77 kB 00:00 2024-09-19T09:15:55.131 INFO:teuthology.orchestra.run.smithi142.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-09-19T09:15:55.207 INFO:teuthology.orchestra.run.smithi142.stdout:(86/156): python3-babel-2.9.1-2.el9.noarch.rpm 1.9 MB/s | 6.0 MB 00:03 2024-09-19T09:15:55.315 INFO:teuthology.orchestra.run.smithi142.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-19T09:15:55.365 INFO:teuthology.orchestra.run.smithi142.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 1.1 MB/s | 269 kB 00:00 2024-09-19T09:15:55.413 INFO:teuthology.orchestra.run.smithi016.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.9 MB/s | 1.4 MB 00:00 2024-09-19T09:15:55.415 INFO:teuthology.orchestra.run.smithi142.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T09:15:55.466 INFO:teuthology.orchestra.run.smithi142.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T09:15:55.516 INFO:teuthology.orchestra.run.smithi142.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.4 MB/s | 279 kB 00:00 2024-09-19T09:15:55.580 INFO:teuthology.orchestra.run.smithi016.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.0 MB/s | 2.4 MB 00:00 2024-09-19T09:15:55.587 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-19T09:15:55.587 INFO:teuthology.orchestra.run.smithi016.stdout:Total 15 MB/s | 226 MB 00:14 2024-09-19T09:15:55.591 INFO:teuthology.orchestra.run.smithi142.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.0 MB/s | 5.1 MB 00:02 2024-09-19T09:15:55.617 INFO:teuthology.orchestra.run.smithi142.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-19T09:15:55.650 INFO:teuthology.orchestra.run.smithi142.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 755 kB/s | 44 kB 00:00 2024-09-19T09:15:55.709 INFO:teuthology.orchestra.run.smithi142.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 617 kB/s | 36 kB 00:00 2024-09-19T09:15:55.767 INFO:teuthology.orchestra.run.smithi142.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-09-19T09:15:55.826 INFO:teuthology.orchestra.run.smithi142.stdout:(97/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.0 MB/s | 221 kB 00:00 2024-09-19T09:15:55.903 INFO:teuthology.orchestra.run.smithi142.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-19T09:15:55.926 INFO:teuthology.orchestra.run.smithi142.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 576 kB/s | 58 kB 00:00 2024-09-19T09:15:55.960 INFO:teuthology.orchestra.run.smithi142.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-19T09:15:56.144 INFO:teuthology.orchestra.run.smithi142.stdout:(101/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-09-19T09:15:56.169 INFO:teuthology.orchestra.run.smithi142.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 263 kB/s | 64 kB 00:00 2024-09-19T09:15:56.194 INFO:teuthology.orchestra.run.smithi142.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 191 kB/s | 45 kB 00:00 2024-09-19T09:15:56.245 INFO:teuthology.orchestra.run.smithi142.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T09:15:56.270 INFO:teuthology.orchestra.run.smithi142.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-19T09:15:56.295 INFO:teuthology.orchestra.run.smithi142.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-09-19T09:15:56.404 INFO:teuthology.orchestra.run.smithi142.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-19T09:15:56.446 INFO:teuthology.orchestra.run.smithi142.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 959 kB/s | 26 kB 00:00 2024-09-19T09:15:56.471 INFO:teuthology.orchestra.run.smithi142.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T09:15:56.496 INFO:teuthology.orchestra.run.smithi142.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T09:15:56.521 INFO:teuthology.orchestra.run.smithi142.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-19T09:15:56.555 INFO:teuthology.orchestra.run.smithi142.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 25 MB/s | 839 kB 00:00 2024-09-19T09:15:56.588 INFO:teuthology.orchestra.run.smithi142.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-19T09:15:56.614 INFO:teuthology.orchestra.run.smithi142.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T09:15:56.639 INFO:teuthology.orchestra.run.smithi142.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T09:15:56.664 INFO:teuthology.orchestra.run.smithi142.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-09-19T09:15:56.681 INFO:teuthology.orchestra.run.smithi142.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 10 MB/s | 172 kB 00:00 2024-09-19T09:15:56.715 INFO:teuthology.orchestra.run.smithi142.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-19T09:15:56.740 INFO:teuthology.orchestra.run.smithi142.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 12 MB/s | 303 kB 00:00 2024-09-19T09:15:56.757 INFO:teuthology.orchestra.run.smithi142.stdout:(120/156): lua-devel-5.4.4-4.el9.x86_64.rpm 36 kB/s | 22 kB 00:00 2024-09-19T09:15:56.791 INFO:teuthology.orchestra.run.smithi142.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.4 MB/s | 220 kB 00:00 2024-09-19T09:15:56.816 INFO:teuthology.orchestra.run.smithi142.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 6.1 MB/s | 144 kB 00:00 2024-09-19T09:15:56.833 INFO:teuthology.orchestra.run.smithi142.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 642 kB/s | 11 kB 00:00 2024-09-19T09:15:56.850 INFO:teuthology.orchestra.run.smithi142.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.1 MB/s | 18 kB 00:00 2024-09-19T09:15:56.852 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-19T09:15:56.875 INFO:teuthology.orchestra.run.smithi142.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 936 kB/s | 23 kB 00:00 2024-09-19T09:15:56.901 INFO:teuthology.orchestra.run.smithi142.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-09-19T09:15:56.926 INFO:teuthology.orchestra.run.smithi142.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-19T09:15:56.976 INFO:teuthology.orchestra.run.smithi142.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 20 MB/s | 1.0 MB 00:00 2024-09-19T09:15:57.001 INFO:teuthology.orchestra.run.smithi142.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T09:15:57.020 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-19T09:15:57.021 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-19T09:15:57.043 INFO:teuthology.orchestra.run.smithi142.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.1 MB/s | 2.0 MB 00:00 2024-09-19T09:15:57.060 INFO:teuthology.orchestra.run.smithi142.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-19T09:15:57.086 INFO:teuthology.orchestra.run.smithi142.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-09-19T09:15:57.111 INFO:teuthology.orchestra.run.smithi142.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 655 kB/s | 16 kB 00:00 2024-09-19T09:15:57.136 INFO:teuthology.orchestra.run.smithi142.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-19T09:15:57.162 INFO:teuthology.orchestra.run.smithi142.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-19T09:15:57.188 INFO:teuthology.orchestra.run.smithi142.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-19T09:15:57.212 INFO:teuthology.orchestra.run.smithi142.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-19T09:15:57.237 INFO:teuthology.orchestra.run.smithi142.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T09:15:57.262 INFO:teuthology.orchestra.run.smithi142.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-19T09:15:57.288 INFO:teuthology.orchestra.run.smithi142.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-19T09:15:57.313 INFO:teuthology.orchestra.run.smithi142.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-19T09:15:57.338 INFO:teuthology.orchestra.run.smithi142.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T09:15:57.363 INFO:teuthology.orchestra.run.smithi142.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T09:15:57.389 INFO:teuthology.orchestra.run.smithi142.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 807 kB/s | 20 kB 00:00 2024-09-19T09:15:57.414 INFO:teuthology.orchestra.run.smithi142.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-19T09:15:57.439 INFO:teuthology.orchestra.run.smithi142.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T09:15:57.490 INFO:teuthology.orchestra.run.smithi142.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-19T09:15:57.724 INFO:teuthology.orchestra.run.smithi142.stdout:(148/156): librados2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-19T09:15:57.782 INFO:teuthology.orchestra.run.smithi142.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-19T09:15:58.024 INFO:teuthology.orchestra.run.smithi142.stdout:(150/156): protobuf-compiler-3.14.0-13.el9.x86_ 465 kB/s | 863 kB 00:01 2024-09-19T09:15:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-19T09:15:58.541 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T09:15:58.600 INFO:teuthology.orchestra.run.smithi142.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 72 kB/s | 41 kB 00:00 2024-09-19T09:15:58.669 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T09:15:58.810 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T09:15:58.851 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T09:15:58.876 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T09:15:58.919 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T09:15:59.250 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T09:15:59.352 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:15:59.373 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-19T09:15:59.373 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-19T09:15:59.456 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:15:59.466 INFO:teuthology.orchestra.run.smithi142.stdout:(152/156): NetworkManager-tui-1.51.0-1.el9.x86_ 287 kB/s | 249 kB 00:00 2024-09-19T09:15:59.478 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:15:59.553 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:15:59.590 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T09:15:59.747 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T09:15:59.796 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T09:15:59.824 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:15:59.944 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:15:59.966 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T09:16:00.029 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T09:16:00.051 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T09:16:00.078 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T09:16:00.270 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T09:16:00.274 INFO:teuthology.orchestra.run.smithi142.stdout:(153/156): NetworkManager-libnm-1.51.0-1.el9.x8 743 kB/s | 1.8 MB 00:02 2024-09-19T09:16:00.384 INFO:teuthology.orchestra.run.smithi142.stdout:(154/156): NetworkManager-1.51.0-1.el9.x86_64.r 893 kB/s | 2.3 MB 00:02 2024-09-19T09:16:00.403 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T09:16:00.517 INFO:teuthology.orchestra.run.smithi142.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.3 MB/s | 1.4 MB 00:01 2024-09-19T09:16:00.525 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T09:16:00.621 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:16:00.688 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:16:00.915 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:16:01.022 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T09:16:01.117 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T09:16:01.152 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T09:16:01.187 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T09:16:01.261 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T09:16:01.316 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T09:16:01.317 INFO:teuthology.orchestra.run.smithi142.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-19T09:16:01.324 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-09-19T09:16:01.324 INFO:teuthology.orchestra.run.smithi142.stdout:Total 12 MB/s | 226 MB 00:18 2024-09-19T09:16:01.353 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T09:16:01.397 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T09:16:01.429 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T09:16:01.457 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T09:16:01.527 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T09:16:01.580 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T09:16:01.610 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T09:16:01.654 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T09:16:01.730 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T09:16:01.773 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T09:16:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T09:16:01.854 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T09:16:01.926 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T09:16:01.968 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T09:16:02.044 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T09:16:02.176 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T09:16:02.229 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T09:16:02.561 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-09-19T09:16:02.714 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-09-19T09:16:02.715 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-09-19T09:16:02.752 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T09:16:02.837 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T09:16:02.865 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T09:16:02.908 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T09:16:02.946 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T09:16:02.990 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T09:16:03.143 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T09:16:03.203 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T09:16:03.256 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T09:16:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T09:16:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T09:16:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T09:16:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T09:16:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T09:16:03.539 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T09:16:03.598 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T09:16:03.733 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T09:16:03.776 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T09:16:03.900 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T09:16:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T09:16:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T09:16:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T09:16:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T09:16:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T09:16:04.240 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T09:16:04.367 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T09:16:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T09:16:04.427 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T09:16:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T09:16:04.500 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T09:16:04.528 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-09-19T09:16:04.529 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-09-19T09:16:04.531 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T09:16:04.561 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T09:16:04.605 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T09:16:04.655 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T09:16:04.696 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T09:16:04.731 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T09:16:04.817 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T09:16:04.862 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T09:16:04.893 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T09:16:04.996 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T09:16:05.043 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T09:16:05.768 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T09:16:05.817 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T09:16:05.850 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T09:16:05.948 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T09:16:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-19T09:16:05.996 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T09:16:06.042 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T09:16:06.127 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T09:16:06.135 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T09:16:06.170 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T09:16:06.182 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T09:16:06.244 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T09:16:06.278 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T09:16:06.303 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T09:16:06.353 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T09:16:06.360 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T09:16:06.392 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T09:16:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T09:16:06.702 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T09:16:06.812 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:16:06.937 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:16:06.971 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:16:07.064 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:16:07.098 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T09:16:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T09:16:07.223 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T09:16:07.293 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T09:16:07.349 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T09:16:07.385 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:16:07.483 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:16:07.519 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T09:16:07.582 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T09:16:07.620 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T09:16:07.642 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T09:16:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T09:16:07.976 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T09:16:08.102 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T09:16:08.218 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:16:08.295 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:16:08.426 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T09:16:08.506 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T09:16:08.534 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T09:16:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:16:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T09:16:08.655 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T09:16:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T09:16:08.798 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T09:16:08.835 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T09:16:08.874 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T09:16:08.919 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T09:16:08.971 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T09:16:08.992 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T09:16:09.011 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T09:16:09.054 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T09:16:09.095 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T09:16:09.123 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T09:16:09.194 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T09:16:09.205 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T09:16:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T09:16:09.262 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T09:16:09.279 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T09:16:09.293 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T09:16:09.336 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T09:16:09.415 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T09:16:09.447 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T09:16:09.491 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T09:16:09.519 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T09:16:09.595 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T09:16:09.651 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T09:16:09.730 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T09:16:09.759 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T09:16:09.800 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T09:16:09.871 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T09:16:09.875 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T09:16:09.902 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T09:16:09.931 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T09:16:10.495 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T09:16:10.586 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T09:16:10.620 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T09:16:10.663 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T09:16:10.701 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T09:16:10.751 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T09:16:10.848 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-09-19T09:16:10.918 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T09:16:10.970 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T09:16:10.997 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T09:16:11.009 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T09:16:11.061 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T09:16:11.062 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T09:16:11.096 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T09:16:11.121 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T09:16:11.136 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T09:16:11.163 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T09:16:11.172 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T09:16:11.208 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T09:16:11.256 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T09:16:11.291 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T09:16:11.335 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T09:16:11.378 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T09:16:11.488 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T09:16:11.530 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T09:16:11.579 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T09:16:11.597 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:16:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T09:16:11.719 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:16:11.727 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T09:16:11.749 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:16:11.762 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T09:16:11.808 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T09:16:11.840 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:16:11.869 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T09:16:11.881 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T09:16:12.018 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T09:16:12.066 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T09:16:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T09:16:12.094 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:16:12.198 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:16:12.228 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T09:16:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T09:16:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T09:16:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T09:16:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T09:16:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T09:16:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T09:16:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T09:16:12.517 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:16:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T09:16:12.539 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T09:16:12.556 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:16:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T09:16:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T09:16:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T09:16:12.683 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T09:16:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T09:16:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T09:16:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T09:16:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T09:16:12.803 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T09:16:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T09:16:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:16:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T09:16:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T09:16:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:16:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T09:16:13.097 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:16:13.135 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T09:16:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T09:16:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T09:16:13.210 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T09:16:13.210 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T09:16:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:16:13.211 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T09:16:13.245 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T09:16:13.320 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T09:16:13.414 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T09:16:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T09:16:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T09:16:13.550 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T09:16:13.596 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T09:16:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T09:16:13.673 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T09:16:13.706 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T09:16:13.734 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T09:16:13.803 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T09:16:13.856 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T09:16:13.887 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T09:16:13.907 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T09:16:13.922 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T09:16:13.951 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T09:16:13.985 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T09:16:13.998 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T09:16:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T09:16:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T09:16:14.089 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T09:16:14.105 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T09:16:14.149 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T09:16:14.175 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T09:16:14.194 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T09:16:14.220 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T09:16:14.284 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T09:16:14.294 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T09:16:14.321 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T09:16:14.427 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T09:16:14.480 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T09:16:14.511 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T09:16:14.543 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T09:16:14.623 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T09:16:15.005 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T09:16:15.084 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T09:16:15.113 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T09:16:15.156 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T09:16:15.194 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T09:16:15.237 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T09:16:15.317 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T09:16:15.385 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T09:16:15.441 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T09:16:15.442 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T09:16:15.494 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T09:16:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T09:16:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T09:16:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T09:16:15.699 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T09:16:15.734 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T09:16:15.778 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T09:16:15.819 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T09:16:15.956 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T09:16:16.004 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T09:16:16.046 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T09:16:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T09:16:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T09:16:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T09:16:16.245 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T09:16:16.270 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T09:16:16.344 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T09:16:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T09:16:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T09:16:16.730 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T09:16:16.764 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T09:16:16.776 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T09:16:16.803 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T09:16:16.835 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T09:16:16.865 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T09:16:16.865 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T09:16:16.893 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T09:16:16.899 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T09:16:16.926 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T09:16:16.933 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T09:16:16.974 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T09:16:17.008 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T09:16:17.094 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T09:16:17.139 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T09:16:17.170 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T09:16:17.253 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T09:16:17.273 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T09:16:17.289 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T09:16:17.321 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T09:16:17.364 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T09:16:17.394 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T09:16:17.430 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T09:16:17.940 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T09:16:17.976 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T09:16:18.042 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T09:16:18.093 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T09:16:18.127 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T09:16:18.234 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T09:16:18.287 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T09:16:18.290 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T09:16:18.327 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T09:16:18.411 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T09:16:18.413 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T09:16:18.446 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T09:16:18.631 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T09:16:18.661 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T09:16:18.737 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T09:16:19.380 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T09:16:19.499 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T09:16:20.711 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T09:16:20.757 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:16:20.791 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:16:20.805 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T09:16:20.841 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T09:16:20.874 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T09:16:21.183 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T09:16:21.221 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T09:16:21.312 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T09:16:21.342 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T09:16:21.368 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:16:21.380 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T09:16:21.405 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T09:16:21.496 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T09:16:21.496 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T09:16:21.496 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T09:16:21.497 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:16:21.539 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T09:16:21.857 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T09:16:21.890 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T09:16:21.961 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T09:16:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T09:16:24.265 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:16:24.313 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:16:24.857 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T09:16:24.872 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T09:16:24.872 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-09-19T09:16:24.873 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-09-19T09:16:24.873 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-09-19T09:16:24.873 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-09-19T09:16:24.873 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-09-19T09:16:24.873 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-09-19T09:16:24.873 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-09-19T09:16:24.873 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:16:24.892 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T09:16:24.985 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T09:16:24.985 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T09:16:24.985 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T09:16:24.986 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:16:25.018 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T09:16:25.049 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T09:16:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T09:16:25.066 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:25.066 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T09:16:25.066 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:16:26.562 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T09:16:26.589 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T09:16:26.589 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:26.589 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T09:16:26.589 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T09:16:26.589 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T09:16:26.589 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:16:26.813 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T09:16:26.836 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T09:16:26.837 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:26.837 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T09:16:26.837 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T09:16:26.837 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T09:16:26.837 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:16:27.272 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T09:16:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T09:16:27.295 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:27.295 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T09:16:27.295 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T09:16:27.295 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T09:16:27.295 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:16:27.402 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T09:16:27.467 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T09:16:27.531 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T09:16:27.717 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T09:16:27.764 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T09:16:27.842 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T09:16:27.939 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T09:16:28.079 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T09:16:28.123 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T09:16:28.163 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T09:16:28.191 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T09:16:28.662 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T09:16:28.690 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T09:16:28.721 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T09:16:28.784 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T09:16:28.821 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T09:16:29.867 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T09:16:29.906 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T09:16:30.863 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T09:16:30.898 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T09:16:31.002 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T09:16:31.160 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T09:16:31.193 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T09:16:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T09:16:31.217 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:31.217 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T09:16:31.217 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T09:16:31.217 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T09:16:31.217 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:16:31.268 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T09:16:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T09:16:31.378 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T09:16:31.433 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T09:16:32.355 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T09:16:32.388 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T09:16:32.411 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T09:16:32.411 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:32.412 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T09:16:32.412 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T09:16:32.412 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T09:16:32.412 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:16:32.463 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T09:16:32.486 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T09:16:32.487 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:32.487 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T09:16:32.487 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:16:32.539 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T09:16:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T09:16:32.563 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:32.563 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T09:16:32.563 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T09:16:32.563 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T09:16:32.563 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:16:32.865 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T09:16:32.888 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T09:16:32.888 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:32.888 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T09:16:32.888 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T09:16:32.888 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T09:16:32.888 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:16:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T09:16:33.819 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-09-19T09:16:33.819 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-09-19T09:16:33.820 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-09-19T09:16:33.820 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-09-19T09:16:33.820 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-09-19T09:16:33.820 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-09-19T09:16:33.820 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-09-19T09:16:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:16:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T09:16:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T09:16:34.020 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:34.020 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T09:16:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:16:35.548 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T09:16:35.576 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T09:16:35.576 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:35.576 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T09:16:35.576 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T09:16:35.576 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T09:16:35.576 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:16:35.815 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T09:16:35.840 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T09:16:35.840 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:35.840 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T09:16:35.840 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T09:16:35.840 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T09:16:35.841 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:16:36.299 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T09:16:36.327 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T09:16:36.327 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:36.327 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T09:16:36.327 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T09:16:36.327 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T09:16:36.327 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:16:36.446 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T09:16:36.497 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T09:16:36.562 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T09:16:36.562 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-09-19T09:16:36.562 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-09-19T09:16:36.563 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-09-19T09:16:36.563 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-09-19T09:16:36.563 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-09-19T09:16:36.563 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /root 2024-09-19T09:16:36.563 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-09-19T09:16:36.563 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:16:36.570 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T09:16:36.739 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T09:16:36.755 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T09:16:36.755 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:36.755 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T09:16:36.755 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:16:36.765 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T09:16:36.818 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T09:16:36.900 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T09:16:37.005 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T09:16:37.149 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T09:16:37.200 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T09:16:37.233 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T09:16:37.260 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T09:16:37.427 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T09:16:37.468 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T09:16:37.509 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T09:16:37.541 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T09:16:37.568 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T09:16:37.682 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T09:16:37.718 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T09:16:37.749 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T09:16:37.749 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T09:16:37.799 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T09:16:37.816 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T09:16:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T09:16:37.840 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T09:16:37.859 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T09:16:37.905 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T09:16:38.225 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T09:16:38.252 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T09:16:38.252 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:38.252 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T09:16:38.252 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T09:16:38.252 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T09:16:38.252 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:16:38.484 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T09:16:38.493 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:16:38.507 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T09:16:38.508 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:38.508 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T09:16:38.508 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T09:16:38.508 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T09:16:38.508 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:16:38.536 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:16:38.557 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:16:38.943 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T09:16:38.950 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T09:16:38.967 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T09:16:38.967 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:38.967 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T09:16:38.967 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T09:16:38.967 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T09:16:38.967 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:16:38.992 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T09:16:39.081 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T09:16:39.138 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T09:16:39.193 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T09:16:39.377 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T09:16:39.406 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T09:16:39.426 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T09:16:39.492 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T09:16:39.495 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T09:16:39.533 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T09:16:39.584 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T09:16:39.614 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T09:16:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T09:16:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T09:16:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T09:16:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T09:16:39.714 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T09:16:39.769 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T09:16:39.789 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T09:16:39.789 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T09:16:39.800 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T09:16:39.828 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T09:16:39.864 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T09:16:39.889 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T09:16:39.895 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:16:39.895 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:16:40.007 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T09:16:40.034 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T09:16:40.074 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:16:40.075 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T09:16:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T09:16:40.145 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T09:16:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:16:40.291 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T09:16:40.313 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T09:16:40.327 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T09:16:40.345 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T09:16:40.366 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T09:16:40.371 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T09:16:40.371 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:40.371 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T09:16:40.371 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T09:16:40.372 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T09:16:40.372 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:16:40.421 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T09:16:40.428 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T09:16:40.434 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T09:16:40.458 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T09:16:40.538 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T09:16:40.597 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T09:16:41.502 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T09:16:41.543 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T09:16:41.573 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T09:16:41.607 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T09:16:41.632 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T09:16:41.633 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:41.633 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T09:16:41.633 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T09:16:41.633 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T09:16:41.633 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:16:41.683 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T09:16:41.707 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T09:16:41.707 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:41.707 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T09:16:41.708 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:16:41.758 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T09:16:41.783 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T09:16:41.784 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:41.784 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T09:16:41.784 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T09:16:41.784 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T09:16:41.784 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:16:42.101 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T09:16:42.126 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T09:16:42.126 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:42.126 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T09:16:42.126 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T09:16:42.126 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T09:16:42.126 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:16:42.505 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T09:16:42.535 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T09:16:42.640 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T09:16:42.806 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T09:16:42.837 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T09:16:42.862 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T09:16:42.862 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:42.862 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T09:16:42.862 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T09:16:42.862 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T09:16:42.862 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:16:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T09:16:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T09:16:43.023 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T09:16:43.078 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T09:16:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T09:16:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T09:16:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T09:16:44.065 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:44.065 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T09:16:44.065 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T09:16:44.065 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T09:16:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:16:44.108 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T09:16:44.131 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T09:16:44.131 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:44.131 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T09:16:44.131 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:16:44.183 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T09:16:44.207 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T09:16:44.207 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:44.207 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T09:16:44.208 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T09:16:44.208 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T09:16:44.208 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:16:44.517 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T09:16:44.541 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T09:16:44.541 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T09:16:44.541 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T09:16:44.541 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T09:16:44.541 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T09:16:44.542 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:16:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:16:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T09:16:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T09:16:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T09:16:45.842 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T09:16:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T09:16:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T09:16:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T09:16:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:16:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:16:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T09:16:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T09:16:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T09:16:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:16:45.843 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T09:16:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T09:16:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T09:16:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T09:16:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T09:16:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T09:16:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T09:16:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T09:16:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T09:16:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T09:16:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T09:16:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T09:16:45.844 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T09:16:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T09:16:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T09:16:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T09:16:45.845 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T09:16:45.857 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T09:16:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T09:16:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T09:16:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T09:16:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T09:16:45.858 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T09:16:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T09:16:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T09:16:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T09:16:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T09:16:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T09:16:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T09:16:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T09:16:45.859 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T09:16:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T09:16:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T09:16:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T09:16:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T09:16:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T09:16:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T09:16:45.860 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T09:16:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T09:16:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T09:16:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T09:16:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T09:16:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T09:16:45.861 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T09:16:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T09:16:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T09:16:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T09:16:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T09:16:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T09:16:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T09:16:45.862 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T09:16:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T09:16:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T09:16:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T09:16:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T09:16:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T09:16:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T09:16:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T09:16:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T09:16:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T09:16:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T09:16:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T09:16:45.863 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T09:16:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T09:16:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T09:16:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T09:16:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T09:16:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T09:16:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T09:16:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T09:16:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T09:16:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T09:16:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T09:16:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T09:16:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T09:16:45.864 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T09:16:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T09:16:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T09:16:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T09:16:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T09:16:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T09:16:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T09:16:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T09:16:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T09:16:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T09:16:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T09:16:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T09:16:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T09:16:45.865 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T09:16:45.866 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T09:16:45.867 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T09:16:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T09:16:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T09:16:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T09:16:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T09:16:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T09:16:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T09:16:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T09:16:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T09:16:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T09:16:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T09:16:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T09:16:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T09:16:45.868 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T09:16:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T09:16:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T09:16:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T09:16:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T09:16:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T09:16:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T09:16:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T09:16:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T09:16:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T09:16:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T09:16:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T09:16:45.869 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T09:16:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T09:16:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T09:16:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T09:16:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T09:16:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T09:16:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T09:16:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T09:16:45.870 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T09:16:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T09:16:47.005 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T09:16:47.055 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T09:16:47.094 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T09:16:47.121 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T09:16:47.326 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T09:16:47.377 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T09:16:47.407 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T09:16:47.440 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T09:16:47.483 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T09:16:48.117 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:16:48.156 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:16:48.178 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:16:48.599 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:16:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:16:48.600 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-09-19T09:16:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T09:16:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T09:16:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T09:16:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T09:16:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T09:16:48.600 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T09:16:48.600 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-19T09:16:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T09:16:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T09:16:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T09:16:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T09:16:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.601 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:48.602 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T09:16:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T09:16:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T09:16:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T09:16:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T09:16:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T09:16:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T09:16:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T09:16:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T09:16:48.603 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T09:16:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T09:16:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T09:16:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T09:16:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T09:16:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T09:16:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T09:16:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T09:16:48.604 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T09:16:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T09:16:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T09:16:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T09:16:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T09:16:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T09:16:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T09:16:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T09:16:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T09:16:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T09:16:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T09:16:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T09:16:48.605 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T09:16:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T09:16:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T09:16:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T09:16:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T09:16:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T09:16:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T09:16:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T09:16:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T09:16:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T09:16:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T09:16:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.606 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T09:16:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T09:16:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T09:16:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T09:16:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T09:16:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T09:16:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T09:16:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T09:16:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T09:16:48.607 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T09:16:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T09:16:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T09:16:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T09:16:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T09:16:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T09:16:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T09:16:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T09:16:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T09:16:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T09:16:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T09:16:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T09:16:48.608 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T09:16:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T09:16:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T09:16:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T09:16:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T09:16:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T09:16:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T09:16:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T09:16:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T09:16:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T09:16:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T09:16:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T09:16:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T09:16:48.609 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T09:16:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T09:16:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T09:16:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T09:16:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T09:16:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T09:16:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T09:16:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T09:16:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T09:16:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T09:16:48.610 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T09:16:48.611 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T09:16:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T09:16:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T09:16:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T09:16:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T09:16:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T09:16:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T09:16:48.613 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:16:48.613 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-19T09:16:49.043 DEBUG:teuthology.parallel:result is None 2024-09-19T09:16:49.079 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T09:16:49.087 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T09:16:49.121 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T09:16:49.169 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T09:16:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T09:16:49.210 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T09:16:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T09:16:49.254 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T09:16:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T09:16:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T09:16:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T09:16:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T09:16:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T09:16:49.446 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T09:16:49.492 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T09:16:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T09:16:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T09:16:49.531 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T09:16:49.560 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T09:16:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T09:16:49.607 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T09:16:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T09:16:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:16:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:16:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:16:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T09:16:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T09:16:49.981 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:16:50.229 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:16:50.271 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:16:50.292 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T09:16:51.128 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T09:16:51.217 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T09:16:51.258 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T09:16:51.348 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T09:16:51.404 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T09:16:51.429 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T09:16:51.455 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T09:16:51.456 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T09:16:51.539 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T09:16:51.539 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T09:16:51.623 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T09:16:51.648 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T09:16:51.653 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:16:51.654 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:16:51.833 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T09:16:51.834 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T09:16:51.926 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T09:16:51.998 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:16:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:16:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T09:16:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T09:16:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T09:16:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T09:16:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T09:16:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T09:16:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T09:16:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:16:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:16:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T09:16:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T09:16:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T09:16:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:16:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T09:16:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T09:16:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T09:16:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T09:16:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T09:16:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T09:16:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T09:16:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T09:16:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T09:16:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T09:16:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T09:16:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T09:16:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T09:16:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T09:16:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T09:16:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T09:16:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T09:16:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T09:16:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T09:16:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T09:16:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T09:16:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T09:16:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T09:16:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T09:16:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T09:16:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T09:16:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T09:16:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T09:16:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T09:16:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T09:16:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T09:16:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T09:16:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T09:16:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T09:16:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T09:16:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T09:16:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T09:16:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T09:16:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T09:16:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T09:16:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T09:16:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T09:16:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T09:16:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T09:16:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T09:16:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T09:16:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T09:16:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T09:16:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T09:16:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T09:16:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T09:16:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T09:16:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T09:16:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T09:16:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T09:16:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T09:16:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T09:16:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T09:16:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T09:16:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T09:16:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T09:16:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T09:16:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T09:16:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T09:16:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T09:16:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T09:16:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T09:16:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T09:16:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T09:16:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T09:16:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T09:16:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T09:16:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T09:16:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T09:16:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T09:16:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T09:16:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T09:16:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T09:16:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T09:16:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T09:16:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T09:16:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T09:16:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T09:16:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T09:16:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T09:16:57.519 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:16:57.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T09:16:57.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T09:16:57.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T09:16:57.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T09:16:57.519 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T09:16:57.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T09:16:57.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T09:16:57.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T09:16:57.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T09:16:57.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T09:16:57.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T09:16:57.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T09:16:57.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T09:16:57.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T09:16:57.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T09:16:57.520 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T09:16:57.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T09:16:57.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T09:16:57.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T09:16:57.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T09:16:57.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T09:16:57.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T09:16:57.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T09:16:57.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T09:16:57.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T09:16:57.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T09:16:57.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T09:16:57.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T09:16:57.521 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T09:16:57.522 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T09:16:57.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T09:16:57.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T09:16:57.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T09:16:57.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T09:16:57.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T09:16:57.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T09:16:57.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T09:16:57.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T09:16:57.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T09:16:57.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T09:16:57.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T09:16:57.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T09:16:57.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T09:16:57.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T09:16:57.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T09:16:57.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T09:16:57.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T09:16:57.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T09:16:57.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T09:16:57.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T09:16:57.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T09:16:57.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T09:16:57.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T09:16:57.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T09:16:57.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T09:16:57.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T09:16:57.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T09:16:57.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T09:16:57.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T09:16:57.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T09:16:57.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T09:16:57.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T09:16:57.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T09:16:57.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T09:16:57.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T09:16:57.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T09:16:57.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T09:16:57.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T09:16:57.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T09:16:57.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T09:16:57.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T09:16:57.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T09:16:57.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T09:16:57.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T09:16:57.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T09:16:57.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T09:16:57.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T09:16:57.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T09:16:57.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T09:16:57.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T09:16:57.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T09:16:57.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T09:16:57.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T09:16:57.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T09:16:57.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T09:16:57.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T09:16:57.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T09:16:57.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T09:16:57.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T09:16:57.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T09:16:57.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T09:16:57.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T09:16:57.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T09:16:57.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T09:16:57.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T09:16:57.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T09:16:57.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T09:16:57.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T09:16:57.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T09:16:57.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T09:16:57.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T09:16:57.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T09:16:57.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T09:16:57.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T09:16:57.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T09:16:57.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T09:16:57.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T09:16:57.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T09:16:57.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T09:16:57.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T09:16:57.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T09:16:57.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T09:16:57.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T09:16:57.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T09:16:57.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T09:16:57.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T09:16:57.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T09:16:57.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T09:16:57.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T09:16:57.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T09:16:57.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T09:16:57.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T09:16:57.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T09:16:57.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T09:16:57.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T09:16:57.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T09:16:57.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T09:16:57.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T09:16:57.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T09:16:57.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T09:16:57.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T09:16:57.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T09:16:57.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T09:16:57.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T09:16:57.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T09:16:57.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T09:16:57.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T09:16:58.491 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:16:58.491 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:16:58.491 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-09-19T09:16:58.491 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T09:16:58.491 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T09:16:58.491 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T09:16:58.491 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T09:16:58.491 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.492 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.492 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T09:16:58.492 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T09:16:58.492 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-19T09:16:58.492 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T09:16:58.492 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T09:16:58.492 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T09:16:58.492 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T09:16:58.492 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.492 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.492 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.492 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:58.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:58.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:58.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:58.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:58.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:58.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:58.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.494 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.494 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.494 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:58.494 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:58.494 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.494 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.494 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:58.494 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T09:16:58.494 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T09:16:58.494 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T09:16:58.494 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T09:16:58.494 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T09:16:58.494 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T09:16:58.495 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T09:16:58.495 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T09:16:58.495 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T09:16:58.495 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T09:16:58.495 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T09:16:58.495 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.495 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.495 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.495 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T09:16:58.495 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T09:16:58.495 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T09:16:58.495 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T09:16:58.495 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T09:16:58.496 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.496 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.496 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T09:16:58.496 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.496 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T09:16:58.496 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T09:16:58.496 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T09:16:58.496 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T09:16:58.496 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T09:16:58.496 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T09:16:58.496 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T09:16:58.496 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T09:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T09:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T09:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T09:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T09:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T09:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T09:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T09:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T09:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T09:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T09:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T09:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T09:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T09:16:58.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T09:16:58.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T09:16:58.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T09:16:58.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T09:16:58.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T09:16:58.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T09:16:58.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T09:16:58.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T09:16:58.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T09:16:58.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T09:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T09:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T09:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T09:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T09:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T09:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T09:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T09:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T09:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T09:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T09:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T09:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T09:16:58.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T09:16:58.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T09:16:58.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T09:16:58.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T09:16:58.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T09:16:58.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T09:16:58.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T09:16:58.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T09:16:58.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T09:16:58.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T09:16:58.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T09:16:58.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T09:16:58.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T09:16:58.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T09:16:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T09:16:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T09:16:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T09:16:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T09:16:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T09:16:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T09:16:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T09:16:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T09:16:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T09:16:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T09:16:58.502 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T09:16:58.503 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.503 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.503 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:58.503 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T09:16:58.503 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T09:16:58.503 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T09:16:58.503 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T09:16:58.503 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T09:16:58.503 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T09:16:58.504 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T09:16:58.504 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:16:58.504 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-19T09:16:59.039 DEBUG:teuthology.parallel:result is None 2024-09-19T09:16:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T09:16:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:16:59.977 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-09-19T09:16:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T09:16:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T09:16:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T09:16:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T09:16:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T09:16:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T09:16:59.978 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-09-19T09:16:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T09:16:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T09:16:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T09:16:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T09:16:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T09:16:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T09:16:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T09:16:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T09:16:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T09:16:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T09:16:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T09:16:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T09:16:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T09:16:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T09:16:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T09:16:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T09:16:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T09:16:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T09:16:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T09:16:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T09:16:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T09:16:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T09:16:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T09:16:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T09:16:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T09:16:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T09:16:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T09:16:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T09:16:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T09:16:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T09:16:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T09:16:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T09:16:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T09:16:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T09:16:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T09:16:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T09:16:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T09:16:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T09:16:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T09:16:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T09:16:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T09:16:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T09:16:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T09:16:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T09:16:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T09:16:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T09:16:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T09:16:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T09:16:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T09:16:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T09:16:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T09:16:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T09:16:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T09:16:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T09:16:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T09:16:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T09:16:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T09:16:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T09:16:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T09:16:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T09:16:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T09:16:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T09:16:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T09:16:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T09:16:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T09:16:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T09:16:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T09:16:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T09:16:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T09:16:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T09:16:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T09:16:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T09:16:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T09:16:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T09:16:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T09:16:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T09:16:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T09:16:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T09:16:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T09:16:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T09:16:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T09:16:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T09:16:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T09:16:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T09:16:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T09:16:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T09:16:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T09:16:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T09:16:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T09:16:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T09:16:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T09:16:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T09:16:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T09:16:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T09:16:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T09:16:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T09:16:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T09:16:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T09:16:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:16:59.991 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-09-19T09:17:00.403 DEBUG:teuthology.parallel:result is None 2024-09-19T09:17:00.404 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:17:00.556 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T09:17:00.584 INFO:teuthology.orchestra.run.smithi016.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T09:17:00.585 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T09:17:00.585 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T09:17:00.587 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:17:00.726 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T09:17:00.753 INFO:teuthology.orchestra.run.smithi097.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T09:17:00.753 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T09:17:00.754 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T09:17:00.757 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:17:00.909 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T09:17:00.940 INFO:teuthology.orchestra.run.smithi142.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T09:17:00.940 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T09:17:00.941 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T09:17:00.942 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T09:17:00.943 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:17:00.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T09:17:00.979 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-19T09:17:00.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T09:17:01.012 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-19T09:17:01.012 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T09:17:01.041 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T09:17:01.042 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:17:01.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T09:17:01.073 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T09:17:01.144 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-19T09:17:01.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T09:17:01.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T09:17:01.241 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-19T09:17:01.241 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T09:17:01.269 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T09:17:01.337 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T09:17:01.338 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:17:01.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T09:17:01.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T09:17:01.439 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-19T09:17:01.439 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T09:17:01.465 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T09:17:01.531 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-19T09:17:01.532 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T09:17:01.561 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T09:17:01.629 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T09:17:01.629 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:17:01.629 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T09:17:01.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T09:17:01.731 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-19T09:17:01.731 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T09:17:01.757 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T09:17:01.821 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-19T09:17:01.821 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T09:17:01.850 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T09:17:01.918 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T09:17:02.020 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-19T09:17:02.020 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:17:02.021 INFO:tasks.cephadm:Cluster fsid is ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:17:02.021 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T09:17:02.021 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.16', 'mon.b': '172.21.15.97', 'mon.c': '172.21.15.142'} 2024-09-19T09:17:02.021 INFO:tasks.cephadm:First mon is mon.a on smithi016 2024-09-19T09:17:02.021 INFO:tasks.cephadm:First mgr is x 2024-09-19T09:17:02.021 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T09:17:02.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-09-19T09:17:02.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2024-09-19T09:17:02.079 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2024-09-19T09:17:02.107 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T09:17:02.108 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:17:02.249 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T09:17:02.385 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:17:02.386 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T09:17:02.387 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T09:17:02.387 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T09:17:02.765 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 09:17 /home/ubuntu/cephtest/cephadm 2024-09-19T09:17:02.765 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T09:17:03.166 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 09:17 /home/ubuntu/cephtest/cephadm 2024-09-19T09:17:03.167 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T09:17:03.559 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 09:17 /home/ubuntu/cephtest/cephadm 2024-09-19T09:17:03.560 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T09:17:03.581 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T09:17:03.601 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T09:17:03.638 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T09:17:03.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T09:17:03.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T09:17:03.646 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T09:17:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T09:17:05.306 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T09:17:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T09:17:32.550 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-19T09:17:32.550 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T09:17:32.550 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T09:17:32.550 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2024-09-19T09:17:32.551 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T09:17:32.551 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-19T09:17:32.551 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-19T09:17:34.476 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-19T09:17:34.476 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T09:17:34.476 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T09:17:34.476 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2024-09-19T09:17:34.476 INFO:teuthology.orchestra.run.smithi142.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T09:17:34.476 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-09-19T09:17:34.476 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-19T09:17:35.108 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:17:35.108 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T09:17:35.108 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T09:17:35.108 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-09-19T09:17:35.108 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T09:17:35.108 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T09:17:35.108 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:17:35.139 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-09-19T09:17:35.170 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2024-09-19T09:17:35.203 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2024-09-19T09:17:35.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-09-19T09:17:35.264 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2024-09-19T09:17:35.293 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2024-09-19T09:17:35.321 INFO:tasks.cephadm:Writing seed config... 2024-09-19T09:17:35.322 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T09:17:35.322 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T09:17:35.322 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T09:17:35.322 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T09:17:35.322 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T09:17:35.322 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T09:17:35.322 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T09:17:35.323 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T09:17:35.323 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T09:17:35.323 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T09:17:35.323 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T09:17:35.323 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T09:17:35.323 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T09:17:35.323 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T09:17:35.323 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T09:17:35.323 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T09:17:35.323 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T09:17:35.323 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-19T09:17:35.323 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T09:17:35.323 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T09:17:35.324 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T09:17:35.324 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T09:17:35.324 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T09:17:35.324 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T09:17:35.324 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T09:17:35.324 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T09:17:35.324 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T09:17:35.324 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T09:17:35.324 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T09:17:35.324 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T09:17:35.324 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T09:17:35.324 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-19T09:17:35.324 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-19T09:17:35.325 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-19T09:17:35.325 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T09:17:35.325 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T09:17:35.325 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-19T09:17:35.325 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-19T09:17:35.325 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T09:17:35.325 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T09:17:35.325 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-19T09:17:35.325 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T09:17:35.325 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T09:17:35.325 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T09:17:35.326 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T09:17:35.326 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T09:17:35.326 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T09:17:35.326 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T09:17:35.326 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-19T09:17:35.327 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:17:35.327 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T09:17:35.343 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ee1ab676-7667-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T09:17:35.344 DEBUG:teuthology.orchestra.run.smithi016:mon.a> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T09:17:35.386 DEBUG:teuthology.orchestra.run.smithi016:mgr.x> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T09:17:35.430 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T09:17:35.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid ee1ab676-7667-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.16 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:17:35.641 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-19T09:17:35.641 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', 'ee1ab676-7667-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.16', '--skip-admin-label'] 2024-09-19T09:17:35.641 INFO:teuthology.orchestra.run.smithi016.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T09:17:35.642 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying podman|docker is present... 2024-09-19T09:17:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-19T09:17:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying lvm2 is present... 2024-09-19T09:17:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying time synchronization is in place... 2024-09-19T09:17:35.678 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T09:17:35.678 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T09:17:35.685 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T09:17:35.685 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-19T09:17:35.692 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-19T09:17:35.698 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-19T09:17:35.698 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-19T09:17:35.698 INFO:teuthology.orchestra.run.smithi016.stdout:Repeating the final host check... 2024-09-19T09:17:35.721 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-19T09:17:35.721 INFO:teuthology.orchestra.run.smithi016.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T09:17:35.721 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl is present 2024-09-19T09:17:35.721 INFO:teuthology.orchestra.run.smithi016.stdout:lvcreate is present 2024-09-19T09:17:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T09:17:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T09:17:35.734 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T09:17:35.734 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-19T09:17:35.741 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-19T09:17:35.748 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-19T09:17:35.748 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-19T09:17:35.748 INFO:teuthology.orchestra.run.smithi016.stdout:Host looks OK 2024-09-19T09:17:35.748 INFO:teuthology.orchestra.run.smithi016.stdout:Cluster fsid: ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:17:35.748 INFO:teuthology.orchestra.run.smithi016.stdout:Acquiring lock 140309460861952 on /run/cephadm/ee1ab676-7667-11ef-bceb-c7b262605968.lock 2024-09-19T09:17:35.748 INFO:teuthology.orchestra.run.smithi016.stdout:Lock 140309460861952 acquired on /run/cephadm/ee1ab676-7667-11ef-bceb-c7b262605968.lock 2024-09-19T09:17:35.749 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 3300 ... 2024-09-19T09:17:35.749 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 6789 ... 2024-09-19T09:17:35.749 INFO:teuthology.orchestra.run.smithi016.stdout:Base mon IP(s) is [172.21.15.16:3300, 172.21.15.16:6789], mon addrv is [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-19T09:17:35.752 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.16 metric 100 2024-09-19T09:17:35.752 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.16 metric 100 2024-09-19T09:17:35.755 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T09:17:35.755 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-19T09:17:35.755 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1788sec hoplimit 64 pref medium 2024-09-19T09:17:35.758 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T09:17:35.758 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T09:17:35.758 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T09:17:35.758 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-19T09:17:35.758 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T09:17:35.758 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T09:17:35.759 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-19T09:17:35.759 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-19T09:17:35.759 INFO:teuthology.orchestra.run.smithi016.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T09:17:35.759 INFO:teuthology.orchestra.run.smithi016.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T09:17:35.760 INFO:teuthology.orchestra.run.smithi016.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T09:17:36.226 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T09:17:36.226 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T09:17:36.227 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T09:17:36.227 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T09:17:36.227 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T09:17:36.227 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T09:17:36.227 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T09:17:37.131 INFO:teuthology.orchestra.run.smithi016.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T09:17:37.132 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T09:17:37.132 INFO:teuthology.orchestra.run.smithi016.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T09:17:38.071 INFO:teuthology.orchestra.run.smithi016.stdout:stat: stdout 167 167 2024-09-19T09:17:38.071 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial keys... 2024-09-19T09:17:38.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQAy7OtmK4edFhAAnqZ1+9ERygzr5CtQYnX1QQ== 2024-09-19T09:17:39.881 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQAz7OtmUrJ6EBAA9sMgYClo1zkVBg9asHz9cw== 2024-09-19T09:17:40.896 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQA07OtmogJeDBAAzuPv7ptxvSOvYIUGjLi9EA== 2024-09-19T09:17:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial monmap... 2024-09-19T09:17:41.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T09:17:41.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T09:17:41.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:17:41.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T09:17:41.752 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool for a [v2:172.21.15.16:3300,v1:172.21.15.16:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T09:17:41.752 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-09-19T09:17:41.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: set fsid to ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:17:41.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T09:17:41.752 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:17:41.753 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mon... 2024-09-19T09:17:42.716 INFO:teuthology.orchestra.run.smithi016.stdout:create mon.a on 2024-09-19T09:17:42.944 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T09:17:43.137 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T09:17:43.785 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ee1ab676-7667-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-ee1ab676-7667-11ef-bceb-c7b262605968.target. 2024-09-19T09:17:43.785 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ee1ab676-7667-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-ee1ab676-7667-11ef-bceb-c7b262605968.target. 2024-09-19T09:17:44.457 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.a 2024-09-19T09:17:44.457 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T09:17:44.781 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ee1ab676-7667-11ef-bceb-c7b262605968.target.wants/ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ee1ab676-7667-11ef-bceb-c7b262605968@.service. 2024-09-19T09:17:45.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:45 smithi016 podman[30597]: 2024-09-19 09:17:44.916506799 +0000 UTC m=+0.016450804 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:17:46.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:46 smithi016 podman[30597]: 2024-09-19 09:17:46.334837283 +0000 UTC m=+1.434781287 container create bff62411db16619bf00e2b584f089d4ae47e71aa4919cface51cc69e5ec2faf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:17:48.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:48 smithi016 podman[30597]: 2024-09-19 09:17:48.194021103 +0000 UTC m=+3.293965107 container init bff62411db16619bf00e2b584f089d4ae47e71aa4919cface51cc69e5ec2faf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T09:17:48.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:48 smithi016 podman[30597]: 2024-09-19 09:17:48.198373079 +0000 UTC m=+3.298317086 container start bff62411db16619bf00e2b584f089d4ae47e71aa4919cface51cc69e5ec2faf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T09:17:48.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:48 smithi016 ceph-mon[30616]: mkfs ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:17:48.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:48 smithi016 ceph-mon[30616]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T09:17:49.099 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T09:17:49.099 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-19T09:17:49.108 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T09:17:49.108 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-19T09:17:49.108 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-19T09:17:49.108 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T09:17:49.108 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon to start... 2024-09-19T09:17:49.109 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon... 2024-09-19T09:17:49.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:49 smithi016 bash[30597]: bff62411db16619bf00e2b584f089d4ae47e71aa4919cface51cc69e5ec2faf9 2024-09-19T09:17:49.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:49 smithi016 systemd[1]: Started Ceph mon.a for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:17:49.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:49 smithi016 ceph-mon[30616]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T09:17:49.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:49 smithi016 ceph-mon[30616]: monmap epoch 1 2024-09-19T09:17:49.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:49 smithi016 ceph-mon[30616]: fsid ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:17:49.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:49 smithi016 ceph-mon[30616]: last_changed 2024-09-19T09:17:41.219429+0000 2024-09-19T09:17:49.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:49 smithi016 ceph-mon[30616]: created 2024-09-19T09:17:41.219429+0000 2024-09-19T09:17:49.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:49 smithi016 ceph-mon[30616]: min_mon_release 19 (squid) 2024-09-19T09:17:49.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:49 smithi016 ceph-mon[30616]: election_strategy: 1 2024-09-19T09:17:49.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:49 smithi016 ceph-mon[30616]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-19T09:17:49.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:49 smithi016 ceph-mon[30616]: fsmap 2024-09-19T09:17:49.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:49 smithi016 ceph-mon[30616]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T09:17:49.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:49 smithi016 ceph-mon[30616]: mgrmap e1: no daemons active 2024-09-19T09:17:52.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:52 smithi016 ceph-mon[30616]: from='client.? 172.21.15.16:0/3954050880' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T09:17:55.228 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T09:17:55.228 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout id: ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:17:55.228 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T09:17:55.228 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-19T09:17:55.228 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout services: 2024-09-19T09:17:55.228 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 4s) 2024-09-19T09:17:55.228 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T09:17:55.229 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T09:17:55.229 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-19T09:17:55.229 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout data: 2024-09-19T09:17:55.229 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T09:17:55.229 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T09:17:55.229 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T09:17:55.229 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T09:17:55.229 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-19T09:17:55.229 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.198+0000 7fa4fa3e0640 1 Processor -- start 2024-09-19T09:17:55.229 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.198+0000 7fa4fa3e0640 1 -- start start 2024-09-19T09:17:55.229 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.198+0000 7fa4fa3e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 0x7fa4f4107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:55.229 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.198+0000 7fa4fa3e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f4108150 con 0x7fa4f4107780 2024-09-19T09:17:55.230 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.198+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 0x7fa4f4107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:55.230 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.198+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 0x7fa4f4107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57604/0 (socket says 172.21.15.16:57604) 2024-09-19T09:17:55.230 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.198+0000 7fa4f3fff640 1 -- 172.21.15.16:0/3732814224 learned_addr learned my addr 172.21.15.16:0/3732814224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:17:55.230 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.199+0000 7fa4f3fff640 1 -- 172.21.15.16:0/3732814224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f4108980 con 0x7fa4f4107780 2024-09-19T09:17:55.230 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.201+0000 7fa4f3fff640 1 --2- 172.21.15.16:0/3732814224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 0x7fa4f4107b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009820 tx=0x7fa4e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afb03cc5f816e00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:55.230 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.201+0000 7fa4f2ffd640 1 -- 172.21.15.16:0/3732814224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4e4037070 con 0x7fa4f4107780 2024-09-19T09:17:55.230 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.201+0000 7fa4f2ffd640 1 -- 172.21.15.16:0/3732814224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e4032440 con 0x7fa4f4107780 2024-09-19T09:17:55.230 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.201+0000 7fa4f2ffd640 1 -- 172.21.15.16:0/3732814224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4e4032740 con 0x7fa4f4107780 2024-09-19T09:17:55.230 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.201+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3732814224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 msgr2=0x7fa4f4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:55.230 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.201+0000 7fa4fa3e0640 1 --2- 172.21.15.16:0/3732814224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 0x7fa4f4107b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009820 tx=0x7fa4e402a270 comp rx=0 tx=0).stop 2024-09-19T09:17:55.230 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.202+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3732814224 shutdown_connections 2024-09-19T09:17:55.230 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.202+0000 7fa4fa3e0640 1 --2- 172.21.15.16:0/3732814224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 0x7fa4f4107b80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:55.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.202+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3732814224 >> 172.21.15.16:0/3732814224 conn(0x7fa4f4102f30 msgr2=0x7fa4f4105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:55.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.202+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3732814224 shutdown_connections 2024-09-19T09:17:55.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.202+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3732814224 wait complete. 2024-09-19T09:17:55.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.202+0000 7fa4fa3e0640 1 Processor -- start 2024-09-19T09:17:55.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.202+0000 7fa4fa3e0640 1 -- start start 2024-09-19T09:17:55.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.203+0000 7fa4fa3e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 0x7fa4f419aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:55.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.203+0000 7fa4fa3e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f4108eb0 con 0x7fa4f4107780 2024-09-19T09:17:55.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.203+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 0x7fa4f419aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:55.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.203+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 0x7fa4f419aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57616/0 (socket says 172.21.15.16:57616) 2024-09-19T09:17:55.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.203+0000 7fa4f3fff640 1 -- 172.21.15.16:0/3954050880 learned_addr learned my addr 172.21.15.16:0/3954050880 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:17:55.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.203+0000 7fa4f3fff640 1 -- 172.21.15.16:0/3954050880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f419b430 con 0x7fa4f4107780 2024-09-19T09:17:55.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.203+0000 7fa4f3fff640 1 --2- 172.21.15.16:0/3954050880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 0x7fa4f419aef0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40090b0 tx=0x7fa4e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:55.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.203+0000 7fa4f17fa640 1 -- 172.21.15.16:0/3954050880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4e4041070 con 0x7fa4f4107780 2024-09-19T09:17:55.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.204+0000 7fa4f17fa640 1 -- 172.21.15.16:0/3954050880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e403c7a0 con 0x7fa4f4107780 2024-09-19T09:17:55.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.204+0000 7fa4f17fa640 1 -- 172.21.15.16:0/3954050880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4e4037040 con 0x7fa4f4107780 2024-09-19T09:17:55.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.204+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3954050880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f419b6c0 con 0x7fa4f4107780 2024-09-19T09:17:55.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.204+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3954050880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f419e3b0 con 0x7fa4f4107780 2024-09-19T09:17:55.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.204+0000 7fa4f17fa640 1 -- 172.21.15.16:0/3954050880 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa4e4044070 con 0x7fa4f4107780 2024-09-19T09:17:55.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.204+0000 7fa4f17fa640 1 -- 172.21.15.16:0/3954050880 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa4e403d3f0 con 0x7fa4f4107780 2024-09-19T09:17:55.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.205+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3954050880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f4109500 con 0x7fa4f4107780 2024-09-19T09:17:55.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.207+0000 7fa4f17fa640 1 -- 172.21.15.16:0/3954050880 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa4e4046450 con 0x7fa4f4107780 2024-09-19T09:17:55.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.270+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3954050880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa4f402d900 con 0x7fa4f4107780 2024-09-19T09:17:55.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.271+0000 7fa4f17fa640 1 -- 172.21.15.16:0/3954050880 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+310 (secure 0 0 0) 0x7fa4e4046630 con 0x7fa4f4107780 2024-09-19T09:17:55.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.271+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3954050880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 msgr2=0x7fa4f419aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:55.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.271+0000 7fa4fa3e0640 1 --2- 172.21.15.16:0/3954050880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 0x7fa4f419aef0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40090b0 tx=0x7fa4e40046c0 comp rx=0 tx=0).stop 2024-09-19T09:17:55.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.271+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3954050880 shutdown_connections 2024-09-19T09:17:55.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.271+0000 7fa4fa3e0640 1 --2- 172.21.15.16:0/3954050880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4f4107780 0x7fa4f419aef0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:55.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.271+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3954050880 >> 172.21.15.16:0/3954050880 conn(0x7fa4f4102f30 msgr2=0x7fa4f418f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:55.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.271+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3954050880 shutdown_connections 2024-09-19T09:17:55.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:52.271+0000 7fa4fa3e0640 1 -- 172.21.15.16:0/3954050880 wait complete. 2024-09-19T09:17:55.233 INFO:teuthology.orchestra.run.smithi016.stdout:mon is available 2024-09-19T09:17:55.233 INFO:teuthology.orchestra.run.smithi016.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T09:17:56.002 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:55 smithi016 ceph-mon[30616]: from='client.? 172.21.15.16:0/1929131670' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T09:17:56.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:55 smithi016 ceph-mon[30616]: from='client.? 172.21.15.16:0/1929131670' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T09:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-19T09:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T09:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-19T09:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T09:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T09:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T09:17:56.293 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.659+0000 7f0704074640 1 Processor -- start 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.659+0000 7f0704074640 1 -- start start 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.659+0000 7f0704074640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 0x7f06fc10a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.659+0000 7f0704074640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc07be30 con 0x7f06fc108140 2024-09-19T09:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.659+0000 7f0701e10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 0x7f06fc10a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.659+0000 7f0701e10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 0x7f06fc10a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39862/0 (socket says 172.21.15.16:39862) 2024-09-19T09:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.659+0000 7f0701e10640 1 -- 172.21.15.16:0/587557114 learned_addr learned my addr 172.21.15.16:0/587557114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.659+0000 7f0701e10640 1 -- 172.21.15.16:0/587557114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc10aa70 con 0x7f06fc108140 2024-09-19T09:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.661+0000 7f0701e10640 1 --2- 172.21.15.16:0/587557114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 0x7f06fc10a530 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009820 tx=0x7f06f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4baf6ca2254da10f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.661+0000 7f0700e0e640 1 -- 172.21.15.16:0/587557114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f8037070 con 0x7f06fc108140 2024-09-19T09:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.661+0000 7f0700e0e640 1 -- 172.21.15.16:0/587557114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f8032440 con 0x7f06fc108140 2024-09-19T09:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.661+0000 7f0700e0e640 1 -- 172.21.15.16:0/587557114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f8032740 con 0x7f06fc108140 2024-09-19T09:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.662+0000 7f0704074640 1 -- 172.21.15.16:0/587557114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 msgr2=0x7f06fc10a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.662+0000 7f0704074640 1 --2- 172.21.15.16:0/587557114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 0x7f06fc10a530 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009820 tx=0x7f06f802a270 comp rx=0 tx=0).stop 2024-09-19T09:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.662+0000 7f0704074640 1 -- 172.21.15.16:0/587557114 shutdown_connections 2024-09-19T09:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.662+0000 7f0704074640 1 --2- 172.21.15.16:0/587557114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 0x7f06fc10a530 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:56.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.662+0000 7f0704074640 1 -- 172.21.15.16:0/587557114 >> 172.21.15.16:0/587557114 conn(0x7f06fc101e20 msgr2=0x7f06fc104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.662+0000 7f0704074640 1 -- 172.21.15.16:0/587557114 shutdown_connections 2024-09-19T09:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.662+0000 7f0704074640 1 -- 172.21.15.16:0/587557114 wait complete. 2024-09-19T09:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f0704074640 1 Processor -- start 2024-09-19T09:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f0704074640 1 -- start start 2024-09-19T09:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f0704074640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 0x7f06fc19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f0704074640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc10afb0 con 0x7f06fc108140 2024-09-19T09:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f0701e10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 0x7f06fc19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f0701e10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 0x7f06fc19aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39876/0 (socket says 172.21.15.16:39876) 2024-09-19T09:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f0701e10640 1 -- 172.21.15.16:0/1929131670 learned_addr learned my addr 172.21.15.16:0/1929131670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f0701e10640 1 -- 172.21.15.16:0/1929131670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc19b410 con 0x7f06fc108140 2024-09-19T09:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f0701e10640 1 --2- 172.21.15.16:0/1929131670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 0x7f06fc19aed0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f06f80090b0 tx=0x7f06f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f06f2ffd640 1 -- 172.21.15.16:0/1929131670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f8041070 con 0x7f06fc108140 2024-09-19T09:17:56.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f06f2ffd640 1 -- 172.21.15.16:0/1929131670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f803c7a0 con 0x7f06fc108140 2024-09-19T09:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f06f2ffd640 1 -- 172.21.15.16:0/1929131670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f8037040 con 0x7f06fc108140 2024-09-19T09:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f0704074640 1 -- 172.21.15.16:0/1929131670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06fc19b6a0 con 0x7f06fc108140 2024-09-19T09:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f0704074640 1 -- 172.21.15.16:0/1929131670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06fc19e390 con 0x7f06fc108140 2024-09-19T09:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f06f2ffd640 1 -- 172.21.15.16:0/1929131670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f06f8044070 con 0x7f06fc108140 2024-09-19T09:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.663+0000 7f06f2ffd640 1 -- 172.21.15.16:0/1929131670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f06f803d3f0 con 0x7f06fc108140 2024-09-19T09:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.665+0000 7f0704074640 1 -- 172.21.15.16:0/1929131670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06fc1085a0 con 0x7f06fc108140 2024-09-19T09:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.667+0000 7f06f2ffd640 1 -- 172.21.15.16:0/1929131670 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f06f8046450 con 0x7f06fc108140 2024-09-19T09:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.727+0000 7f0704074640 1 -- 172.21.15.16:0/1929131670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f06fc02d910 con 0x7f06fc108140 2024-09-19T09:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.730+0000 7f06f2ffd640 1 -- 172.21.15.16:0/1929131670 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f06f8046630 con 0x7f06fc108140 2024-09-19T09:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.730+0000 7f06f2ffd640 1 -- 172.21.15.16:0/1929131670 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f06f8056070 con 0x7f06fc108140 2024-09-19T09:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.731+0000 7f0704074640 1 -- 172.21.15.16:0/1929131670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 msgr2=0x7f06fc19aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.731+0000 7f0704074640 1 --2- 172.21.15.16:0/1929131670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 0x7f06fc19aed0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f06f80090b0 tx=0x7f06f80046c0 comp rx=0 tx=0).stop 2024-09-19T09:17:56.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.731+0000 7f0704074640 1 -- 172.21.15.16:0/1929131670 shutdown_connections 2024-09-19T09:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.731+0000 7f0704074640 1 --2- 172.21.15.16:0/1929131670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06fc108140 0x7f06fc19aed0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.731+0000 7f0704074640 1 -- 172.21.15.16:0/1929131670 >> 172.21.15.16:0/1929131670 conn(0x7f06fc101e20 msgr2=0x7f06fc102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.731+0000 7f0704074640 1 -- 172.21.15.16:0/1929131670 shutdown_connections 2024-09-19T09:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:55.731+0000 7f0704074640 1 -- 172.21.15.16:0/1929131670 wait complete. 2024-09-19T09:17:56.298 INFO:teuthology.orchestra.run.smithi016.stdout:Generating new minimal ceph.conf... 2024-09-19T09:17:57.002 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:56 smithi016 ceph-mon[30616]: from='client.? 172.21.15.16:0/2106687842' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:17:57.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.685+0000 7f853c791640 1 Processor -- start 2024-09-19T09:17:57.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.685+0000 7f853c791640 1 -- start start 2024-09-19T09:17:57.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.685+0000 7f853c791640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 0x7f853410a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.685+0000 7f853c791640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f853407bdd0 con 0x7f85341081e0 2024-09-19T09:17:57.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.685+0000 7f853a52d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 0x7f853410a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.685+0000 7f853a52d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 0x7f853410a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39880/0 (socket says 172.21.15.16:39880) 2024-09-19T09:17:57.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.685+0000 7f853a52d640 1 -- 172.21.15.16:0/2473610714 learned_addr learned my addr 172.21.15.16:0/2473610714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:17:57.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.686+0000 7f853a52d640 1 -- 172.21.15.16:0/2473610714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853410ab10 con 0x7f85341081e0 2024-09-19T09:17:57.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.686+0000 7f853a52d640 1 --2- 172.21.15.16:0/2473610714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 0x7f853410a5d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8530009820 tx=0x7f853002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c6786846fda43d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:57.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.687+0000 7f853952b640 1 -- 172.21.15.16:0/2473610714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8530037070 con 0x7f85341081e0 2024-09-19T09:17:57.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.687+0000 7f853952b640 1 -- 172.21.15.16:0/2473610714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8530032900 con 0x7f85341081e0 2024-09-19T09:17:57.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.687+0000 7f853952b640 1 -- 172.21.15.16:0/2473610714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8530030850 con 0x7f85341081e0 2024-09-19T09:17:57.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.687+0000 7f853c791640 1 -- 172.21.15.16:0/2473610714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 msgr2=0x7f853410a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:57.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.687+0000 7f853c791640 1 --2- 172.21.15.16:0/2473610714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 0x7f853410a5d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8530009820 tx=0x7f853002a270 comp rx=0 tx=0).stop 2024-09-19T09:17:57.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.687+0000 7f853c791640 1 -- 172.21.15.16:0/2473610714 shutdown_connections 2024-09-19T09:17:57.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.687+0000 7f853c791640 1 --2- 172.21.15.16:0/2473610714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 0x7f853410a5d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.687+0000 7f853c791640 1 -- 172.21.15.16:0/2473610714 >> 172.21.15.16:0/2473610714 conn(0x7f8534101e20 msgr2=0x7f8534104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:57.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.687+0000 7f853c791640 1 -- 172.21.15.16:0/2473610714 shutdown_connections 2024-09-19T09:17:57.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.687+0000 7f853c791640 1 -- 172.21.15.16:0/2473610714 wait complete. 2024-09-19T09:17:57.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.688+0000 7f853c791640 1 Processor -- start 2024-09-19T09:17:57.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.688+0000 7f853c791640 1 -- start start 2024-09-19T09:17:57.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.688+0000 7f853c791640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 0x7f853419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:57.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.688+0000 7f853c791640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f853410bb50 con 0x7f85341081e0 2024-09-19T09:17:57.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.688+0000 7f853a52d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 0x7f853419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:57.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.688+0000 7f853a52d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 0x7f853419b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39890/0 (socket says 172.21.15.16:39890) 2024-09-19T09:17:57.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.688+0000 7f853a52d640 1 -- 172.21.15.16:0/2106687842 learned_addr learned my addr 172.21.15.16:0/2106687842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:17:57.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.688+0000 7f853a52d640 1 -- 172.21.15.16:0/2106687842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853419bc50 con 0x7f85341081e0 2024-09-19T09:17:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.689+0000 7f853a52d640 1 --2- 172.21.15.16:0/2106687842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 0x7f853419b710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f85300097f0 tx=0x7f8530030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.689+0000 7f85277fe640 1 -- 172.21.15.16:0/2106687842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8530037040 con 0x7f85341081e0 2024-09-19T09:17:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.689+0000 7f85277fe640 1 -- 172.21.15.16:0/2106687842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8530040070 con 0x7f85341081e0 2024-09-19T09:17:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.689+0000 7f853c791640 1 -- 172.21.15.16:0/2106687842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853419bee0 con 0x7f85341081e0 2024-09-19T09:17:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.689+0000 7f85277fe640 1 -- 172.21.15.16:0/2106687842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f853003a390 con 0x7f85341081e0 2024-09-19T09:17:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.689+0000 7f853c791640 1 -- 172.21.15.16:0/2106687842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853419ebd0 con 0x7f85341081e0 2024-09-19T09:17:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.689+0000 7f85277fe640 1 -- 172.21.15.16:0/2106687842 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f853003a5e0 con 0x7f85341081e0 2024-09-19T09:17:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.689+0000 7f85277fe640 1 -- 172.21.15.16:0/2106687842 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8530045950 con 0x7f85341081e0 2024-09-19T09:17:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.690+0000 7f853c791640 1 -- 172.21.15.16:0/2106687842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8534108640 con 0x7f85341081e0 2024-09-19T09:17:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.692+0000 7f85277fe640 1 -- 172.21.15.16:0/2106687842 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8530045460 con 0x7f85341081e0 2024-09-19T09:17:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.752+0000 7f853c791640 1 -- 172.21.15.16:0/2106687842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f853419eda0 con 0x7f85341081e0 2024-09-19T09:17:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.753+0000 7f85277fe640 1 -- 172.21.15.16:0/2106687842 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f853003b430 con 0x7f85341081e0 2024-09-19T09:17:57.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.753+0000 7f853c791640 1 -- 172.21.15.16:0/2106687842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 msgr2=0x7f853419b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:57.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.753+0000 7f853c791640 1 --2- 172.21.15.16:0/2106687842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 0x7f853419b710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f85300097f0 tx=0x7f8530030d80 comp rx=0 tx=0).stop 2024-09-19T09:17:57.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.753+0000 7f853c791640 1 -- 172.21.15.16:0/2106687842 shutdown_connections 2024-09-19T09:17:57.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.753+0000 7f853c791640 1 --2- 172.21.15.16:0/2106687842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85341081e0 0x7f853419b710 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:57.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.753+0000 7f853c791640 1 -- 172.21.15.16:0/2106687842 >> 172.21.15.16:0/2106687842 conn(0x7f8534101e20 msgr2=0x7f8534102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:57.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.753+0000 7f853c791640 1 -- 172.21.15.16:0/2106687842 shutdown_connections 2024-09-19T09:17:57.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:56.753+0000 7f853c791640 1 -- 172.21.15.16:0/2106687842 wait complete. 2024-09-19T09:17:57.276 INFO:teuthology.orchestra.run.smithi016.stdout:Restarting the monitor... 2024-09-19T09:17:57.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:57 smithi016 systemd[1]: Stopping Ceph mon.a for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:17:57.968 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:57 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a[30612]: 2024-09-19T09:17:57.584+0000 7ff92946c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:17:57.969 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:57 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a[30612]: 2024-09-19T09:17:57.584+0000 7ff92946c640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T09:17:57.969 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:57 smithi016 podman[30856]: 2024-09-19 09:17:57.769282573 +0000 UTC m=+0.306515488 container died bff62411db16619bf00e2b584f089d4ae47e71aa4919cface51cc69e5ec2faf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:17:58.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 podman[30856]: 2024-09-19 09:17:58.059605524 +0000 UTC m=+0.596838442 container cleanup bff62411db16619bf00e2b584f089d4ae47e71aa4919cface51cc69e5ec2faf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T09:17:58.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 bash[30856]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a 2024-09-19T09:17:58.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 podman[30869]: 2024-09-19 09:17:58.285519377 +0000 UTC m=+0.513183779 container remove bff62411db16619bf00e2b584f089d4ae47e71aa4919cface51cc69e5ec2faf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:17:58.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 systemd[1]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T09:17:58.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 systemd[1]: Stopped Ceph mon.a for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:17:58.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 systemd[1]: Starting Ceph mon.a for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:17:58.621 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 podman[30960]: 2024-09-19 09:17:58.619822241 +0000 UTC m=+0.110253803 container create 844da19cfdd1689faec127b82d749a038d28ebc637edc44abbacd02751e9942b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T09:17:58.897 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 podman[30960]: 2024-09-19 09:17:58.52589444 +0000 UTC m=+0.016325977 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T09:17:58.897 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 podman[30960]: 2024-09-19 09:17:58.822219322 +0000 UTC m=+0.312650852 container init 844da19cfdd1689faec127b82d749a038d28ebc637edc44abbacd02751e9942b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T09:17:58.897 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 podman[30960]: 2024-09-19 09:17:58.825530412 +0000 UTC m=+0.315961931 container start 844da19cfdd1689faec127b82d749a038d28ebc637edc44abbacd02751e9942b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-19T09:17:58.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T09:17:58.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T09:17:58.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: pidfile_write: ignore empty --pid-file 2024-09-19T09:17:58.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: load: jerasure load: lrc 2024-09-19T09:17:58.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: RocksDB version: 7.9.2 2024-09-19T09:17:58.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Git sha 0 2024-09-19T09:17:58.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T09:17:58.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: DB SUMMARY 2024-09-19T09:17:58.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: DB Session ID: NL4QEG7I4JEGG9M74IZI 2024-09-19T09:17:58.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: CURRENT file: CURRENT 2024-09-19T09:17:58.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T09:17:58.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T09:17:58.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T09:17:58.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106483 ; 2024-09-19T09:17:58.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.error_if_exists: 0 2024-09-19T09:17:58.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.create_if_missing: 0 2024-09-19T09:17:58.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.paranoid_checks: 1 2024-09-19T09:17:58.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T09:17:58.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T09:17:58.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T09:17:58.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.env: 0x559bc315c940 2024-09-19T09:17:58.899 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T09:17:58.900 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.info_log: 0x559bc3b456e0 2024-09-19T09:17:58.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T09:17:58.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.statistics: (nil) 2024-09-19T09:17:58.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.use_fsync: 0 2024-09-19T09:17:58.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_log_file_size: 0 2024-09-19T09:17:58.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T09:17:58.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T09:17:58.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T09:17:58.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T09:17:58.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.allow_fallocate: 1 2024-09-19T09:17:58.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T09:17:58.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T09:17:58.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.use_direct_reads: 0 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.db_log_dir: 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.wal_dir: 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.write_buffer_manager: 0x559bc3b54dc0 2024-09-19T09:17:58.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.unordered_write: 0 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.row_cache: None 2024-09-19T09:17:58.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.wal_filter: None 2024-09-19T09:17:58.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T09:17:58.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T09:17:58.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.two_write_queues: 0 2024-09-19T09:17:58.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T09:17:58.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.wal_compression: 0 2024-09-19T09:17:58.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.atomic_flush: 0 2024-09-19T09:17:58.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T09:17:58.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T09:17:58.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T09:17:58.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.log_readahead_size: 0 2024-09-19T09:17:58.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T09:17:58.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T09:17:58.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T09:17:58.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T09:17:58.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T09:17:58.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T09:17:58.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T09:17:58.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_background_jobs: 2 2024-09-19T09:17:58.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_background_compactions: -1 2024-09-19T09:17:58.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_subcompactions: 1 2024-09-19T09:17:58.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T09:17:58.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T09:17:58.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T09:17:58.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T09:17:58.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T09:17:58.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T09:17:58.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T09:17:58.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T09:17:58.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_open_files: -1 2024-09-19T09:17:58.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T09:17:58.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T09:17:58.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T09:17:58.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T09:17:58.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_background_flushes: -1 2024-09-19T09:17:58.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Compression algorithms supported: 2024-09-19T09:17:58.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: kZSTD supported: 0 2024-09-19T09:17:58.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: kXpressCompression supported: 0 2024-09-19T09:17:58.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: kBZip2Compression supported: 0 2024-09-19T09:17:58.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T09:17:58.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: kLZ4Compression supported: 1 2024-09-19T09:17:58.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: kZlibCompression supported: 1 2024-09-19T09:17:58.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T09:17:58.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: kSnappyCompression supported: 1 2024-09-19T09:17:58.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T09:17:58.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T09:17:58.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T09:17:58.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T09:17:58.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T09:17:58.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.merge_operator: 2024-09-19T09:17:58.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compaction_filter: None 2024-09-19T09:17:58.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T09:17:58.907 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T09:17:58.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T09:17:58.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T09:17:58.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559bc3b45320) 2024-09-19T09:17:58.908 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-09-19T09:17:58.908 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T09:17:58.908 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T09:17:58.908 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-09-19T09:17:58.908 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_type: 0 2024-09-19T09:17:58.908 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_index_type: 0 2024-09-19T09:17:58.908 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_shortening: 1 2024-09-19T09:17:58.908 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T09:17:58.908 INFO:journalctl@ceph.mon.a.smithi016.stdout: checksum: 4 2024-09-19T09:17:58.908 INFO:journalctl@ceph.mon.a.smithi016.stdout: no_block_cache: 0 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache: 0x559bc3b391f0 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_options: 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: capacity : 536870912 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_shard_bits : 4 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: strict_capacity_limit : 0 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_compressed: (nil) 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: persistent_cache: (nil) 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size: 4096 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size_deviation: 10 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_restart_interval: 16 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_block_restart_interval: 1 2024-09-19T09:17:58.909 INFO:journalctl@ceph.mon.a.smithi016.stdout: metadata_block_size: 4096 2024-09-19T09:17:58.910 INFO:journalctl@ceph.mon.a.smithi016.stdout: partition_filters: 0 2024-09-19T09:17:58.910 INFO:journalctl@ceph.mon.a.smithi016.stdout: use_delta_encoding: 1 2024-09-19T09:17:58.910 INFO:journalctl@ceph.mon.a.smithi016.stdout: filter_policy: bloomfilter 2024-09-19T09:17:58.910 INFO:journalctl@ceph.mon.a.smithi016.stdout: whole_key_filtering: 1 2024-09-19T09:17:58.910 INFO:journalctl@ceph.mon.a.smithi016.stdout: verify_compression: 0 2024-09-19T09:17:58.910 INFO:journalctl@ceph.mon.a.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-09-19T09:17:58.910 INFO:journalctl@ceph.mon.a.smithi016.stdout: format_version: 5 2024-09-19T09:17:58.910 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_index_compression: 1 2024-09-19T09:17:58.910 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_align: 0 2024-09-19T09:17:58.910 INFO:journalctl@ceph.mon.a.smithi016.stdout: max_auto_readahead_size: 262144 2024-09-19T09:17:58.910 INFO:journalctl@ceph.mon.a.smithi016.stdout: prepopulate_block_cache: 0 2024-09-19T09:17:58.910 INFO:journalctl@ceph.mon.a.smithi016.stdout: initial_auto_readahead_size: 8192 2024-09-19T09:17:58.910 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T09:17:58.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T09:17:58.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T09:17:58.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compression: NoCompression 2024-09-19T09:17:58.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T09:17:58.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T09:17:58.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T09:17:58.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.num_levels: 7 2024-09-19T09:17:58.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T09:17:58.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T09:17:58.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T09:17:58.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T09:17:58.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T09:17:58.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T09:17:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T09:17:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T09:17:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T09:17:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T09:17:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T09:17:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T09:17:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T09:17:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T09:17:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T09:17:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T09:17:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T09:17:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T09:17:58.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T09:17:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T09:17:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T09:17:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T09:17:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T09:17:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T09:17:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T09:17:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T09:17:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T09:17:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T09:17:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T09:17:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T09:17:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T09:17:58.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T09:17:58.914 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T09:17:58.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T09:17:58.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T09:17:58.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T09:17:58.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T09:17:58.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T09:17:58.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T09:17:58.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T09:17:58.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T09:17:58.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.inplace_update_support: 0 2024-09-19T09:17:58.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T09:17:58.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T09:17:58.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T09:17:58.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.bloom_locality: 0 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.max_successive_merges: 0 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.ttl: 2592000 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.enable_blob_files: false 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.min_blob_size: 0 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T09:17:58.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T09:17:58.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T09:17:58.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T09:17:58.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T09:17:58.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T09:17:58.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T09:17:58.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T09:17:58.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T09:17:58.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T09:17:58.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1ca33d69-d03c-407a-866d-e16a2937dd47 2024-09-19T09:17:58.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726737478852282, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T09:17:58.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T09:17:58.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726737478853876, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101307, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 338, "table_properties": {"data_size": 99192, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14560, "raw_average_key_size": 49, "raw_value_size": 91314, "raw_average_value_size": 311, "num_data_blocks": 12, "num_entries": 293, "num_filter_entries": 293, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726737478, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1ca33d69-d03c-407a-866d-e16a2937dd47", "db_session_id": "NL4QEG7I4JEGG9M74IZI", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T09:17:58.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726737478854018, "job": 1, "event": "recovery_finished"} 2024-09-19T09:17:58.918 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T09:17:58.918 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T09:17:58.918 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559bc3b66e00 2024-09-19T09:17:58.918 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: DB pointer 0x559bc3c62000 2024-09-19T09:17:58.918 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T09:17:58.918 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T09:17:58.918 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** DB Stats ** 2024-09-19T09:17:58.918 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T09:17:58.918 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T09:17:58.918 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T09:17:58.918 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T09:17:58.918 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T09:17:58.918 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: L0 2/0 100.79 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: Sum 2/0 100.79 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 102.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T09:17:58.919 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-19T09:17:58.920 INFO:journalctl@ceph.mon.a.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T09:17:58.920 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-19T09:17:58.920 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T09:17:58.920 INFO:journalctl@ceph.mon.a.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T09:17:58.920 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T09:17:58.920 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T09:17:58.920 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T09:17:58.920 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T09:17:58.920 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative compaction: 0.00 GB write, 20.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T09:17:58.920 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval compaction: 0.00 GB write, 20.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T09:17:58.920 INFO:journalctl@ceph.mon.a.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T09:17:58.920 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache BinnedLRUCache@0x559bc3b391f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.8e-05 secs_since: 0 2024-09-19T09:17:58.920 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T09:17:58.921 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-19T09:17:58.921 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T09:17:58.921 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: starting mon.a rank 0 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:17:58.921 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: mon.a@-1(???) e1 preinit fsid ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:17:58.921 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: mon.a@-1(???).mds e1 new map 2024-09-19T09:17:58.921 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: mon.a@-1(???).mds e1 print_map 2024-09-19T09:17:58.921 INFO:journalctl@ceph.mon.a.smithi016.stdout: e1 2024-09-19T09:17:58.921 INFO:journalctl@ceph.mon.a.smithi016.stdout: btime 2024-09-19T09:17:48:242055+0000 2024-09-19T09:17:58.921 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T09:17:58.921 INFO:journalctl@ceph.mon.a.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T09:17:58.921 INFO:journalctl@ceph.mon.a.smithi016.stdout: legacy client fscid: -1 2024-09-19T09:17:58.921 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-19T09:17:58.921 INFO:journalctl@ceph.mon.a.smithi016.stdout: No filesystems configured 2024-09-19T09:17:58.922 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T09:17:58.922 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T09:17:58.922 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T09:17:58.922 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T09:17:58.922 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T09:17:58.922 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 bash[30960]: 844da19cfdd1689faec127b82d749a038d28ebc637edc44abbacd02751e9942b 2024-09-19T09:17:58.922 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 systemd[1]: Started Ceph mon.a for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:17:58.922 INFO:teuthology.orchestra.run.smithi016.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T09:17:59.208 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T09:17:59.208 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: monmap epoch 1 2024-09-19T09:17:59.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: fsid ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:17:59.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: last_changed 2024-09-19T09:17:41.219429+0000 2024-09-19T09:17:59.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: created 2024-09-19T09:17:41.219429+0000 2024-09-19T09:17:59.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: min_mon_release 19 (squid) 2024-09-19T09:17:59.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: election_strategy: 1 2024-09-19T09:17:59.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-19T09:17:59.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: fsmap 2024-09-19T09:17:59.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T09:17:59.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:17:58 smithi016 ceph-mon[30979]: mgrmap e1: no daemons active 2024-09-19T09:17:59.860 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.292+0000 7fa7b825c640 1 Processor -- start 2024-09-19T09:17:59.860 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.292+0000 7fa7b825c640 1 -- start start 2024-09-19T09:17:59.860 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.292+0000 7fa7b825c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 0x7fa7b0107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:59.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.292+0000 7fa7b825c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b0108170 con 0x7fa7b01077a0 2024-09-19T09:17:59.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.293+0000 7fa7b5ff8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 0x7fa7b0107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:59.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.293+0000 7fa7b5ff8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 0x7fa7b0107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39906/0 (socket says 172.21.15.16:39906) 2024-09-19T09:17:59.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.293+0000 7fa7b5ff8640 1 -- 172.21.15.16:0/293660878 learned_addr learned my addr 172.21.15.16:0/293660878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:17:59.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.293+0000 7fa7b5ff8640 1 -- 172.21.15.16:0/293660878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b01089a0 con 0x7fa7b01077a0 2024-09-19T09:17:59.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.295+0000 7fa7b5ff8640 1 --2- 172.21.15.16:0/293660878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 0x7fa7b0107ba0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009e80 tx=0x7fa7ac02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae941a18405fdcf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:59.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.295+0000 7fa7b4ff6640 1 -- 172.21.15.16:0/293660878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7ac036070 con 0x7fa7b01077a0 2024-09-19T09:17:59.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.295+0000 7fa7b4ff6640 1 -- 172.21.15.16:0/293660878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa7ac0318c0 con 0x7fa7b01077a0 2024-09-19T09:17:59.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.295+0000 7fa7b4ff6640 1 -- 172.21.15.16:0/293660878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7ac007ab0 con 0x7fa7b01077a0 2024-09-19T09:17:59.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.296+0000 7fa7b825c640 1 -- 172.21.15.16:0/293660878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 msgr2=0x7fa7b0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:59.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.296+0000 7fa7b825c640 1 --2- 172.21.15.16:0/293660878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 0x7fa7b0107ba0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009e80 tx=0x7fa7ac02a7a0 comp rx=0 tx=0).stop 2024-09-19T09:17:59.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.296+0000 7fa7b825c640 1 -- 172.21.15.16:0/293660878 shutdown_connections 2024-09-19T09:17:59.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.296+0000 7fa7b825c640 1 --2- 172.21.15.16:0/293660878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 0x7fa7b0107ba0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.296+0000 7fa7b825c640 1 -- 172.21.15.16:0/293660878 >> 172.21.15.16:0/293660878 conn(0x7fa7b0102f70 msgr2=0x7fa7b0105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.296+0000 7fa7b825c640 1 -- 172.21.15.16:0/293660878 shutdown_connections 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.296+0000 7fa7b825c640 1 -- 172.21.15.16:0/293660878 wait complete. 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.297+0000 7fa7b825c640 1 Processor -- start 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.297+0000 7fa7b825c640 1 -- start start 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.297+0000 7fa7b825c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 0x7fa7b019b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.297+0000 7fa7b825c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b010bd50 con 0x7fa7b01077a0 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.297+0000 7fa7b5ff8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 0x7fa7b019b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.297+0000 7fa7b5ff8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 0x7fa7b019b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39908/0 (socket says 172.21.15.16:39908) 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.297+0000 7fa7b5ff8640 1 -- 172.21.15.16:0/2682960280 learned_addr learned my addr 172.21.15.16:0/2682960280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.297+0000 7fa7b5ff8640 1 -- 172.21.15.16:0/2682960280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b019bc60 con 0x7fa7b01077a0 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.298+0000 7fa7b5ff8640 1 --2- 172.21.15.16:0/2682960280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 0x7fa7b019b720 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac02ad90 tx=0x7fa7ac031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.298+0000 7fa7a6ffd640 1 -- 172.21.15.16:0/2682960280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7ac036050 con 0x7fa7b01077a0 2024-09-19T09:17:59.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.298+0000 7fa7a6ffd640 1 -- 172.21.15.16:0/2682960280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa7ac042070 con 0x7fa7b01077a0 2024-09-19T09:17:59.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.298+0000 7fa7b825c640 1 -- 172.21.15.16:0/2682960280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b019bef0 con 0x7fa7b01077a0 2024-09-19T09:17:59.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.298+0000 7fa7a6ffd640 1 -- 172.21.15.16:0/2682960280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7ac03d7e0 con 0x7fa7b01077a0 2024-09-19T09:17:59.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.298+0000 7fa7b825c640 1 -- 172.21.15.16:0/2682960280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b019ebe0 con 0x7fa7b01077a0 2024-09-19T09:17:59.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.298+0000 7fa7a6ffd640 1 -- 172.21.15.16:0/2682960280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa7ac03da90 con 0x7fa7b01077a0 2024-09-19T09:17:59.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.298+0000 7fa7a6ffd640 1 -- 172.21.15.16:0/2682960280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa7ac007e00 con 0x7fa7b01077a0 2024-09-19T09:17:59.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.299+0000 7fa7b825c640 1 -- 172.21.15.16:0/2682960280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b0107c20 con 0x7fa7b01077a0 2024-09-19T09:17:59.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.301+0000 7fa7a6ffd640 1 -- 172.21.15.16:0/2682960280 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa7ac0072f0 con 0x7fa7b01077a0 2024-09-19T09:17:59.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.360+0000 7fa7b825c640 1 -- 172.21.15.16:0/2682960280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fa7b019f1c0 con 0x7fa7b01077a0 2024-09-19T09:17:59.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.362+0000 7fa7a6ffd640 1 -- 172.21.15.16:0/2682960280 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fa7ac047460 con 0x7fa7b01077a0 2024-09-19T09:17:59.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.363+0000 7fa7b825c640 1 -- 172.21.15.16:0/2682960280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 msgr2=0x7fa7b019b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:17:59.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.363+0000 7fa7b825c640 1 --2- 172.21.15.16:0/2682960280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 0x7fa7b019b720 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac02ad90 tx=0x7fa7ac031ca0 comp rx=0 tx=0).stop 2024-09-19T09:17:59.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.363+0000 7fa7b825c640 1 -- 172.21.15.16:0/2682960280 shutdown_connections 2024-09-19T09:17:59.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.363+0000 7fa7b825c640 1 --2- 172.21.15.16:0/2682960280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b01077a0 0x7fa7b019b720 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:17:59.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.363+0000 7fa7b825c640 1 -- 172.21.15.16:0/2682960280 >> 172.21.15.16:0/2682960280 conn(0x7fa7b0102f70 msgr2=0x7fa7b0103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:17:59.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.363+0000 7fa7b825c640 1 -- 172.21.15.16:0/2682960280 shutdown_connections 2024-09-19T09:17:59.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:17:59.363+0000 7fa7b825c640 1 -- 172.21.15.16:0/2682960280 wait complete. 2024-09-19T09:17:59.864 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T09:18:00.085 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:18:00.085 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mgr... 2024-09-19T09:18:00.085 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T09:18:00.086 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T09:18:00.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:00 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2682960280' entity='client.admin' 2024-09-19T09:18:01.192 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.x 2024-09-19T09:18:01.192 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T09:18:01.437 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ee1ab676-7667-11ef-bceb-c7b262605968.target.wants/ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-ee1ab676-7667-11ef-bceb-c7b262605968@.service. 2024-09-19T09:18:01.753 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:01 smithi016 podman[31193]: 2024-09-19 09:18:01.694337169 +0000 UTC m=+0.118628323 container create ef968a83280163064e30438e5e73405fcd1250f9c82c24a4d89c8ff7e681217c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:18:02.020 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T09:18:02.020 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-19T09:18:02.028 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T09:18:02.028 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-19T09:18:02.028 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-19T09:18:02.028 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T09:18:02.037 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T09:18:02.037 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-19T09:18:02.045 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T09:18:02.045 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-19T09:18:02.045 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-19T09:18:02.045 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T09:18:02.045 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr to start... 2024-09-19T09:18:02.045 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr... 2024-09-19T09:18:02.087 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:01 smithi016 podman[31193]: 2024-09-19 09:18:01.896525776 +0000 UTC m=+0.320816901 container init ef968a83280163064e30438e5e73405fcd1250f9c82c24a4d89c8ff7e681217c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True) 2024-09-19T09:18:02.088 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:01 smithi016 podman[31193]: 2024-09-19 09:18:01.899220367 +0000 UTC m=+0.323511480 container start ef968a83280163064e30438e5e73405fcd1250f9c82c24a4d89c8ff7e681217c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:18:02.088 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:02 smithi016 bash[31193]: ef968a83280163064e30438e5e73405fcd1250f9c82c24a4d89c8ff7e681217c 2024-09-19T09:18:02.088 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:02 smithi016 systemd[1]: Started Ceph mgr.x for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:18:02.088 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:02 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:02.023+0000 7fd5f3378040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T09:18:02.088 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:02 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:02.086+0000 7fd5f3378040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T09:18:02.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:02 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:02.244+0000 7fd5f3378040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T09:18:02.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:02 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:02.313+0000 7fd5f3378040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T09:18:02.624 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:02 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:02.473+0000 7fd5f3378040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:18:02.624 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:02 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:02.622+0000 7fd5f3378040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T09:18:02.963 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:02 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2267198823' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T09:18:03.222 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:02 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:02.962+0000 7fd5f3378040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T09:18:03.223 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:03 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:03.026+0000 7fd5f3378040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T09:18:03.223 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:03 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:03.221+0000 7fd5f3378040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T09:18:03.260 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-19T09:18:03.260 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:18:03.260 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:18:03.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T09:18:03.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T09:18:03.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T09:18:03.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T09:18:03.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:03.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T09:18:03.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T09:18:03.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-19T09:18:03.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:18:03.261 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T09:18:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T09:18:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:18:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-19T09:18:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T09:18:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:18:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T09:18:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T09:18:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T09:18:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:18:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T09:18:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T09:18:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T09:18:03.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T09:18:03.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T09:18:03.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T09:18:03.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:03.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T09:18:03.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T09:18:03.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T09:18:03.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T09:18:03.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T09:18:03.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T09:18:03.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T09:18:03.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T09:18:03.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T09:18:03.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:03.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T09:18:03.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:18:03.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T09:17:48:242055+0000", 2024-09-19T09:18:03.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T09:18:03.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T09:18:03.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:03.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T09:18:03.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T09:18:03.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T09:18:03.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T09:18:03.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T09:18:03.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T09:18:03.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T09:18:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:18:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T09:18:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T09:18:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:18:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T09:17:48.242794+0000", 2024-09-19T09:18:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T09:18:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T09:18:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:18:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.589+0000 7f18884bb640 1 Processor -- start 2024-09-19T09:18:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.589+0000 7f18884bb640 1 -- start start 2024-09-19T09:18:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.589+0000 7f18884bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 0x7f1880107b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.589+0000 7f18884bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1880108100 con 0x7f1880107730 2024-09-19T09:18:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.590+0000 7f1886257640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 0x7f1880107b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.590+0000 7f1886257640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 0x7f1880107b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39930/0 (socket says 172.21.15.16:39930) 2024-09-19T09:18:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.590+0000 7f1886257640 1 -- 172.21.15.16:0/1422079602 learned_addr learned my addr 172.21.15.16:0/1422079602 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.590+0000 7f1886257640 1 -- 172.21.15.16:0/1422079602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1880108930 con 0x7f1880107730 2024-09-19T09:18:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.591+0000 7f1886257640 1 --2- 172.21.15.16:0/1422079602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 0x7f1880107b30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f187c009820 tx=0x7f187c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4881d574e5c3e3cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.591+0000 7f1885255640 1 -- 172.21.15.16:0/1422079602 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f187c037070 con 0x7f1880107730 2024-09-19T09:18:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.591+0000 7f1885255640 1 -- 172.21.15.16:0/1422079602 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f187c032900 con 0x7f1880107730 2024-09-19T09:18:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.591+0000 7f1885255640 1 -- 172.21.15.16:0/1422079602 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f187c030850 con 0x7f1880107730 2024-09-19T09:18:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.591+0000 7f18884bb640 1 -- 172.21.15.16:0/1422079602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 msgr2=0x7f1880107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.592+0000 7f18884bb640 1 --2- 172.21.15.16:0/1422079602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 0x7f1880107b30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f187c009820 tx=0x7f187c02a270 comp rx=0 tx=0).stop 2024-09-19T09:18:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.592+0000 7f18884bb640 1 -- 172.21.15.16:0/1422079602 shutdown_connections 2024-09-19T09:18:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.592+0000 7f18884bb640 1 --2- 172.21.15.16:0/1422079602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 0x7f1880107b30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.592+0000 7f18884bb640 1 -- 172.21.15.16:0/1422079602 >> 172.21.15.16:0/1422079602 conn(0x7f1880102ea0 msgr2=0x7f1880105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.592+0000 7f18884bb640 1 -- 172.21.15.16:0/1422079602 shutdown_connections 2024-09-19T09:18:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.592+0000 7f18884bb640 1 -- 172.21.15.16:0/1422079602 wait complete. 2024-09-19T09:18:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.592+0000 7f18884bb640 1 Processor -- start 2024-09-19T09:18:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.592+0000 7f18884bb640 1 -- start start 2024-09-19T09:18:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.593+0000 7f18884bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 0x7f188007daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.593+0000 7f18884bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f188010bce0 con 0x7f1880107730 2024-09-19T09:18:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.593+0000 7f1886257640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 0x7f188007daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.593+0000 7f1886257640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 0x7f188007daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39938/0 (socket says 172.21.15.16:39938) 2024-09-19T09:18:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.593+0000 7f1886257640 1 -- 172.21.15.16:0/2267198823 learned_addr learned my addr 172.21.15.16:0/2267198823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.593+0000 7f1886257640 1 -- 172.21.15.16:0/2267198823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188007e030 con 0x7f1880107730 2024-09-19T09:18:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.593+0000 7f1886257640 1 --2- 172.21.15.16:0/2267198823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 0x7f188007daf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f187c0097f0 tx=0x7f187c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.593+0000 7f18777fe640 1 -- 172.21.15.16:0/2267198823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f187c037040 con 0x7f1880107730 2024-09-19T09:18:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.593+0000 7f18777fe640 1 -- 172.21.15.16:0/2267198823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f187c040070 con 0x7f1880107730 2024-09-19T09:18:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.593+0000 7f18884bb640 1 -- 172.21.15.16:0/2267198823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188007c220 con 0x7f1880107730 2024-09-19T09:18:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.593+0000 7f18777fe640 1 -- 172.21.15.16:0/2267198823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f187c03a390 con 0x7f1880107730 2024-09-19T09:18:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.593+0000 7f18884bb640 1 -- 172.21.15.16:0/2267198823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188007c700 con 0x7f1880107730 2024-09-19T09:18:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.594+0000 7f18777fe640 1 -- 172.21.15.16:0/2267198823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f187c03a630 con 0x7f1880107730 2024-09-19T09:18:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.594+0000 7f18777fe640 1 -- 172.21.15.16:0/2267198823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f187c0459a0 con 0x7f1880107730 2024-09-19T09:18:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.594+0000 7f18884bb640 1 -- 172.21.15.16:0/2267198823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1880107bb0 con 0x7f1880107730 2024-09-19T09:18:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.597+0000 7f18777fe640 1 -- 172.21.15.16:0/2267198823 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f187c045460 con 0x7f1880107730 2024-09-19T09:18:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.661+0000 7f18884bb640 1 -- 172.21.15.16:0/2267198823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f188007cc30 con 0x7f1880107730 2024-09-19T09:18:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.662+0000 7f18777fe640 1 -- 172.21.15.16:0/2267198823 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f187c03b430 con 0x7f1880107730 2024-09-19T09:18:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.662+0000 7f18884bb640 1 -- 172.21.15.16:0/2267198823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 msgr2=0x7f188007daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.662+0000 7f18884bb640 1 --2- 172.21.15.16:0/2267198823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 0x7f188007daf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f187c0097f0 tx=0x7f187c030d80 comp rx=0 tx=0).stop 2024-09-19T09:18:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.662+0000 7f18884bb640 1 -- 172.21.15.16:0/2267198823 shutdown_connections 2024-09-19T09:18:03.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.663+0000 7f18884bb640 1 --2- 172.21.15.16:0/2267198823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1880107730 0x7f188007daf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:03.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.663+0000 7f18884bb640 1 -- 172.21.15.16:0/2267198823 >> 172.21.15.16:0/2267198823 conn(0x7f1880102ea0 msgr2=0x7f1880103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:03.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.663+0000 7f18884bb640 1 -- 172.21.15.16:0/2267198823 shutdown_connections 2024-09-19T09:18:03.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:02.663+0000 7f18884bb640 1 -- 172.21.15.16:0/2267198823 wait complete. 2024-09-19T09:18:03.282 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (1/15)... 2024-09-19T09:18:04.502 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:04 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:04.145+0000 7fd5f3378040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T09:18:04.503 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:04 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:04.216+0000 7fd5f3378040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T09:18:05.238 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:04 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:04.968+0000 7fd5f3378040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T09:18:05.313 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:05 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:05.088+0000 7fd5f3378040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T09:18:05.503 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:05 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:05.237+0000 7fd5f3378040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:18:05.503 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:05 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:05.311+0000 7fd5f3378040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T09:18:05.503 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:05 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:05.467+0000 7fd5f3378040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T09:18:05.775 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:05 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:05.541+0000 7fd5f3378040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T09:18:05.775 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:05 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:05.603+0000 7fd5f3378040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T09:18:05.775 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:05 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:05.774+0000 7fd5f3378040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T09:18:06.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:05 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2111854113' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T09:18:06.066 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:05 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:05.846+0000 7fd5f3378040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T09:18:06.066 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:05 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:05.915+0000 7fd5f3378040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T09:18:06.066 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:06 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:06.002+0000 7fd5f3378040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T09:18:06.066 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:06 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:06.064+0000 7fd5f3378040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T09:18:06.377 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:06 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:06.127+0000 7fd5f3378040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T09:18:06.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-19T09:18:06.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:18:06.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:18:06.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T09:18:06.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T09:18:06.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T09:18:06.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T09:18:06.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:06.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T09:18:06.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T09:18:06.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-19T09:18:06.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:18:06.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T09:18:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T09:18:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:18:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-19T09:18:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T09:18:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:18:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T09:18:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T09:18:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T09:18:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:18:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T09:18:06.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T09:18:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T09:18:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T09:18:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T09:18:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T09:18:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T09:18:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T09:18:06.393 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T09:18:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T09:18:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T09:18:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T09:18:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T09:18:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T09:18:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T09:18:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T09:18:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:18:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T09:17:48:242055+0000", 2024-09-19T09:18:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T09:18:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T09:18:06.394 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:06.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T09:18:06.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T09:18:06.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T09:18:06.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T09:18:06.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T09:18:06.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T09:18:06.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T09:18:06.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:18:06.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T09:18:06.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:06.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T09:18:06.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:18:06.395 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T09:17:48.242794+0000", 2024-09-19T09:18:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T09:18:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T09:18:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:18:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.709+0000 7f656ba33640 1 Processor -- start 2024-09-19T09:18:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.709+0000 7f656ba33640 1 -- start start 2024-09-19T09:18:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.709+0000 7f656ba33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 0x7f656410a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.709+0000 7f656ba33640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f656407be30 con 0x7f6564108140 2024-09-19T09:18:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.710+0000 7f65697cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 0x7f656410a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.710+0000 7f65697cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 0x7f656410a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40998/0 (socket says 172.21.15.16:40998) 2024-09-19T09:18:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.710+0000 7f65697cf640 1 -- 172.21.15.16:0/1630188084 learned_addr learned my addr 172.21.15.16:0/1630188084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:06.396 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.710+0000 7f65697cf640 1 -- 172.21.15.16:0/1630188084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656410aa70 con 0x7f6564108140 2024-09-19T09:18:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.711+0000 7f65697cf640 1 --2- 172.21.15.16:0/1630188084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 0x7f656410a530 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6560009820 tx=0x7f656002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190682041069fc7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.711+0000 7f655bfff640 1 -- 172.21.15.16:0/1630188084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6560037070 con 0x7f6564108140 2024-09-19T09:18:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.711+0000 7f655bfff640 1 -- 172.21.15.16:0/1630188084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6560032900 con 0x7f6564108140 2024-09-19T09:18:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.711+0000 7f655bfff640 1 -- 172.21.15.16:0/1630188084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6560030850 con 0x7f6564108140 2024-09-19T09:18:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.711+0000 7f656ba33640 1 -- 172.21.15.16:0/1630188084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 msgr2=0x7f656410a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.711+0000 7f656ba33640 1 --2- 172.21.15.16:0/1630188084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 0x7f656410a530 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6560009820 tx=0x7f656002a270 comp rx=0 tx=0).stop 2024-09-19T09:18:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.711+0000 7f656ba33640 1 -- 172.21.15.16:0/1630188084 shutdown_connections 2024-09-19T09:18:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.711+0000 7f656ba33640 1 --2- 172.21.15.16:0/1630188084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 0x7f656410a530 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.711+0000 7f656ba33640 1 -- 172.21.15.16:0/1630188084 >> 172.21.15.16:0/1630188084 conn(0x7f6564101e20 msgr2=0x7f6564104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.712+0000 7f656ba33640 1 -- 172.21.15.16:0/1630188084 shutdown_connections 2024-09-19T09:18:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.712+0000 7f656ba33640 1 -- 172.21.15.16:0/1630188084 wait complete. 2024-09-19T09:18:06.397 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.712+0000 7f656ba33640 1 Processor -- start 2024-09-19T09:18:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.712+0000 7f656ba33640 1 -- start start 2024-09-19T09:18:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.713+0000 7f656ba33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 0x7f656419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.713+0000 7f656ba33640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f656410bb10 con 0x7f6564108140 2024-09-19T09:18:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.713+0000 7f65697cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 0x7f656419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.713+0000 7f65697cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 0x7f656419b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41008/0 (socket says 172.21.15.16:41008) 2024-09-19T09:18:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.713+0000 7f65697cf640 1 -- 172.21.15.16:0/2111854113 learned_addr learned my addr 172.21.15.16:0/2111854113 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.713+0000 7f65697cf640 1 -- 172.21.15.16:0/2111854113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656419bc00 con 0x7f6564108140 2024-09-19T09:18:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.713+0000 7f65697cf640 1 --2- 172.21.15.16:0/2111854113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 0x7f656419b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6560032ce0 tx=0x7f6560032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.713+0000 7f655a7fc640 1 -- 172.21.15.16:0/2111854113 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6560042070 con 0x7f6564108140 2024-09-19T09:18:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.713+0000 7f655a7fc640 1 -- 172.21.15.16:0/2111854113 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f65600308c0 con 0x7f6564108140 2024-09-19T09:18:06.398 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.713+0000 7f656ba33640 1 -- 172.21.15.16:0/2111854113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656419be90 con 0x7f6564108140 2024-09-19T09:18:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.714+0000 7f656ba33640 1 -- 172.21.15.16:0/2111854113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f656419eb80 con 0x7f6564108140 2024-09-19T09:18:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.714+0000 7f655a7fc640 1 -- 172.21.15.16:0/2111854113 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6560037040 con 0x7f6564108140 2024-09-19T09:18:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.714+0000 7f655a7fc640 1 -- 172.21.15.16:0/2111854113 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6560045070 con 0x7f6564108140 2024-09-19T09:18:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.714+0000 7f655a7fc640 1 -- 172.21.15.16:0/2111854113 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6560047b90 con 0x7f6564108140 2024-09-19T09:18:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.714+0000 7f656ba33640 1 -- 172.21.15.16:0/2111854113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65641085a0 con 0x7f6564108140 2024-09-19T09:18:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.718+0000 7f655a7fc640 1 -- 172.21.15.16:0/2111854113 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f656003c800 con 0x7f6564108140 2024-09-19T09:18:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.783+0000 7f656ba33640 1 -- 172.21.15.16:0/2111854113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f656419f100 con 0x7f6564108140 2024-09-19T09:18:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.783+0000 7f655a7fc640 1 -- 172.21.15.16:0/2111854113 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f656003c9e0 con 0x7f6564108140 2024-09-19T09:18:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.785+0000 7f656ba33640 1 -- 172.21.15.16:0/2111854113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 msgr2=0x7f656419b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.785+0000 7f656ba33640 1 --2- 172.21.15.16:0/2111854113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 0x7f656419b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6560032ce0 tx=0x7f6560032d10 comp rx=0 tx=0).stop 2024-09-19T09:18:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.785+0000 7f656ba33640 1 -- 172.21.15.16:0/2111854113 shutdown_connections 2024-09-19T09:18:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.785+0000 7f656ba33640 1 --2- 172.21.15.16:0/2111854113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6564108140 0x7f656419b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:06.399 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.785+0000 7f656ba33640 1 -- 172.21.15.16:0/2111854113 >> 172.21.15.16:0/2111854113 conn(0x7f6564101e20 msgr2=0x7f65641029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:06.400 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.785+0000 7f656ba33640 1 -- 172.21.15.16:0/2111854113 shutdown_connections 2024-09-19T09:18:06.400 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:05.785+0000 7f656ba33640 1 -- 172.21.15.16:0/2111854113 wait complete. 2024-09-19T09:18:06.400 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (2/15)... 2024-09-19T09:18:07.253 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:06 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:06.959+0000 7fd5f3378040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T09:18:07.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:07 smithi016 ceph-mon[30979]: Activating manager daemon x 2024-09-19T09:18:07.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:07 smithi016 ceph-mon[30979]: mgrmap e2: x(active, starting, since 0.00306783s) 2024-09-19T09:18:07.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:07 smithi016 ceph-mon[30979]: from='mgr.14100 172.21.15.16:0/3903712556' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T09:18:07.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:07 smithi016 ceph-mon[30979]: from='mgr.14100 172.21.15.16:0/3903712556' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T09:18:07.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:07 smithi016 ceph-mon[30979]: from='mgr.14100 172.21.15.16:0/3903712556' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T09:18:07.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:07 smithi016 ceph-mon[30979]: from='mgr.14100 172.21.15.16:0/3903712556' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:18:07.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:07 smithi016 ceph-mon[30979]: from='mgr.14100 172.21.15.16:0/3903712556' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T09:18:07.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:07 smithi016 ceph-mon[30979]: Manager daemon x is now available 2024-09-19T09:18:07.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:07 smithi016 ceph-mon[30979]: from='mgr.14100 172.21.15.16:0/3903712556' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T09:18:07.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:07 smithi016 ceph-mon[30979]: from='mgr.14100 172.21.15.16:0/3903712556' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T09:18:07.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:07 smithi016 ceph-mon[30979]: from='mgr.14100 172.21.15.16:0/3903712556' entity='mgr.x' 2024-09-19T09:18:07.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:07 smithi016 ceph-mon[30979]: from='mgr.14100 172.21.15.16:0/3903712556' entity='mgr.x' 2024-09-19T09:18:07.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:07 smithi016 ceph-mon[30979]: from='mgr.14100 172.21.15.16:0/3903712556' entity='mgr.x' 2024-09-19T09:18:09.079 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:08 smithi016 ceph-mon[30979]: mgrmap e3: x(active, since 1.00735s) 2024-09-19T09:18:09.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-19T09:18:09.646 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:18:09.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:18:09.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T09:18:09.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T09:18:09.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T09:18:09.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T09:18:09.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:09.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T09:18:09.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T09:18:09.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-19T09:18:09.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:18:09.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T09:18:09.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T09:18:09.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:18:09.648 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-19T09:18:09.648 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T09:18:09.648 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:18:09.648 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T09:18:09.648 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T09:18:09.648 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:09.648 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T09:18:09.648 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:18:09.648 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T09:18:09.648 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T09:18:09.649 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T09:18:09.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T09:18:09.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T09:18:09.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T09:18:09.657 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T09:18:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T09:18:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T09:18:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T09:18:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T09:18:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T09:18:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T09:18:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T09:18:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T09:18:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T09:18:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:18:09.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T09:17:48:242055+0000", 2024-09-19T09:18:09.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T09:18:09.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T09:18:09.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:09.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T09:18:09.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T09:18:09.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T09:18:09.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T09:18:09.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T09:18:09.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T09:18:09.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T09:18:09.659 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-19T09:18:09.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T09:18:09.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:09.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T09:18:09.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T09:18:09.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T09:17:48.242794+0000", 2024-09-19T09:18:09.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T09:18:09.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-19T09:18:09.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T09:18:09.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:18:09.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.824+0000 7fcea9d64640 1 Processor -- start 2024-09-19T09:18:09.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.824+0000 7fcea9d64640 1 -- start start 2024-09-19T09:18:09.660 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.824+0000 7fcea9d64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 0x7fcea4106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.824+0000 7fcea9d64640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4106ff0 con 0x7fcea4106620 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.825+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 0x7fcea4106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.825+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 0x7fcea4106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41074/0 (socket says 172.21.15.16:41074) 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.825+0000 7fcea37fe640 1 -- 172.21.15.16:0/2958654208 learned_addr learned my addr 172.21.15.16:0/2958654208 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.825+0000 7fcea37fe640 1 -- 172.21.15.16:0/2958654208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea4107820 con 0x7fcea4106620 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.826+0000 7fcea37fe640 1 --2- 172.21.15.16:0/2958654208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 0x7fcea4106a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fce94009820 tx=0x7fce9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b67915c6b5aab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.826+0000 7fcea27fc640 1 -- 172.21.15.16:0/2958654208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce94037070 con 0x7fcea4106620 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.826+0000 7fcea27fc640 1 -- 172.21.15.16:0/2958654208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fce94032900 con 0x7fcea4106620 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.826+0000 7fcea27fc640 1 -- 172.21.15.16:0/2958654208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce94030850 con 0x7fcea4106620 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.827+0000 7fcea9d64640 1 -- 172.21.15.16:0/2958654208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 msgr2=0x7fcea4106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.827+0000 7fcea9d64640 1 --2- 172.21.15.16:0/2958654208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 0x7fcea4106a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fce94009820 tx=0x7fce9402a270 comp rx=0 tx=0).stop 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.827+0000 7fcea9d64640 1 -- 172.21.15.16:0/2958654208 shutdown_connections 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.827+0000 7fcea9d64640 1 --2- 172.21.15.16:0/2958654208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 0x7fcea4106a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.661 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.827+0000 7fcea9d64640 1 -- 172.21.15.16:0/2958654208 >> 172.21.15.16:0/2958654208 conn(0x7fcea4101e70 msgr2=0x7fcea4104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:09.662 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.827+0000 7fcea9d64640 1 -- 172.21.15.16:0/2958654208 shutdown_connections 2024-09-19T09:18:09.662 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.827+0000 7fcea9d64640 1 -- 172.21.15.16:0/2958654208 wait complete. 2024-09-19T09:18:09.662 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.827+0000 7fcea9d64640 1 Processor -- start 2024-09-19T09:18:09.662 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.828+0000 7fcea9d64640 1 -- start start 2024-09-19T09:18:09.662 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.828+0000 7fcea9d64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 0x7fcea419b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.662 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.828+0000 7fcea9d64640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcea410abd0 con 0x7fcea4106620 2024-09-19T09:18:09.662 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.828+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 0x7fcea419b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.662 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.828+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 0x7fcea419b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41086/0 (socket says 172.21.15.16:41086) 2024-09-19T09:18:09.662 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.828+0000 7fcea37fe640 1 -- 172.21.15.16:0/3839426551 learned_addr learned my addr 172.21.15.16:0/3839426551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:09.662 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.828+0000 7fcea37fe640 1 -- 172.21.15.16:0/3839426551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea419bc20 con 0x7fcea4106620 2024-09-19T09:18:09.662 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.828+0000 7fcea37fe640 1 --2- 172.21.15.16:0/3839426551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 0x7fcea419b6e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fce940097f0 tx=0x7fce94030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:09.662 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.829+0000 7fcea0ff9640 1 -- 172.21.15.16:0/3839426551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce94037040 con 0x7fcea4106620 2024-09-19T09:18:09.662 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.829+0000 7fcea0ff9640 1 -- 172.21.15.16:0/3839426551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fce94040070 con 0x7fcea4106620 2024-09-19T09:18:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.829+0000 7fcea9d64640 1 -- 172.21.15.16:0/3839426551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea419beb0 con 0x7fcea4106620 2024-09-19T09:18:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.829+0000 7fcea0ff9640 1 -- 172.21.15.16:0/3839426551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce9403a390 con 0x7fcea4106620 2024-09-19T09:18:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.829+0000 7fcea9d64640 1 -- 172.21.15.16:0/3839426551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea419eba0 con 0x7fcea4106620 2024-09-19T09:18:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.830+0000 7fcea0ff9640 1 -- 172.21.15.16:0/3839426551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7fce94032900 con 0x7fcea4106620 2024-09-19T09:18:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.830+0000 7fcea0ff9640 1 --2- 172.21.15.16:0/3839426551 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7fce8c0390a0 0x7fce8c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.830+0000 7fcea9d64640 1 -- 172.21.15.16:0/3839426551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea4106aa0 con 0x7fcea4106620 2024-09-19T09:18:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.830+0000 7fcea0ff9640 1 -- 172.21.15.16:0/3839426551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fce9406c150 con 0x7fcea4106620 2024-09-19T09:18:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.830+0000 7fcea2ffd640 1 --2- 172.21.15.16:0/3839426551 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7fce8c0390a0 0x7fce8c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.830+0000 7fcea2ffd640 1 --2- 172.21.15.16:0/3839426551 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7fce8c0390a0 0x7fce8c03b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fce980098c0 tx=0x7fce98006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.835+0000 7fcea0ff9640 1 -- 172.21.15.16:0/3839426551 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fce94032bf0 con 0x7fcea4106620 2024-09-19T09:18:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:08.968+0000 7fcea0ff9640 1 -- 172.21.15.16:0/3839426551 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fce94044ca0 con 0x7fcea4106620 2024-09-19T09:18:09.663 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:09.078+0000 7fcea9d64640 1 -- 172.21.15.16:0/3839426551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcea402d9f0 con 0x7fcea4106620 2024-09-19T09:18:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:09.079+0000 7fcea0ff9640 1 -- 172.21.15.16:0/3839426551 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fce94038250 con 0x7fcea4106620 2024-09-19T09:18:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:09.080+0000 7fcea9d64640 1 -- 172.21.15.16:0/3839426551 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7fce8c0390a0 msgr2=0x7fce8c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:09.080+0000 7fcea9d64640 1 --2- 172.21.15.16:0/3839426551 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7fce8c0390a0 0x7fce8c03b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fce980098c0 tx=0x7fce98006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:09.080+0000 7fcea9d64640 1 -- 172.21.15.16:0/3839426551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 msgr2=0x7fcea419b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:09.080+0000 7fcea9d64640 1 --2- 172.21.15.16:0/3839426551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 0x7fcea419b6e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fce940097f0 tx=0x7fce94030d80 comp rx=0 tx=0).stop 2024-09-19T09:18:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:09.081+0000 7fcea9d64640 1 -- 172.21.15.16:0/3839426551 shutdown_connections 2024-09-19T09:18:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:09.081+0000 7fcea9d64640 1 --2- 172.21.15.16:0/3839426551 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7fce8c0390a0 0x7fce8c03b560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:09.081+0000 7fcea9d64640 1 --2- 172.21.15.16:0/3839426551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea4106620 0x7fcea419b6e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:09.081+0000 7fcea9d64640 1 -- 172.21.15.16:0/3839426551 >> 172.21.15.16:0/3839426551 conn(0x7fcea4101e70 msgr2=0x7fcea4102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:09.081+0000 7fcea9d64640 1 -- 172.21.15.16:0/3839426551 shutdown_connections 2024-09-19T09:18:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:09.081+0000 7fcea9d64640 1 -- 172.21.15.16:0/3839426551 wait complete. 2024-09-19T09:18:09.664 INFO:teuthology.orchestra.run.smithi016.stdout:mgr is available 2024-09-19T09:18:10.043 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:09 smithi016 ceph-mon[30979]: mgrmap e4: x(active, since 2s) 2024-09-19T09:18:10.043 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:09 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3839426551' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T09:18:10.842 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-19T09:18:10.842 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T09:18:10.842 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:18:10.842 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-19T09:18:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T09:18:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T09:18:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T09:18:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T09:18:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-19T09:18:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T09:18:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T09:18:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-19T09:18:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T09:18:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T09:18:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T09:18:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T09:18:10.843 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T09:18:10.844 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T09:18:10.844 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.042+0000 7f496e425640 1 Processor -- start 2024-09-19T09:18:10.844 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.042+0000 7f496e425640 1 -- start start 2024-09-19T09:18:10.844 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.042+0000 7f496e425640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 0x7f496810a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:10.844 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.042+0000 7f496e425640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f496807bdb0 con 0x7f4968108200 2024-09-19T09:18:10.844 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.042+0000 7f4967fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 0x7f496810a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:10.844 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.043+0000 7f4967fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 0x7f496810a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41092/0 (socket says 172.21.15.16:41092) 2024-09-19T09:18:10.844 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.043+0000 7f4967fff640 1 -- 172.21.15.16:0/4076000364 learned_addr learned my addr 172.21.15.16:0/4076000364 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:10.844 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.043+0000 7f4967fff640 1 -- 172.21.15.16:0/4076000364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496810ab30 con 0x7f4968108200 2024-09-19T09:18:10.844 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.044+0000 7f4967fff640 1 --2- 172.21.15.16:0/4076000364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 0x7f496810a5f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4958009820 tx=0x7f495802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b5bf5bf37930b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:10.844 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.044+0000 7f4966ffd640 1 -- 172.21.15.16:0/4076000364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4958037070 con 0x7f4968108200 2024-09-19T09:18:10.844 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.044+0000 7f4966ffd640 1 -- 172.21.15.16:0/4076000364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4958032900 con 0x7f4968108200 2024-09-19T09:18:10.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.044+0000 7f4966ffd640 1 -- 172.21.15.16:0/4076000364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4958030850 con 0x7f4968108200 2024-09-19T09:18:10.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.045+0000 7f496e425640 1 -- 172.21.15.16:0/4076000364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 msgr2=0x7f496810a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:10.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.045+0000 7f496e425640 1 --2- 172.21.15.16:0/4076000364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 0x7f496810a5f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4958009820 tx=0x7f495802a270 comp rx=0 tx=0).stop 2024-09-19T09:18:10.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.045+0000 7f496e425640 1 -- 172.21.15.16:0/4076000364 shutdown_connections 2024-09-19T09:18:10.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.045+0000 7f496e425640 1 --2- 172.21.15.16:0/4076000364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 0x7f496810a5f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:10.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.045+0000 7f496e425640 1 -- 172.21.15.16:0/4076000364 >> 172.21.15.16:0/4076000364 conn(0x7f4968101e20 msgr2=0x7f4968104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:10.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.045+0000 7f496e425640 1 -- 172.21.15.16:0/4076000364 shutdown_connections 2024-09-19T09:18:10.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.045+0000 7f496e425640 1 -- 172.21.15.16:0/4076000364 wait complete. 2024-09-19T09:18:10.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.045+0000 7f496e425640 1 Processor -- start 2024-09-19T09:18:10.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.045+0000 7f496e425640 1 -- start start 2024-09-19T09:18:10.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.046+0000 7f496e425640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 0x7f496819b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:10.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.046+0000 7f496e425640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f496810bb10 con 0x7f4968108200 2024-09-19T09:18:10.845 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.046+0000 7f4967fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 0x7f496819b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:10.846 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.046+0000 7f4967fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 0x7f496819b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41098/0 (socket says 172.21.15.16:41098) 2024-09-19T09:18:10.846 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.046+0000 7f4967fff640 1 -- 172.21.15.16:0/4285591758 learned_addr learned my addr 172.21.15.16:0/4285591758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:10.846 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.046+0000 7f4967fff640 1 -- 172.21.15.16:0/4285591758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496819bc50 con 0x7f4968108200 2024-09-19T09:18:10.846 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.046+0000 7f4967fff640 1 --2- 172.21.15.16:0/4285591758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 0x7f496819b710 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49580097f0 tx=0x7f4958030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:10.846 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.046+0000 7f49657fa640 1 -- 172.21.15.16:0/4285591758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4958037040 con 0x7f4968108200 2024-09-19T09:18:10.846 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.047+0000 7f49657fa640 1 -- 172.21.15.16:0/4285591758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4958040070 con 0x7f4968108200 2024-09-19T09:18:10.847 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.047+0000 7f496e425640 1 -- 172.21.15.16:0/4285591758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496819bee0 con 0x7f4968108200 2024-09-19T09:18:10.847 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.047+0000 7f49657fa640 1 -- 172.21.15.16:0/4285591758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f495803a390 con 0x7f4968108200 2024-09-19T09:18:10.847 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.047+0000 7f496e425640 1 -- 172.21.15.16:0/4285591758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496819ebd0 con 0x7f4968108200 2024-09-19T09:18:10.847 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.047+0000 7f49657fa640 1 -- 172.21.15.16:0/4285591758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f495803a5e0 con 0x7f4968108200 2024-09-19T09:18:10.847 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.048+0000 7f49657fa640 1 --2- 172.21.15.16:0/4285591758 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f49500391b0 0x7f495003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:10.847 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.048+0000 7f49657fa640 1 -- 172.21.15.16:0/4285591758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f495806c3d0 con 0x7f4968108200 2024-09-19T09:18:10.847 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.048+0000 7f496e425640 1 -- 172.21.15.16:0/4285591758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4968108660 con 0x7f4968108200 2024-09-19T09:18:10.847 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.048+0000 7f49677fe640 1 --2- 172.21.15.16:0/4285591758 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f49500391b0 0x7f495003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:10.847 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.048+0000 7f49677fe640 1 --2- 172.21.15.16:0/4285591758 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f49500391b0 0x7f495003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f495c0098c0 tx=0x7f495c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:10.847 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.053+0000 7f49657fa640 1 -- 172.21.15.16:0/4285591758 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f495806c130 con 0x7f4968108200 2024-09-19T09:18:10.848 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.230+0000 7f496e425640 1 -- 172.21.15.16:0/4285591758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f496819c360 con 0x7f4968108200 2024-09-19T09:18:10.848 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.231+0000 7f49657fa640 1 -- 172.21.15.16:0/4285591758 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f495803a8d0 con 0x7f4968108200 2024-09-19T09:18:10.848 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.233+0000 7f496e425640 1 -- 172.21.15.16:0/4285591758 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f49500391b0 msgr2=0x7f495003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:10.848 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.233+0000 7f496e425640 1 --2- 172.21.15.16:0/4285591758 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f49500391b0 0x7f495003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f495c0098c0 tx=0x7f495c006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:10.848 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.233+0000 7f496e425640 1 -- 172.21.15.16:0/4285591758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 msgr2=0x7f496819b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:10.848 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.233+0000 7f496e425640 1 --2- 172.21.15.16:0/4285591758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 0x7f496819b710 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49580097f0 tx=0x7f4958030d80 comp rx=0 tx=0).stop 2024-09-19T09:18:10.848 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.234+0000 7f496e425640 1 -- 172.21.15.16:0/4285591758 shutdown_connections 2024-09-19T09:18:10.848 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.234+0000 7f496e425640 1 --2- 172.21.15.16:0/4285591758 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f49500391b0 0x7f495003b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:10.848 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.234+0000 7f496e425640 1 --2- 172.21.15.16:0/4285591758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4968108200 0x7f496819b710 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:10.848 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.234+0000 7f496e425640 1 -- 172.21.15.16:0/4285591758 >> 172.21.15.16:0/4285591758 conn(0x7f4968101e20 msgr2=0x7f49681029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:10.848 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.234+0000 7f496e425640 1 -- 172.21.15.16:0/4285591758 shutdown_connections 2024-09-19T09:18:10.848 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:10.234+0000 7f496e425640 1 -- 172.21.15.16:0/4285591758 wait complete. 2024-09-19T09:18:10.849 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling cephadm module... 2024-09-19T09:18:11.159 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:10 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/4285591758' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T09:18:12.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:11 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/4074534263' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T09:18:12.257 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:12 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: ignoring --setuser ceph since I am not root 2024-09-19T09:18:12.257 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:12 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: ignoring --setgroup ceph since I am not root 2024-09-19T09:18:12.258 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:12 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:12.122+0000 7f979ec6d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T09:18:12.258 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:12 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:12.181+0000 7f979ec6d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T09:18:12.563 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:12 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:12.338+0000 7f979ec6d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T09:18:12.563 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:12 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:12.408+0000 7f979ec6d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T09:18:12.563 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:12 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:12.562+0000 7f979ec6d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:18:12.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.247+0000 7fc771219640 1 Processor -- start 2024-09-19T09:18:12.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.247+0000 7fc771219640 1 -- start start 2024-09-19T09:18:12.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.247+0000 7fc771219640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 0x7fc76c107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:12.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.247+0000 7fc771219640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c108150 con 0x7fc76c107780 2024-09-19T09:18:12.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.247+0000 7fc76ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 0x7fc76c107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:12.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.247+0000 7fc76ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 0x7fc76c107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41114/0 (socket says 172.21.15.16:41114) 2024-09-19T09:18:12.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.247+0000 7fc76ad9d640 1 -- 172.21.15.16:0/1430810964 learned_addr learned my addr 172.21.15.16:0/1430810964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:12.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.247+0000 7fc76ad9d640 1 -- 172.21.15.16:0/1430810964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c108980 con 0x7fc76c107780 2024-09-19T09:18:12.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.248+0000 7fc76ad9d640 1 --2- 172.21.15.16:0/1430810964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 0x7fc76c107b80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009820 tx=0x7fc75c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f40cc559189317 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:12.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.248+0000 7fc769d9b640 1 -- 172.21.15.16:0/1430810964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc75c037070 con 0x7fc76c107780 2024-09-19T09:18:12.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.248+0000 7fc769d9b640 1 -- 172.21.15.16:0/1430810964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc75c032900 con 0x7fc76c107780 2024-09-19T09:18:12.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.248+0000 7fc769d9b640 1 -- 172.21.15.16:0/1430810964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc75c030850 con 0x7fc76c107780 2024-09-19T09:18:12.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.249+0000 7fc771219640 1 -- 172.21.15.16:0/1430810964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 msgr2=0x7fc76c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:12.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.249+0000 7fc771219640 1 --2- 172.21.15.16:0/1430810964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 0x7fc76c107b80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009820 tx=0x7fc75c02a270 comp rx=0 tx=0).stop 2024-09-19T09:18:12.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.249+0000 7fc771219640 1 -- 172.21.15.16:0/1430810964 shutdown_connections 2024-09-19T09:18:12.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.249+0000 7fc771219640 1 --2- 172.21.15.16:0/1430810964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 0x7fc76c107b80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:12.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.249+0000 7fc771219640 1 -- 172.21.15.16:0/1430810964 >> 172.21.15.16:0/1430810964 conn(0x7fc76c102f30 msgr2=0x7fc76c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:12.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.249+0000 7fc771219640 1 -- 172.21.15.16:0/1430810964 shutdown_connections 2024-09-19T09:18:12.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.249+0000 7fc771219640 1 -- 172.21.15.16:0/1430810964 wait complete. 2024-09-19T09:18:12.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.250+0000 7fc771219640 1 Processor -- start 2024-09-19T09:18:12.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.250+0000 7fc771219640 1 -- start start 2024-09-19T09:18:12.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.250+0000 7fc771219640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 0x7fc76c07daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:12.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.250+0000 7fc771219640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c10bd30 con 0x7fc76c107780 2024-09-19T09:18:12.591 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.250+0000 7fc76ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 0x7fc76c07daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:12.591 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.250+0000 7fc76ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 0x7fc76c07daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41128/0 (socket says 172.21.15.16:41128) 2024-09-19T09:18:12.591 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.250+0000 7fc76ad9d640 1 -- 172.21.15.16:0/4074534263 learned_addr learned my addr 172.21.15.16:0/4074534263 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:12.591 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.251+0000 7fc76ad9d640 1 -- 172.21.15.16:0/4074534263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c07e030 con 0x7fc76c107780 2024-09-19T09:18:12.591 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.251+0000 7fc76ad9d640 1 --2- 172.21.15.16:0/4074534263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 0x7fc76c07daf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009940 tx=0x7fc75c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:12.592 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.251+0000 7fc753fff640 1 -- 172.21.15.16:0/4074534263 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc75c042070 con 0x7fc76c107780 2024-09-19T09:18:12.592 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.251+0000 7fc771219640 1 -- 172.21.15.16:0/4074534263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c07c220 con 0x7fc76c107780 2024-09-19T09:18:12.592 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.251+0000 7fc771219640 1 -- 172.21.15.16:0/4074534263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c07c760 con 0x7fc76c107780 2024-09-19T09:18:12.592 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.251+0000 7fc753fff640 1 -- 172.21.15.16:0/4074534263 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc75c0308c0 con 0x7fc76c107780 2024-09-19T09:18:12.592 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.251+0000 7fc753fff640 1 -- 172.21.15.16:0/4074534263 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc75c037040 con 0x7fc76c107780 2024-09-19T09:18:12.592 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.252+0000 7fc753fff640 1 -- 172.21.15.16:0/4074534263 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fc75c045070 con 0x7fc76c107780 2024-09-19T09:18:12.592 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.252+0000 7fc753fff640 1 --2- 172.21.15.16:0/4074534263 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7fc754039160 0x7fc75403b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:12.592 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.252+0000 7fc771219640 1 -- 172.21.15.16:0/4074534263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76c107c00 con 0x7fc76c107780 2024-09-19T09:18:12.592 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.252+0000 7fc753fff640 1 -- 172.21.15.16:0/4074534263 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc75c06c3e0 con 0x7fc76c107780 2024-09-19T09:18:12.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.253+0000 7fc76a59c640 1 --2- 172.21.15.16:0/4074534263 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7fc754039160 0x7fc75403b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:12.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.253+0000 7fc76a59c640 1 --2- 172.21.15.16:0/4074534263 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7fc754039160 0x7fc75403b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc7600098c0 tx=0x7fc760006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:12.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.257+0000 7fc753fff640 1 -- 172.21.15.16:0/4074534263 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc75c0392e0 con 0x7fc76c107780 2024-09-19T09:18:12.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.484+0000 7fc771219640 1 -- 172.21.15.16:0/4074534263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fc76c07d040 con 0x7fc76c107780 2024-09-19T09:18:12.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.970+0000 7fc753fff640 1 -- 172.21.15.16:0/4074534263 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fc75c045520 con 0x7fc76c107780 2024-09-19T09:18:12.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.972+0000 7fc753fff640 1 -- 172.21.15.16:0/4074534263 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fc75c046780 con 0x7fc76c107780 2024-09-19T09:18:12.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.974+0000 7fc771219640 1 -- 172.21.15.16:0/4074534263 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7fc754039160 msgr2=0x7fc75403b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:12.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.974+0000 7fc771219640 1 --2- 172.21.15.16:0/4074534263 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7fc754039160 0x7fc75403b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc7600098c0 tx=0x7fc760006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:12.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.974+0000 7fc771219640 1 -- 172.21.15.16:0/4074534263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 msgr2=0x7fc76c07daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:12.593 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.974+0000 7fc771219640 1 --2- 172.21.15.16:0/4074534263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 0x7fc76c07daf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009940 tx=0x7fc75c032ce0 comp rx=0 tx=0).stop 2024-09-19T09:18:12.595 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.974+0000 7fc771219640 1 -- 172.21.15.16:0/4074534263 shutdown_connections 2024-09-19T09:18:12.596 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.974+0000 7fc771219640 1 --2- 172.21.15.16:0/4074534263 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7fc754039160 0x7fc75403b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:12.596 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.974+0000 7fc771219640 1 --2- 172.21.15.16:0/4074534263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc76c107780 0x7fc76c07daf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:12.596 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.974+0000 7fc771219640 1 -- 172.21.15.16:0/4074534263 >> 172.21.15.16:0/4074534263 conn(0x7fc76c102f30 msgr2=0x7fc76c103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:12.596 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.974+0000 7fc771219640 1 -- 172.21.15.16:0/4074534263 shutdown_connections 2024-09-19T09:18:12.596 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:11.974+0000 7fc771219640 1 -- 172.21.15.16:0/4074534263 wait complete. 2024-09-19T09:18:12.843 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:12 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:12.714+0000 7f979ec6d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T09:18:13.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:12 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/4074534263' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T09:18:13.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:12 smithi016 ceph-mon[30979]: mgrmap e5: x(active, since 5s) 2024-09-19T09:18:13.122 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:13 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:13.056+0000 7f979ec6d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T09:18:13.122 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:13 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:13.120+0000 7f979ec6d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T09:18:13.502 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:13 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:13.318+0000 7f979ec6d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T09:18:13.903 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:18:13.903 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T09:18:13.903 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T09:18:13.903 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T09:18:13.903 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T09:18:13.903 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:18:13.903 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.028+0000 7f59054f9640 1 Processor -- start 2024-09-19T09:18:13.903 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.028+0000 7f59054f9640 1 -- start start 2024-09-19T09:18:13.903 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.028+0000 7f59054f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 0x7f5900107b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.903 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.028+0000 7f59054f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5900108100 con 0x7f5900107730 2024-09-19T09:18:13.903 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.028+0000 7f58feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 0x7f5900107b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.904 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.028+0000 7f58feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 0x7f5900107b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41152/0 (socket says 172.21.15.16:41152) 2024-09-19T09:18:13.904 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.028+0000 7f58feffd640 1 -- 172.21.15.16:0/1451631388 learned_addr learned my addr 172.21.15.16:0/1451631388 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:13.904 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.028+0000 7f58feffd640 1 -- 172.21.15.16:0/1451631388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5900108930 con 0x7f5900107730 2024-09-19T09:18:13.904 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.029+0000 7f58feffd640 1 --2- 172.21.15.16:0/1451631388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 0x7f5900107b30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009820 tx=0x7f58f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa72c50d2006b170 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:13.904 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.029+0000 7f58fdffb640 1 -- 172.21.15.16:0/1451631388 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58f0037070 con 0x7f5900107730 2024-09-19T09:18:13.904 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.029+0000 7f58fdffb640 1 -- 172.21.15.16:0/1451631388 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58f0032900 con 0x7f5900107730 2024-09-19T09:18:13.904 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.029+0000 7f58fdffb640 1 -- 172.21.15.16:0/1451631388 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58f0030850 con 0x7f5900107730 2024-09-19T09:18:13.904 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.030+0000 7f59054f9640 1 -- 172.21.15.16:0/1451631388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 msgr2=0x7f5900107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:13.904 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.030+0000 7f59054f9640 1 --2- 172.21.15.16:0/1451631388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 0x7f5900107b30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009820 tx=0x7f58f002a270 comp rx=0 tx=0).stop 2024-09-19T09:18:13.904 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.030+0000 7f59054f9640 1 -- 172.21.15.16:0/1451631388 shutdown_connections 2024-09-19T09:18:13.904 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.030+0000 7f59054f9640 1 --2- 172.21.15.16:0/1451631388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 0x7f5900107b30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.904 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.030+0000 7f59054f9640 1 -- 172.21.15.16:0/1451631388 >> 172.21.15.16:0/1451631388 conn(0x7f5900102ea0 msgr2=0x7f5900105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:13.904 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.030+0000 7f59054f9640 1 -- 172.21.15.16:0/1451631388 shutdown_connections 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.030+0000 7f59054f9640 1 -- 172.21.15.16:0/1451631388 wait complete. 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.031+0000 7f59054f9640 1 Processor -- start 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.031+0000 7f59054f9640 1 -- start start 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.031+0000 7f59054f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 0x7f590019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.031+0000 7f59054f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f590010bce0 con 0x7f5900107730 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.031+0000 7f58feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 0x7f590019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.031+0000 7f58feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 0x7f590019b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41166/0 (socket says 172.21.15.16:41166) 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.031+0000 7f58feffd640 1 -- 172.21.15.16:0/2800497909 learned_addr learned my addr 172.21.15.16:0/2800497909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.032+0000 7f58feffd640 1 -- 172.21.15.16:0/2800497909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590019bc90 con 0x7f5900107730 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.032+0000 7f58feffd640 1 --2- 172.21.15.16:0/2800497909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 0x7f590019b750 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f58f00097f0 tx=0x7f58f0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.032+0000 7f58e7fff640 1 -- 172.21.15.16:0/2800497909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58f0037040 con 0x7f5900107730 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.032+0000 7f59054f9640 1 -- 172.21.15.16:0/2800497909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590019bf20 con 0x7f5900107730 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.032+0000 7f58e7fff640 1 -- 172.21.15.16:0/2800497909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58f0040070 con 0x7f5900107730 2024-09-19T09:18:13.905 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.032+0000 7f58e7fff640 1 -- 172.21.15.16:0/2800497909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58f003a390 con 0x7f5900107730 2024-09-19T09:18:13.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.032+0000 7f59054f9640 1 -- 172.21.15.16:0/2800497909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f590019ec10 con 0x7f5900107730 2024-09-19T09:18:13.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.033+0000 7f58e7fff640 1 -- 172.21.15.16:0/2800497909 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f58f003a5e0 con 0x7f5900107730 2024-09-19T09:18:13.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.033+0000 7f59054f9640 1 -- 172.21.15.16:0/2800497909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5900107bb0 con 0x7f5900107730 2024-09-19T09:18:13.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.033+0000 7f58e7fff640 1 --2- 172.21.15.16:0/2800497909 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f58e8039200 0x7f58e803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:13.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.033+0000 7f58e7fff640 1 -- 172.21.15.16:0/2800497909 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f58f006c3e0 con 0x7f5900107730 2024-09-19T09:18:13.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.033+0000 7f58fe7fc640 1 -- 172.21.15.16:0/2800497909 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f58e8039200 msgr2=0x7f58e803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/183909517 2024-09-19T09:18:13.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.033+0000 7f58fe7fc640 1 --2- 172.21.15.16:0/2800497909 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f58e8039200 0x7f58e803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:18:13.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.042+0000 7f58e7fff640 1 -- 172.21.15.16:0/2800497909 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f58f003b550 con 0x7f5900107730 2024-09-19T09:18:13.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.234+0000 7f58fe7fc640 1 -- 172.21.15.16:0/2800497909 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f58e8039200 msgr2=0x7f58e803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/183909517 2024-09-19T09:18:13.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.234+0000 7f58fe7fc640 1 --2- 172.21.15.16:0/2800497909 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f58e8039200 0x7f58e803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T09:18:13.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.268+0000 7f59054f9640 1 -- 172.21.15.16:0/2800497909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f590019f2e0 con 0x7f5900107730 2024-09-19T09:18:13.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.268+0000 7f58e7fff640 1 -- 172.21.15.16:0/2800497909 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f58f0043760 con 0x7f5900107730 2024-09-19T09:18:13.906 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.270+0000 7f59054f9640 1 -- 172.21.15.16:0/2800497909 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f58e8039200 msgr2=0x7f58e803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:18:13.907 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.270+0000 7f59054f9640 1 --2- 172.21.15.16:0/2800497909 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f58e8039200 0x7f58e803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.907 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.270+0000 7f59054f9640 1 -- 172.21.15.16:0/2800497909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 msgr2=0x7f590019b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:13.907 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.270+0000 7f59054f9640 1 --2- 172.21.15.16:0/2800497909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 0x7f590019b750 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f58f00097f0 tx=0x7f58f0030d80 comp rx=0 tx=0).stop 2024-09-19T09:18:13.907 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.270+0000 7f59054f9640 1 -- 172.21.15.16:0/2800497909 shutdown_connections 2024-09-19T09:18:13.907 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.270+0000 7f59054f9640 1 --2- 172.21.15.16:0/2800497909 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7f58e8039200 0x7f58e803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.907 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.270+0000 7f59054f9640 1 --2- 172.21.15.16:0/2800497909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5900107730 0x7f590019b750 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:13.907 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.270+0000 7f59054f9640 1 -- 172.21.15.16:0/2800497909 >> 172.21.15.16:0/2800497909 conn(0x7f5900102ea0 msgr2=0x7f5900103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:13.907 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.271+0000 7f59054f9640 1 -- 172.21.15.16:0/2800497909 shutdown_connections 2024-09-19T09:18:13.907 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:13.271+0000 7f59054f9640 1 -- 172.21.15.16:0/2800497909 wait complete. 2024-09-19T09:18:13.907 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-19T09:18:13.908 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 5... 2024-09-19T09:18:14.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:13 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2800497909' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T09:18:14.503 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:14 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:14.258+0000 7f979ec6d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T09:18:14.503 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:14 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:14.332+0000 7f979ec6d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T09:18:15.218 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:15 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:15.096+0000 7f979ec6d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T09:18:15.219 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:15 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:15.217+0000 7f979ec6d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T09:18:15.502 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:15 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:15.368+0000 7f979ec6d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:18:15.503 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:15 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:15.434+0000 7f979ec6d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T09:18:15.891 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:15 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:15.590+0000 7f979ec6d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T09:18:15.891 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:15 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:15.661+0000 7f979ec6d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T09:18:15.891 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:15 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:15.724+0000 7f979ec6d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T09:18:15.891 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:15 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:15.890+0000 7f979ec6d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T09:18:16.177 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:15 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:15.958+0000 7f979ec6d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T09:18:16.177 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:16 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:16.027+0000 7f979ec6d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T09:18:16.177 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:16 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:16.113+0000 7f979ec6d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T09:18:16.177 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:16 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:16.176+0000 7f979ec6d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T09:18:16.502 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:16 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:16.238+0000 7f979ec6d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T09:18:17.503 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:17.067+0000 7f979ec6d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T09:18:17.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: Active manager daemon x restarted 2024-09-19T09:18:17.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: Activating manager daemon x 2024-09-19T09:18:17.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T09:18:17.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: mgrmap e6: x(active, starting, since 0.00352299s) 2024-09-19T09:18:17.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:18:17.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T09:18:17.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T09:18:17.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T09:18:17.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T09:18:17.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: Manager daemon x is now available 2024-09-19T09:18:17.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' 2024-09-19T09:18:17.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' 2024-09-19T09:18:17.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:18:17.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:18:17.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T09:18:17.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:17 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T09:18:18.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:18 smithi016 ceph-mon[30979]: Found migration_current of "None". Setting to last migration. 2024-09-19T09:18:18.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:18 smithi016 ceph-mon[30979]: mgrmap e7: x(active, since 1.00535s) 2024-09-19T09:18:18.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:18:18.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T09:18:18.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T09:18:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:18:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.320+0000 7efd0e4cc640 1 Processor -- start 2024-09-19T09:18:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.320+0000 7efd0e4cc640 1 -- start start 2024-09-19T09:18:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.320+0000 7efd0e4cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 0x7efd0810a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.320+0000 7efd0e4cc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd0807be30 con 0x7efd08108150 2024-09-19T09:18:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.320+0000 7efd07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 0x7efd0810a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.320+0000 7efd07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 0x7efd0810a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41180/0 (socket says 172.21.15.16:41180) 2024-09-19T09:18:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.320+0000 7efd07fff640 1 -- 172.21.15.16:0/2289609192 learned_addr learned my addr 172.21.15.16:0/2289609192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.321+0000 7efd07fff640 1 -- 172.21.15.16:0/2289609192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0810aa80 con 0x7efd08108150 2024-09-19T09:18:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.321+0000 7efd07fff640 1 --2- 172.21.15.16:0/2289609192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 0x7efd0810a540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009820 tx=0x7efcf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc3129142e548b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.322+0000 7efd06ffd640 1 -- 172.21.15.16:0/2289609192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcf8037070 con 0x7efd08108150 2024-09-19T09:18:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.322+0000 7efd06ffd640 1 -- 172.21.15.16:0/2289609192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efcf8032900 con 0x7efd08108150 2024-09-19T09:18:18.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.322+0000 7efd06ffd640 1 -- 172.21.15.16:0/2289609192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcf8030850 con 0x7efd08108150 2024-09-19T09:18:18.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.322+0000 7efd0e4cc640 1 -- 172.21.15.16:0/2289609192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 msgr2=0x7efd0810a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:18.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.322+0000 7efd0e4cc640 1 --2- 172.21.15.16:0/2289609192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 0x7efd0810a540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009820 tx=0x7efcf802a270 comp rx=0 tx=0).stop 2024-09-19T09:18:18.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.322+0000 7efd0e4cc640 1 -- 172.21.15.16:0/2289609192 shutdown_connections 2024-09-19T09:18:18.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.323+0000 7efd0e4cc640 1 --2- 172.21.15.16:0/2289609192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 0x7efd0810a540 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:18.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.323+0000 7efd0e4cc640 1 -- 172.21.15.16:0/2289609192 >> 172.21.15.16:0/2289609192 conn(0x7efd08101d90 msgr2=0x7efd081041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:18.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.323+0000 7efd0e4cc640 1 -- 172.21.15.16:0/2289609192 shutdown_connections 2024-09-19T09:18:18.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.323+0000 7efd0e4cc640 1 -- 172.21.15.16:0/2289609192 wait complete. 2024-09-19T09:18:18.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.323+0000 7efd0e4cc640 1 Processor -- start 2024-09-19T09:18:18.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.324+0000 7efd0e4cc640 1 -- start start 2024-09-19T09:18:18.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.324+0000 7efd0e4cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 0x7efd0819b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:18.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.324+0000 7efd0e4cc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd0810bb20 con 0x7efd08108150 2024-09-19T09:18:18.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.324+0000 7efd07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 0x7efd0819b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:18.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.324+0000 7efd07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 0x7efd0819b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41188/0 (socket says 172.21.15.16:41188) 2024-09-19T09:18:18.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.324+0000 7efd07fff640 1 -- 172.21.15.16:0/3529164288 learned_addr learned my addr 172.21.15.16:0/3529164288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:18.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.324+0000 7efd07fff640 1 -- 172.21.15.16:0/3529164288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0819bbf0 con 0x7efd08108150 2024-09-19T09:18:18.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.325+0000 7efd07fff640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 0x7efd0819b6b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efcf80097f0 tx=0x7efcf8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:18.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.325+0000 7efd057fa640 1 -- 172.21.15.16:0/3529164288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcf8037040 con 0x7efd08108150 2024-09-19T09:18:18.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.325+0000 7efd057fa640 1 -- 172.21.15.16:0/3529164288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efcf8040070 con 0x7efd08108150 2024-09-19T09:18:18.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.325+0000 7efd0e4cc640 1 -- 172.21.15.16:0/3529164288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0819be80 con 0x7efd08108150 2024-09-19T09:18:18.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.325+0000 7efd0e4cc640 1 -- 172.21.15.16:0/3529164288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0819eb70 con 0x7efd08108150 2024-09-19T09:18:18.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.326+0000 7efd057fa640 1 -- 172.21.15.16:0/3529164288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcf803a390 con 0x7efd08108150 2024-09-19T09:18:18.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.326+0000 7efd057fa640 1 -- 172.21.15.16:0/3529164288 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7efcf803a660 con 0x7efd08108150 2024-09-19T09:18:18.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.326+0000 7efd057fa640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7efcf0039200 0x7efcf003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:18.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.326+0000 7efd057fa640 1 -- 172.21.15.16:0/3529164288 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efcf806d490 con 0x7efd08108150 2024-09-19T09:18:18.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.326+0000 7efd0e4cc640 1 -- 172.21.15.16:0/3529164288 --> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efcf4000d40 con 0x7efcf0039200 2024-09-19T09:18:18.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.326+0000 7efd077fe640 1 -- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7efcf0039200 msgr2=0x7efcf003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/183909517 2024-09-19T09:18:18.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.326+0000 7efd077fe640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7efcf0039200 0x7efcf003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:18:18.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.527+0000 7efd077fe640 1 -- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7efcf0039200 msgr2=0x7efcf003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/183909517 2024-09-19T09:18:18.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.527+0000 7efd077fe640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7efcf0039200 0x7efcf003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T09:18:18.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.928+0000 7efd077fe640 1 -- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7efcf0039200 msgr2=0x7efcf003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/183909517 2024-09-19T09:18:18.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:14.928+0000 7efd077fe640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7efcf0039200 0x7efcf003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T09:18:18.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:15.729+0000 7efd077fe640 1 -- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7efcf0039200 msgr2=0x7efcf003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/183909517 2024-09-19T09:18:18.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:15.729+0000 7efd077fe640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7efcf0039200 0x7efcf003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T09:18:18.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:17.071+0000 7efd057fa640 1 -- 172.21.15.16:0/3529164288 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7efcf806c340 con 0x7efd08108150 2024-09-19T09:18:18.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:17.071+0000 7efd057fa640 1 -- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7efcf0039200 msgr2=0x7efcf003b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:18:18.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:17.071+0000 7efd057fa640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7efcf0039200 0x7efcf003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:18.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.072+0000 7efd057fa640 1 -- 172.21.15.16:0/3529164288 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7efcf80457e0 con 0x7efd08108150 2024-09-19T09:18:18.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.072+0000 7efd057fa640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7efcf003c940 0x7efcf003ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:18.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.072+0000 7efd057fa640 1 -- 172.21.15.16:0/3529164288 --> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efcf8043a20 con 0x7efcf003c940 2024-09-19T09:18:18.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.072+0000 7efd077fe640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7efcf003c940 0x7efcf003ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:18.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.073+0000 7efd077fe640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7efcf003c940 0x7efcf003ed30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efcfc003980 tx=0x7efcfc0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:18.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.074+0000 7efd057fa640 1 -- 172.21.15.16:0/3529164288 <== mgr.14118 v2:172.21.15.16:6800/1680178094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7efcf8043a20 con 0x7efcf003c940 2024-09-19T09:18:18.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.081+0000 7efd0e4cc640 1 -- 172.21.15.16:0/3529164288 --> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7efcf40027b0 con 0x7efcf003c940 2024-09-19T09:18:18.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.081+0000 7efd057fa640 1 -- 172.21.15.16:0/3529164288 <== mgr.14118 v2:172.21.15.16:6800/1680178094 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7efcf40027b0 con 0x7efcf003c940 2024-09-19T09:18:18.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.081+0000 7efd0e4cc640 1 -- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7efcf003c940 msgr2=0x7efcf003ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:18.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.081+0000 7efd0e4cc640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7efcf003c940 0x7efcf003ed30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efcfc003980 tx=0x7efcfc0092b0 comp rx=0 tx=0).stop 2024-09-19T09:18:18.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.081+0000 7efd0e4cc640 1 -- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 msgr2=0x7efd0819b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:18.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.081+0000 7efd0e4cc640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 0x7efd0819b6b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efcf80097f0 tx=0x7efcf8030d80 comp rx=0 tx=0).stop 2024-09-19T09:18:18.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.081+0000 7efd0e4cc640 1 -- 172.21.15.16:0/3529164288 shutdown_connections 2024-09-19T09:18:18.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.081+0000 7efd0e4cc640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7efcf003c940 0x7efcf003ed30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:18.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.081+0000 7efd0e4cc640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:6800/183909517,v1:172.21.15.16:6801/183909517] conn(0x7efcf0039200 0x7efcf003b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:18.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.082+0000 7efd0e4cc640 1 --2- 172.21.15.16:0/3529164288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd08108150 0x7efd0819b6b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:18.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.082+0000 7efd0e4cc640 1 -- 172.21.15.16:0/3529164288 >> 172.21.15.16:0/3529164288 conn(0x7efd08101d90 msgr2=0x7efd08102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:18.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.082+0000 7efd0e4cc640 1 -- 172.21.15.16:0/3529164288 shutdown_connections 2024-09-19T09:18:18.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:18.082+0000 7efd0e4cc640 1 -- 172.21.15.16:0/3529164288 wait complete. 2024-09-19T09:18:18.701 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 5 is available 2024-09-19T09:18:18.701 INFO:teuthology.orchestra.run.smithi016.stdout:Setting orchestrator backend to cephadm... 2024-09-19T09:18:19.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.143+0000 7f9362e1b640 1 Processor -- start 2024-09-19T09:18:19.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.143+0000 7f9362e1b640 1 -- start start 2024-09-19T09:18:19.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.143+0000 7f9362e1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 0x7f935c10a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.144+0000 7f9362e1b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f935c07bdd0 con 0x7f935c1081a0 2024-09-19T09:18:19.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.144+0000 7f9360bb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 0x7f935c10a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.144+0000 7f9360bb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 0x7f935c10a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60794/0 (socket says 172.21.15.16:60794) 2024-09-19T09:18:19.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.144+0000 7f9360bb7640 1 -- 172.21.15.16:0/3787543286 learned_addr learned my addr 172.21.15.16:0/3787543286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:19.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.144+0000 7f9360bb7640 1 -- 172.21.15.16:0/3787543286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c10aad0 con 0x7f935c1081a0 2024-09-19T09:18:19.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.145+0000 7f9360bb7640 1 --2- 172.21.15.16:0/3787543286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 0x7f935c10a590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9350009820 tx=0x7f935002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653be4356918c667 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.145+0000 7f935b7fe640 1 -- 172.21.15.16:0/3787543286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9350037070 con 0x7f935c1081a0 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.145+0000 7f935b7fe640 1 -- 172.21.15.16:0/3787543286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9350032900 con 0x7f935c1081a0 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.145+0000 7f935b7fe640 1 -- 172.21.15.16:0/3787543286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9350030850 con 0x7f935c1081a0 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.145+0000 7f9362e1b640 1 -- 172.21.15.16:0/3787543286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 msgr2=0x7f935c10a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.145+0000 7f9362e1b640 1 --2- 172.21.15.16:0/3787543286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 0x7f935c10a590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9350009820 tx=0x7f935002a270 comp rx=0 tx=0).stop 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.146+0000 7f9362e1b640 1 -- 172.21.15.16:0/3787543286 shutdown_connections 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.146+0000 7f9362e1b640 1 --2- 172.21.15.16:0/3787543286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 0x7f935c10a590 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.146+0000 7f9362e1b640 1 -- 172.21.15.16:0/3787543286 >> 172.21.15.16:0/3787543286 conn(0x7f935c101e20 msgr2=0x7f935c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.146+0000 7f9362e1b640 1 -- 172.21.15.16:0/3787543286 shutdown_connections 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.146+0000 7f9362e1b640 1 -- 172.21.15.16:0/3787543286 wait complete. 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.146+0000 7f9362e1b640 1 Processor -- start 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.146+0000 7f9362e1b640 1 -- start start 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.146+0000 7f9362e1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 0x7f935c19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.146+0000 7f9362e1b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f935c10bb10 con 0x7f935c1081a0 2024-09-19T09:18:19.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.146+0000 7f9360bb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 0x7f935c19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.146+0000 7f9360bb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 0x7f935c19b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60808/0 (socket says 172.21.15.16:60808) 2024-09-19T09:18:19.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.146+0000 7f9360bb7640 1 -- 172.21.15.16:0/3863519906 learned_addr learned my addr 172.21.15.16:0/3863519906 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:19.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.147+0000 7f9360bb7640 1 -- 172.21.15.16:0/3863519906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c19bc10 con 0x7f935c1081a0 2024-09-19T09:18:19.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.147+0000 7f9360bb7640 1 --2- 172.21.15.16:0/3863519906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 0x7f935c19b6d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9350032ce0 tx=0x7f9350032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:19.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.147+0000 7f9359ffb640 1 -- 172.21.15.16:0/3863519906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9350042070 con 0x7f935c1081a0 2024-09-19T09:18:19.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.147+0000 7f9359ffb640 1 -- 172.21.15.16:0/3863519906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f93500308c0 con 0x7f935c1081a0 2024-09-19T09:18:19.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.147+0000 7f9362e1b640 1 -- 172.21.15.16:0/3863519906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935c19bea0 con 0x7f935c1081a0 2024-09-19T09:18:19.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.147+0000 7f9359ffb640 1 -- 172.21.15.16:0/3863519906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9350037040 con 0x7f935c1081a0 2024-09-19T09:18:19.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.147+0000 7f9362e1b640 1 -- 172.21.15.16:0/3863519906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935c19eb90 con 0x7f935c1081a0 2024-09-19T09:18:19.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.148+0000 7f9359ffb640 1 -- 172.21.15.16:0/3863519906 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f9350045070 con 0x7f935c1081a0 2024-09-19T09:18:19.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.148+0000 7f9362e1b640 1 -- 172.21.15.16:0/3863519906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935c108600 con 0x7f935c1081a0 2024-09-19T09:18:19.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.148+0000 7f9359ffb640 1 --2- 172.21.15.16:0/3863519906 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f934c0390a0 0x7f934c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:19.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.148+0000 7f9359ffb640 1 -- 172.21.15.16:0/3863519906 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f935006c170 con 0x7f935c1081a0 2024-09-19T09:18:19.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.149+0000 7f935bfff640 1 --2- 172.21.15.16:0/3863519906 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f934c0390a0 0x7f934c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:19.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.149+0000 7f935bfff640 1 --2- 172.21.15.16:0/3863519906 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f934c0390a0 0x7f934c03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f93480098c0 tx=0x7f9348006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:19.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.157+0000 7f9359ffb640 1 -- 172.21.15.16:0/3863519906 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f935004b170 con 0x7f935c1081a0 2024-09-19T09:18:19.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.347+0000 7f9362e1b640 1 -- 172.21.15.16:0/3863519906 --> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f935c1030f0 con 0x7f934c0390a0 2024-09-19T09:18:19.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.352+0000 7f9359ffb640 1 -- 172.21.15.16:0/3863519906 <== mgr.14118 v2:172.21.15.16:6800/1680178094 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f935c1030f0 con 0x7f934c0390a0 2024-09-19T09:18:19.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.354+0000 7f9362e1b640 1 -- 172.21.15.16:0/3863519906 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f934c0390a0 msgr2=0x7f934c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:19.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.354+0000 7f9362e1b640 1 --2- 172.21.15.16:0/3863519906 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f934c0390a0 0x7f934c03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f93480098c0 tx=0x7f9348006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.354+0000 7f9362e1b640 1 -- 172.21.15.16:0/3863519906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 msgr2=0x7f935c19b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:19.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.354+0000 7f9362e1b640 1 --2- 172.21.15.16:0/3863519906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 0x7f935c19b6d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9350032ce0 tx=0x7f9350032d10 comp rx=0 tx=0).stop 2024-09-19T09:18:19.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.354+0000 7f9362e1b640 1 -- 172.21.15.16:0/3863519906 shutdown_connections 2024-09-19T09:18:19.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.354+0000 7f9362e1b640 1 --2- 172.21.15.16:0/3863519906 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f934c0390a0 0x7f934c03b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.354+0000 7f9362e1b640 1 --2- 172.21.15.16:0/3863519906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f935c1081a0 0x7f935c19b6d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:19.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.354+0000 7f9362e1b640 1 -- 172.21.15.16:0/3863519906 >> 172.21.15.16:0/3863519906 conn(0x7f935c101e20 msgr2=0x7f935c1029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:19.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.354+0000 7f9362e1b640 1 -- 172.21.15.16:0/3863519906 shutdown_connections 2024-09-19T09:18:19.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:19.354+0000 7f9362e1b640 1 -- 172.21.15.16:0/3863519906 wait complete. 2024-09-19T09:18:20.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:19 smithi016 ceph-mon[30979]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T09:18:20.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:19 smithi016 ceph-mon[30979]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T09:18:20.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:19 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' 2024-09-19T09:18:20.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:19 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' 2024-09-19T09:18:20.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:19 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' 2024-09-19T09:18:20.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:19 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:18:20.201 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:20.163+0000 7f970b792640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:09:18:20] ENGINE Error in HTTPServer.serve 2024-09-19T09:18:20.201 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: Traceback (most recent call last): 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: self._connections.run(self.expiration_interval) 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: self._run(expiration_interval) 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: s = self.context.wrap_socket( 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: return self.sslsocket_class._create( 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T09:18:20.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: self.do_handshake() 2024-09-19T09:18:20.203 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T09:18:20.203 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: self._sslobj.do_handshake() 2024-09-19T09:18:20.203 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T09:18:20.203 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:21.173 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-mon[30979]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:18:21.174 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-mon[30979]: mgrmap e8: x(active, since 2s) 2024-09-19T09:18:21.174 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:20 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:18:21.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T09:18:21.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.381+0000 7fd64b6c2640 1 Processor -- start 2024-09-19T09:18:21.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.381+0000 7fd64b6c2640 1 -- start start 2024-09-19T09:18:21.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.381+0000 7fd64b6c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 0x7fd644105e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.381+0000 7fd64b6c2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6441063d0 con 0x7fd644105a00 2024-09-19T09:18:21.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.381+0000 7fd64945e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 0x7fd644105e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.382+0000 7fd64945e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 0x7fd644105e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60814/0 (socket says 172.21.15.16:60814) 2024-09-19T09:18:21.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.382+0000 7fd64945e640 1 -- 172.21.15.16:0/2693256028 learned_addr learned my addr 172.21.15.16:0/2693256028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:21.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.382+0000 7fd64945e640 1 -- 172.21.15.16:0/2693256028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd644106770 con 0x7fd644105a00 2024-09-19T09:18:21.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.382+0000 7fd64945e640 1 --2- 172.21.15.16:0/2693256028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 0x7fd644105e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd640009820 tx=0x7fd64002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9269b3586bc526 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:21.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.383+0000 7fd63bfff640 1 -- 172.21.15.16:0/2693256028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd640037070 con 0x7fd644105a00 2024-09-19T09:18:21.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.383+0000 7fd63bfff640 1 -- 172.21.15.16:0/2693256028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd640032900 con 0x7fd644105a00 2024-09-19T09:18:21.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.383+0000 7fd63bfff640 1 -- 172.21.15.16:0/2693256028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd640030850 con 0x7fd644105a00 2024-09-19T09:18:21.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.383+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2693256028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 msgr2=0x7fd644105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:21.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.383+0000 7fd64b6c2640 1 --2- 172.21.15.16:0/2693256028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 0x7fd644105e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd640009820 tx=0x7fd64002a270 comp rx=0 tx=0).stop 2024-09-19T09:18:21.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.383+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2693256028 shutdown_connections 2024-09-19T09:18:21.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.383+0000 7fd64b6c2640 1 --2- 172.21.15.16:0/2693256028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 0x7fd644105e00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.383+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2693256028 >> 172.21.15.16:0/2693256028 conn(0x7fd6441015d0 msgr2=0x7fd6441039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.383+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2693256028 shutdown_connections 2024-09-19T09:18:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.383+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2693256028 wait complete. 2024-09-19T09:18:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.384+0000 7fd64b6c2640 1 Processor -- start 2024-09-19T09:18:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.384+0000 7fd64b6c2640 1 -- start start 2024-09-19T09:18:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.384+0000 7fd64b6c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 0x7fd64419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.384+0000 7fd64b6c2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6441079b0 con 0x7fd644105a00 2024-09-19T09:18:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.384+0000 7fd64945e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 0x7fd64419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.384+0000 7fd64945e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 0x7fd64419d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60816/0 (socket says 172.21.15.16:60816) 2024-09-19T09:18:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.384+0000 7fd64945e640 1 -- 172.21.15.16:0/2747586767 learned_addr learned my addr 172.21.15.16:0/2747586767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.384+0000 7fd64945e640 1 -- 172.21.15.16:0/2747586767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64419de90 con 0x7fd644105a00 2024-09-19T09:18:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.384+0000 7fd64945e640 1 --2- 172.21.15.16:0/2747586767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 0x7fd64419d950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd640031000 tx=0x7fd6400326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.385+0000 7fd63a7fc640 1 -- 172.21.15.16:0/2747586767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd640042070 con 0x7fd644105a00 2024-09-19T09:18:21.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.385+0000 7fd63a7fc640 1 -- 172.21.15.16:0/2747586767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd640032be0 con 0x7fd644105a00 2024-09-19T09:18:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.385+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2747586767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64419e120 con 0x7fd644105a00 2024-09-19T09:18:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.385+0000 7fd63a7fc640 1 -- 172.21.15.16:0/2747586767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd640037040 con 0x7fd644105a00 2024-09-19T09:18:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.385+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2747586767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6441a0e10 con 0x7fd644105a00 2024-09-19T09:18:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.386+0000 7fd63a7fc640 1 -- 172.21.15.16:0/2747586767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fd640045070 con 0x7fd644105a00 2024-09-19T09:18:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.386+0000 7fd63a7fc640 1 --2- 172.21.15.16:0/2747586767 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fd630039200 0x7fd63003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.386+0000 7fd63a7fc640 1 -- 172.21.15.16:0/2747586767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd64006d2a0 con 0x7fd644105a00 2024-09-19T09:18:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.386+0000 7fd648c5d640 1 --2- 172.21.15.16:0/2747586767 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fd630039200 0x7fd63003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.386+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2747586767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd644105e00 con 0x7fd644105a00 2024-09-19T09:18:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.386+0000 7fd648c5d640 1 --2- 172.21.15.16:0/2747586767 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fd630039200 0x7fd63003b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6340098c0 tx=0x7fd634006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.391+0000 7fd63a7fc640 1 -- 172.21.15.16:0/2747586767 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd640042210 con 0x7fd644105a00 2024-09-19T09:18:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.567+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2747586767 --> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fd644074c10 con 0x7fd630039200 2024-09-19T09:18:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.568+0000 7fd63a7fc640 1 -- 172.21.15.16:0/2747586767 <== mgr.14118 v2:172.21.15.16:6800/1680178094 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fd644074c10 con 0x7fd630039200 2024-09-19T09:18:21.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.570+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2747586767 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fd630039200 msgr2=0x7fd63003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:21.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.570+0000 7fd64b6c2640 1 --2- 172.21.15.16:0/2747586767 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fd630039200 0x7fd63003b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6340098c0 tx=0x7fd634006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.570+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2747586767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 msgr2=0x7fd64419d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:21.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.570+0000 7fd64b6c2640 1 --2- 172.21.15.16:0/2747586767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 0x7fd64419d950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd640031000 tx=0x7fd6400326e0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.570+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2747586767 shutdown_connections 2024-09-19T09:18:21.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.570+0000 7fd64b6c2640 1 --2- 172.21.15.16:0/2747586767 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fd630039200 0x7fd63003b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.570+0000 7fd64b6c2640 1 --2- 172.21.15.16:0/2747586767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd644105a00 0x7fd64419d950 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:21.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.570+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2747586767 >> 172.21.15.16:0/2747586767 conn(0x7fd6441015d0 msgr2=0x7fd644193f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:21.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.570+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2747586767 shutdown_connections 2024-09-19T09:18:21.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:20.570+0000 7fd64b6c2640 1 -- 172.21.15.16:0/2747586767 wait complete. 2024-09-19T09:18:21.185 INFO:teuthology.orchestra.run.smithi016.stdout:Generating ssh key... 2024-09-19T09:18:22.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:21 smithi016 ceph-mon[30979]: [19/Sep/2024:09:18:20] ENGINE Bus STARTING 2024-09-19T09:18:22.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:21 smithi016 ceph-mon[30979]: [19/Sep/2024:09:18:20] ENGINE Serving on https://172.21.15.16:7150 2024-09-19T09:18:22.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:21 smithi016 ceph-mon[30979]: [19/Sep/2024:09:18:20] ENGINE Error in HTTPServer.serve 2024-09-19T09:18:22.253 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-19T09:18:22.253 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-19T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T09:18:22.255 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-19T09:18:22.255 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T09:18:22.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:21 smithi016 ceph-mon[30979]: [19/Sep/2024:09:18:20] ENGINE Serving on http://172.21.15.16:8765 2024-09-19T09:18:22.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:21 smithi016 ceph-mon[30979]: [19/Sep/2024:09:18:20] ENGINE Bus STARTED 2024-09-19T09:18:22.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:21 smithi016 ceph-mon[30979]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:18:22.255 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: Generating public/private rsa key pair. 2024-09-19T09:18:22.255 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: Your identification has been saved in /tmp/tmp1jy_02xj/key 2024-09-19T09:18:22.255 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: Your public key has been saved in /tmp/tmp1jy_02xj/key.pub 2024-09-19T09:18:22.255 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: The key fingerprint is: 2024-09-19T09:18:22.255 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: SHA256:xTCBiOCwqJtoMZZ5glGH23rfaq30/zYdX22EsubT5XM ceph-ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:18:22.255 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: The key's randomart image is: 2024-09-19T09:18:22.256 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: +---[RSA 3072]----+ 2024-09-19T09:18:22.256 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: |o..o.. .+. | 2024-09-19T09:18:22.256 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: |+oo.. . + | 2024-09-19T09:18:22.256 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: |+. o o . | 2024-09-19T09:18:22.256 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: |o.+ . . . . .| 2024-09-19T09:18:22.256 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: |oB o S o ..| 2024-09-19T09:18:22.256 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: |oo* . o . =| 2024-09-19T09:18:22.256 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: |+. . ..o o o *.| 2024-09-19T09:18:22.256 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: |. .o.o = o.E| 2024-09-19T09:18:22.256 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: | .oo...o.o o| 2024-09-19T09:18:22.256 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: +----[SHA256]-----+ 2024-09-19T09:18:22.587 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.600+0000 7f418bbaa640 1 Processor -- start 2024-09-19T09:18:22.587 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.600+0000 7f418bbaa640 1 -- start start 2024-09-19T09:18:22.587 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.600+0000 7f418bbaa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 0x7f418410a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:22.587 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.600+0000 7f418bbaa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f418407be30 con 0x7f4184108180 2024-09-19T09:18:22.587 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.601+0000 7f4189946640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 0x7f418410a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:22.587 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.601+0000 7f4189946640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 0x7f418410a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60830/0 (socket says 172.21.15.16:60830) 2024-09-19T09:18:22.587 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.601+0000 7f4189946640 1 -- 172.21.15.16:0/53720075 learned_addr learned my addr 172.21.15.16:0/53720075 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:22.587 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.601+0000 7f4189946640 1 -- 172.21.15.16:0/53720075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418410aab0 con 0x7f4184108180 2024-09-19T09:18:22.588 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.602+0000 7f4189946640 1 --2- 172.21.15.16:0/53720075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 0x7f418410a570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4180009820 tx=0x7f418002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53579a7deb627e82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:22.588 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.602+0000 7f4188944640 1 -- 172.21.15.16:0/53720075 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4180037070 con 0x7f4184108180 2024-09-19T09:18:22.588 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.602+0000 7f4188944640 1 -- 172.21.15.16:0/53720075 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4180032900 con 0x7f4184108180 2024-09-19T09:18:22.588 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.602+0000 7f4188944640 1 -- 172.21.15.16:0/53720075 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4180030850 con 0x7f4184108180 2024-09-19T09:18:22.588 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.603+0000 7f418bbaa640 1 -- 172.21.15.16:0/53720075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 msgr2=0x7f418410a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:22.588 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.603+0000 7f418bbaa640 1 --2- 172.21.15.16:0/53720075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 0x7f418410a570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4180009820 tx=0x7f418002a270 comp rx=0 tx=0).stop 2024-09-19T09:18:22.588 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.603+0000 7f418bbaa640 1 -- 172.21.15.16:0/53720075 shutdown_connections 2024-09-19T09:18:22.588 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.603+0000 7f418bbaa640 1 --2- 172.21.15.16:0/53720075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 0x7f418410a570 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:22.588 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.603+0000 7f418bbaa640 1 -- 172.21.15.16:0/53720075 >> 172.21.15.16:0/53720075 conn(0x7f4184101e20 msgr2=0x7f4184104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:22.588 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.603+0000 7f418bbaa640 1 -- 172.21.15.16:0/53720075 shutdown_connections 2024-09-19T09:18:22.588 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.603+0000 7f418bbaa640 1 -- 172.21.15.16:0/53720075 wait complete. 2024-09-19T09:18:22.588 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.603+0000 7f418bbaa640 1 Processor -- start 2024-09-19T09:18:22.588 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.603+0000 7f418bbaa640 1 -- start start 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.604+0000 7f418bbaa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 0x7f418419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.604+0000 7f418bbaa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f418410bb50 con 0x7f4184108180 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.604+0000 7f4189946640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 0x7f418419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.604+0000 7f4189946640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 0x7f418419d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60838/0 (socket says 172.21.15.16:60838) 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.604+0000 7f4189946640 1 -- 172.21.15.16:0/96818053 learned_addr learned my addr 172.21.15.16:0/96818053 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.604+0000 7f4189946640 1 -- 172.21.15.16:0/96818053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418419dea0 con 0x7f4184108180 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.604+0000 7f4189946640 1 --2- 172.21.15.16:0/96818053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 0x7f418419d960 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4180032ce0 tx=0x7f4180032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.604+0000 7f417affd640 1 -- 172.21.15.16:0/96818053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4180042070 con 0x7f4184108180 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.604+0000 7f417affd640 1 -- 172.21.15.16:0/96818053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f41800308c0 con 0x7f4184108180 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.604+0000 7f418bbaa640 1 -- 172.21.15.16:0/96818053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418419e130 con 0x7f4184108180 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.604+0000 7f417affd640 1 -- 172.21.15.16:0/96818053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4180037040 con 0x7f4184108180 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.604+0000 7f418bbaa640 1 -- 172.21.15.16:0/96818053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41841a0e20 con 0x7f4184108180 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.605+0000 7f417affd640 1 -- 172.21.15.16:0/96818053 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4180045070 con 0x7f4184108180 2024-09-19T09:18:22.589 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.605+0000 7f418bbaa640 1 -- 172.21.15.16:0/96818053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41841085e0 con 0x7f4184108180 2024-09-19T09:18:22.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.605+0000 7f417affd640 1 --2- 172.21.15.16:0/96818053 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f41700391b0 0x7f417003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:22.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.606+0000 7f417affd640 1 -- 172.21.15.16:0/96818053 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f418006c590 con 0x7f4184108180 2024-09-19T09:18:22.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.606+0000 7f4189145640 1 --2- 172.21.15.16:0/96818053 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f41700391b0 0x7f417003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:22.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.606+0000 7f4189145640 1 --2- 172.21.15.16:0/96818053 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f41700391b0 0x7f417003b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f41740098c0 tx=0x7f4174006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:22.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.610+0000 7f417affd640 1 -- 172.21.15.16:0/96818053 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f418003cd00 con 0x7f4184108180 2024-09-19T09:18:22.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:21.787+0000 7f418bbaa640 1 -- 172.21.15.16:0/96818053 --> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f4184103070 con 0x7f41700391b0 2024-09-19T09:18:22.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.011+0000 7f417affd640 1 -- 172.21.15.16:0/96818053 <== mgr.14118 v2:172.21.15.16:6800/1680178094 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4184103070 con 0x7f41700391b0 2024-09-19T09:18:22.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.013+0000 7f418bbaa640 1 -- 172.21.15.16:0/96818053 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f41700391b0 msgr2=0x7f417003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:22.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.013+0000 7f418bbaa640 1 --2- 172.21.15.16:0/96818053 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f41700391b0 0x7f417003b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f41740098c0 tx=0x7f4174006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:22.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.013+0000 7f418bbaa640 1 -- 172.21.15.16:0/96818053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 msgr2=0x7f418419d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:22.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.013+0000 7f418bbaa640 1 --2- 172.21.15.16:0/96818053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 0x7f418419d960 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4180032ce0 tx=0x7f4180032d10 comp rx=0 tx=0).stop 2024-09-19T09:18:22.590 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.013+0000 7f418bbaa640 1 -- 172.21.15.16:0/96818053 shutdown_connections 2024-09-19T09:18:22.591 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.013+0000 7f418bbaa640 1 --2- 172.21.15.16:0/96818053 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f41700391b0 0x7f417003b670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:22.591 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.013+0000 7f418bbaa640 1 --2- 172.21.15.16:0/96818053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4184108180 0x7f418419d960 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:22.591 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.013+0000 7f418bbaa640 1 -- 172.21.15.16:0/96818053 >> 172.21.15.16:0/96818053 conn(0x7f4184101e20 msgr2=0x7f4184102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:22.591 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.013+0000 7f418bbaa640 1 -- 172.21.15.16:0/96818053 shutdown_connections 2024-09-19T09:18:22.591 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.013+0000 7f418bbaa640 1 -- 172.21.15.16:0/96818053 wait complete. 2024-09-19T09:18:23.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:23 smithi016 ceph-mon[30979]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:18:23.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:23 smithi016 ceph-mon[30979]: Generating ssh key... 2024-09-19T09:18:23.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:23 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' 2024-09-19T09:18:23.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:23 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' 2024-09-19T09:18:23.810 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuHle617f3RXOILWbfJhgf0L8Xhfr2UqtJVK408oKQMn7I4t6Z2F/OhcaxF3OtvfJ/s3yJKqTMI3vUdBYng95srxt0SE8bQzTsQlvRpRZa69D4k6qAC9oqE5X8Q66lZezws/7/5fOf6etvwizQPlzwQA14g+9JTo2rdCpndH7DEkwIYSrRUCUKMUAH/mRGS/zCJfWPmB5BUqXhwTUpiH8SvdwUKi3hswz6DaxB1/KHjTOqSTnAek2zfDVkAYcD/1ABjVSib+eUPA0Oabxp61VCF7gkSSskwRF1Bt9UI8K78jMXkSVLKJq5nQ/ydf6wCpUaRt+13PdcyFVSW/DU1ZSlHjWaCqiDa4mvEwRcTmbEBHAlDyUdFvCKySSoEh/FQwna1/dfXo+T1wWWLaVklDzuowB9s3w8NTfKkGzIZPaSnNsSUxoA5UxV9eVCy3EwzdYIg6nzj7E8slEvVNglslSiGCnBKvffFlp/ZRYHm3jU2tgbh9tuKnVwU0xjj6lsBeU= ceph-ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:18:23.810 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.987+0000 7feb08873640 1 Processor -- start 2024-09-19T09:18:23.810 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.987+0000 7feb08873640 1 -- start start 2024-09-19T09:18:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.987+0000 7feb08873640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 0x7feb0410a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.987+0000 7feb08873640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb0407bdf0 con 0x7feb04108180 2024-09-19T09:18:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.987+0000 7feb0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 0x7feb0410a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.987+0000 7feb0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 0x7feb0410a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60854/0 (socket says 172.21.15.16:60854) 2024-09-19T09:18:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.987+0000 7feb0259c640 1 -- 172.21.15.16:0/1548035107 learned_addr learned my addr 172.21.15.16:0/1548035107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.988+0000 7feb0259c640 1 -- 172.21.15.16:0/1548035107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0410aab0 con 0x7feb04108180 2024-09-19T09:18:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.988+0000 7feb0259c640 1 --2- 172.21.15.16:0/1548035107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 0x7feb0410a570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009820 tx=0x7feaf402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1226403cd38492 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.988+0000 7feb0159a640 1 -- 172.21.15.16:0/1548035107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaf4037070 con 0x7feb04108180 2024-09-19T09:18:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.989+0000 7feb0159a640 1 -- 172.21.15.16:0/1548035107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feaf4032900 con 0x7feb04108180 2024-09-19T09:18:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.989+0000 7feb0159a640 1 -- 172.21.15.16:0/1548035107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaf4030850 con 0x7feb04108180 2024-09-19T09:18:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.989+0000 7feb08873640 1 -- 172.21.15.16:0/1548035107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 msgr2=0x7feb0410a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.989+0000 7feb08873640 1 --2- 172.21.15.16:0/1548035107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 0x7feb0410a570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009820 tx=0x7feaf402a270 comp rx=0 tx=0).stop 2024-09-19T09:18:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.989+0000 7feb08873640 1 -- 172.21.15.16:0/1548035107 shutdown_connections 2024-09-19T09:18:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.989+0000 7feb08873640 1 --2- 172.21.15.16:0/1548035107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 0x7feb0410a570 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.989+0000 7feb08873640 1 -- 172.21.15.16:0/1548035107 >> 172.21.15.16:0/1548035107 conn(0x7feb04101e20 msgr2=0x7feb04104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.989+0000 7feb08873640 1 -- 172.21.15.16:0/1548035107 shutdown_connections 2024-09-19T09:18:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.989+0000 7feb08873640 1 -- 172.21.15.16:0/1548035107 wait complete. 2024-09-19T09:18:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.990+0000 7feb08873640 1 Processor -- start 2024-09-19T09:18:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.990+0000 7feb08873640 1 -- start start 2024-09-19T09:18:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.990+0000 7feb08873640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 0x7feb0419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.990+0000 7feb08873640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb0410baf0 con 0x7feb04108180 2024-09-19T09:18:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.990+0000 7feb0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 0x7feb0419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.990+0000 7feb0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 0x7feb0419b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60864/0 (socket says 172.21.15.16:60864) 2024-09-19T09:18:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.990+0000 7feb0259c640 1 -- 172.21.15.16:0/559045408 learned_addr learned my addr 172.21.15.16:0/559045408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.991+0000 7feb0259c640 1 -- 172.21.15.16:0/559045408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0419bc10 con 0x7feb04108180 2024-09-19T09:18:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.991+0000 7feb0259c640 1 --2- 172.21.15.16:0/559045408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 0x7feb0419b6d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feaf4032ce0 tx=0x7feaf4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.991+0000 7feaf37fe640 1 -- 172.21.15.16:0/559045408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaf4042070 con 0x7feb04108180 2024-09-19T09:18:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.991+0000 7feaf37fe640 1 -- 172.21.15.16:0/559045408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feaf40308c0 con 0x7feb04108180 2024-09-19T09:18:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.991+0000 7feaf37fe640 1 -- 172.21.15.16:0/559045408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaf4037040 con 0x7feb04108180 2024-09-19T09:18:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.991+0000 7feb08873640 1 -- 172.21.15.16:0/559045408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0419bea0 con 0x7feb04108180 2024-09-19T09:18:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.991+0000 7feb08873640 1 -- 172.21.15.16:0/559045408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0419eb90 con 0x7feb04108180 2024-09-19T09:18:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.992+0000 7feaf37fe640 1 -- 172.21.15.16:0/559045408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7feaf4045070 con 0x7feb04108180 2024-09-19T09:18:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.992+0000 7feaf37fe640 1 --2- 172.21.15.16:0/559045408 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7feaec039200 0x7feaec03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.992+0000 7feaf37fe640 1 -- 172.21.15.16:0/559045408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feaf406c390 con 0x7feb04108180 2024-09-19T09:18:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.992+0000 7feb01d9b640 1 --2- 172.21.15.16:0/559045408 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7feaec039200 0x7feaec03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.992+0000 7feb08873640 1 -- 172.21.15.16:0/559045408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb041085e0 con 0x7feb04108180 2024-09-19T09:18:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.993+0000 7feb01d9b640 1 --2- 172.21.15.16:0/559045408 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7feaec039200 0x7feaec03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feaf80098c0 tx=0x7feaf8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:22.997+0000 7feaf37fe640 1 -- 172.21.15.16:0/559045408 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feaf4045360 con 0x7feb04108180 2024-09-19T09:18:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:23.173+0000 7feb08873640 1 -- 172.21.15.16:0/559045408 --> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7feb04103120 con 0x7feaec039200 2024-09-19T09:18:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:23.173+0000 7feaf37fe640 1 -- 172.21.15.16:0/559045408 <== mgr.14118 v2:172.21.15.16:6800/1680178094 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7feb04103120 con 0x7feaec039200 2024-09-19T09:18:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:23.175+0000 7feb08873640 1 -- 172.21.15.16:0/559045408 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7feaec039200 msgr2=0x7feaec03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:23.175+0000 7feb08873640 1 --2- 172.21.15.16:0/559045408 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7feaec039200 0x7feaec03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feaf80098c0 tx=0x7feaf8006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:23.175+0000 7feb08873640 1 -- 172.21.15.16:0/559045408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 msgr2=0x7feb0419b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:23.175+0000 7feb08873640 1 --2- 172.21.15.16:0/559045408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 0x7feb0419b6d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feaf4032ce0 tx=0x7feaf4032d10 comp rx=0 tx=0).stop 2024-09-19T09:18:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:23.175+0000 7feb08873640 1 -- 172.21.15.16:0/559045408 shutdown_connections 2024-09-19T09:18:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:23.175+0000 7feb08873640 1 --2- 172.21.15.16:0/559045408 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7feaec039200 0x7feaec03b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:23.175+0000 7feb08873640 1 --2- 172.21.15.16:0/559045408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb04108180 0x7feb0419b6d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:23.175+0000 7feb08873640 1 -- 172.21.15.16:0/559045408 >> 172.21.15.16:0/559045408 conn(0x7feb04101e20 msgr2=0x7feb04102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:23.175+0000 7feb08873640 1 -- 172.21.15.16:0/559045408 shutdown_connections 2024-09-19T09:18:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:23.175+0000 7feb08873640 1 -- 172.21.15.16:0/559045408 wait complete. 2024-09-19T09:18:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T09:18:23.815 INFO:teuthology.orchestra.run.smithi016.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T09:18:23.815 INFO:teuthology.orchestra.run.smithi016.stdout:Adding host smithi016... 2024-09-19T09:18:24.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:24 smithi016 ceph-mon[30979]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:18:25.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:25 smithi016 ceph-mon[30979]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:18:26.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:26 smithi016 ceph-mon[30979]: Deploying cephadm binary to smithi016 2024-09-19T09:18:29.001 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Added host 'smithi016' with addr '172.21.15.16' 2024-09-19T09:18:29.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.313+0000 7fa247eca640 1 Processor -- start 2024-09-19T09:18:29.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.314+0000 7fa247eca640 1 -- start start 2024-09-19T09:18:29.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.314+0000 7fa247eca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 0x7fa240107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:29.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.314+0000 7fa247eca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa240108150 con 0x7fa240107780 2024-09-19T09:18:29.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.314+0000 7fa245c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 0x7fa240107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:29.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.314+0000 7fa245c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 0x7fa240107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60878/0 (socket says 172.21.15.16:60878) 2024-09-19T09:18:29.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.314+0000 7fa245c66640 1 -- 172.21.15.16:0/135037375 learned_addr learned my addr 172.21.15.16:0/135037375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:29.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.314+0000 7fa245c66640 1 -- 172.21.15.16:0/135037375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa240108980 con 0x7fa240107780 2024-09-19T09:18:29.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.315+0000 7fa245c66640 1 --2- 172.21.15.16:0/135037375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 0x7fa240107b80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009820 tx=0x7fa23c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287c073f2b900209 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:29.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.315+0000 7fa244c64640 1 -- 172.21.15.16:0/135037375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa23c037070 con 0x7fa240107780 2024-09-19T09:18:29.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.315+0000 7fa244c64640 1 -- 172.21.15.16:0/135037375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa23c032900 con 0x7fa240107780 2024-09-19T09:18:29.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.315+0000 7fa244c64640 1 -- 172.21.15.16:0/135037375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa23c030850 con 0x7fa240107780 2024-09-19T09:18:29.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.316+0000 7fa247eca640 1 -- 172.21.15.16:0/135037375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 msgr2=0x7fa240107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:29.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.316+0000 7fa247eca640 1 --2- 172.21.15.16:0/135037375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 0x7fa240107b80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009820 tx=0x7fa23c02a270 comp rx=0 tx=0).stop 2024-09-19T09:18:29.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.316+0000 7fa247eca640 1 -- 172.21.15.16:0/135037375 shutdown_connections 2024-09-19T09:18:29.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.316+0000 7fa247eca640 1 --2- 172.21.15.16:0/135037375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 0x7fa240107b80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:29.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.316+0000 7fa247eca640 1 -- 172.21.15.16:0/135037375 >> 172.21.15.16:0/135037375 conn(0x7fa240102f30 msgr2=0x7fa240105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:29.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.316+0000 7fa247eca640 1 -- 172.21.15.16:0/135037375 shutdown_connections 2024-09-19T09:18:29.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.316+0000 7fa247eca640 1 -- 172.21.15.16:0/135037375 wait complete. 2024-09-19T09:18:29.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.317+0000 7fa247eca640 1 Processor -- start 2024-09-19T09:18:29.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.317+0000 7fa247eca640 1 -- start start 2024-09-19T09:18:29.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.317+0000 7fa247eca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 0x7fa24019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:29.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.317+0000 7fa247eca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa24010bd30 con 0x7fa240107780 2024-09-19T09:18:29.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.317+0000 7fa245c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 0x7fa24019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:29.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.317+0000 7fa245c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 0x7fa24019b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60892/0 (socket says 172.21.15.16:60892) 2024-09-19T09:18:29.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.317+0000 7fa245c66640 1 -- 172.21.15.16:0/2210915382 learned_addr learned my addr 172.21.15.16:0/2210915382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:29.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.317+0000 7fa245c66640 1 -- 172.21.15.16:0/2210915382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24019bc50 con 0x7fa240107780 2024-09-19T09:18:29.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.318+0000 7fa245c66640 1 --2- 172.21.15.16:0/2210915382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 0x7fa24019b710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0097f0 tx=0x7fa23c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:29.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.318+0000 7fa236ffd640 1 -- 172.21.15.16:0/2210915382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa23c037040 con 0x7fa240107780 2024-09-19T09:18:29.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.318+0000 7fa236ffd640 1 -- 172.21.15.16:0/2210915382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa23c040070 con 0x7fa240107780 2024-09-19T09:18:29.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.318+0000 7fa247eca640 1 -- 172.21.15.16:0/2210915382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24019bee0 con 0x7fa240107780 2024-09-19T09:18:29.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.318+0000 7fa236ffd640 1 -- 172.21.15.16:0/2210915382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa23c03a390 con 0x7fa240107780 2024-09-19T09:18:29.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.318+0000 7fa247eca640 1 -- 172.21.15.16:0/2210915382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24019ebd0 con 0x7fa240107780 2024-09-19T09:18:29.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.319+0000 7fa236ffd640 1 -- 172.21.15.16:0/2210915382 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa23c03b430 con 0x7fa240107780 2024-09-19T09:18:29.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.319+0000 7fa247eca640 1 -- 172.21.15.16:0/2210915382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa240107c00 con 0x7fa240107780 2024-09-19T09:18:29.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.319+0000 7fa236ffd640 1 --2- 172.21.15.16:0/2210915382 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fa22c039200 0x7fa22c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:29.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.319+0000 7fa236ffd640 1 -- 172.21.15.16:0/2210915382 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa23c06c7d0 con 0x7fa240107780 2024-09-19T09:18:29.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.319+0000 7fa245465640 1 --2- 172.21.15.16:0/2210915382 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fa22c039200 0x7fa22c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:29.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.320+0000 7fa245465640 1 --2- 172.21.15.16:0/2210915382 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fa22c039200 0x7fa22c03b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa2300098c0 tx=0x7fa230006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:29.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.324+0000 7fa236ffd640 1 -- 172.21.15.16:0/2210915382 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa23c06c120 con 0x7fa240107780 2024-09-19T09:18:29.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:24.507+0000 7fa247eca640 1 -- 172.21.15.16:0/2210915382 --> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}) -- 0x7fa2401041b0 con 0x7fa22c039200 2024-09-19T09:18:29.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:28.406+0000 7fa236ffd640 1 -- 172.21.15.16:0/2210915382 <== mgr.14118 v2:172.21.15.16:6800/1680178094 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fa2401041b0 con 0x7fa22c039200 2024-09-19T09:18:29.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:28.409+0000 7fa247eca640 1 -- 172.21.15.16:0/2210915382 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fa22c039200 msgr2=0x7fa22c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:29.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:28.409+0000 7fa247eca640 1 --2- 172.21.15.16:0/2210915382 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fa22c039200 0x7fa22c03b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa2300098c0 tx=0x7fa230006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:29.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:28.409+0000 7fa247eca640 1 -- 172.21.15.16:0/2210915382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 msgr2=0x7fa24019b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:29.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:28.409+0000 7fa247eca640 1 --2- 172.21.15.16:0/2210915382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 0x7fa24019b710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0097f0 tx=0x7fa23c030d80 comp rx=0 tx=0).stop 2024-09-19T09:18:29.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:28.409+0000 7fa247eca640 1 -- 172.21.15.16:0/2210915382 shutdown_connections 2024-09-19T09:18:29.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:28.409+0000 7fa247eca640 1 --2- 172.21.15.16:0/2210915382 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fa22c039200 0x7fa22c03b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:29.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:28.409+0000 7fa247eca640 1 --2- 172.21.15.16:0/2210915382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa240107780 0x7fa24019b710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:29.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:28.409+0000 7fa247eca640 1 -- 172.21.15.16:0/2210915382 >> 172.21.15.16:0/2210915382 conn(0x7fa240102f30 msgr2=0x7fa240103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:29.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:28.409+0000 7fa247eca640 1 -- 172.21.15.16:0/2210915382 shutdown_connections 2024-09-19T09:18:29.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:28.409+0000 7fa247eca640 1 -- 172.21.15.16:0/2210915382 wait complete. 2024-09-19T09:18:29.006 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mon service... 2024-09-19T09:18:29.697 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:29 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' 2024-09-19T09:18:29.697 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:29 smithi016 ceph-mon[30979]: Added host smithi016 2024-09-19T09:18:29.697 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:29 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:18:30.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T09:18:30.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.465+0000 7f47d1117640 1 Processor -- start 2024-09-19T09:18:30.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.465+0000 7f47d1117640 1 -- start start 2024-09-19T09:18:30.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.466+0000 7f47d1117640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 0x7f47cc10a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.466+0000 7f47d1117640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc07bdd0 con 0x7f47cc1081a0 2024-09-19T09:18:30.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.466+0000 7f47cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 0x7f47cc10a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.466+0000 7f47cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 0x7f47cc10a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51272/0 (socket says 172.21.15.16:51272) 2024-09-19T09:18:30.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.466+0000 7f47cad9d640 1 -- 172.21.15.16:0/4178075891 learned_addr learned my addr 172.21.15.16:0/4178075891 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:30.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.466+0000 7f47cad9d640 1 -- 172.21.15.16:0/4178075891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc10aad0 con 0x7f47cc1081a0 2024-09-19T09:18:30.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.467+0000 7f47cad9d640 1 --2- 172.21.15.16:0/4178075891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 0x7f47cc10a590 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009820 tx=0x7f47bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301a6c2f6d0048df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:30.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.467+0000 7f47ca59c640 1 -- 172.21.15.16:0/4178075891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47bc037070 con 0x7f47cc1081a0 2024-09-19T09:18:30.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.467+0000 7f47ca59c640 1 -- 172.21.15.16:0/4178075891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f47bc032900 con 0x7f47cc1081a0 2024-09-19T09:18:30.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.467+0000 7f47ca59c640 1 -- 172.21.15.16:0/4178075891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47bc030850 con 0x7f47cc1081a0 2024-09-19T09:18:30.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.468+0000 7f47d1117640 1 -- 172.21.15.16:0/4178075891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 msgr2=0x7f47cc10a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:30.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.468+0000 7f47d1117640 1 --2- 172.21.15.16:0/4178075891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 0x7f47cc10a590 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009820 tx=0x7f47bc02a270 comp rx=0 tx=0).stop 2024-09-19T09:18:30.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.468+0000 7f47d1117640 1 -- 172.21.15.16:0/4178075891 shutdown_connections 2024-09-19T09:18:30.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.468+0000 7f47d1117640 1 --2- 172.21.15.16:0/4178075891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 0x7f47cc10a590 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.468+0000 7f47d1117640 1 -- 172.21.15.16:0/4178075891 >> 172.21.15.16:0/4178075891 conn(0x7f47cc101e20 msgr2=0x7f47cc104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:30.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.468+0000 7f47d1117640 1 -- 172.21.15.16:0/4178075891 shutdown_connections 2024-09-19T09:18:30.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.468+0000 7f47d1117640 1 -- 172.21.15.16:0/4178075891 wait complete. 2024-09-19T09:18:30.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.468+0000 7f47d1117640 1 Processor -- start 2024-09-19T09:18:30.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.469+0000 7f47d1117640 1 -- start start 2024-09-19T09:18:30.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.469+0000 7f47d1117640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 0x7f47cc19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.469+0000 7f47d1117640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc10bb10 con 0x7f47cc1081a0 2024-09-19T09:18:30.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.469+0000 7f47cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 0x7f47cc19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.469+0000 7f47cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 0x7f47cc19b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51278/0 (socket says 172.21.15.16:51278) 2024-09-19T09:18:30.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.469+0000 7f47cad9d640 1 -- 172.21.15.16:0/3762250141 learned_addr learned my addr 172.21.15.16:0/3762250141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:30.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.469+0000 7f47cad9d640 1 -- 172.21.15.16:0/3762250141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc19bc10 con 0x7f47cc1081a0 2024-09-19T09:18:30.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.469+0000 7f47cad9d640 1 --2- 172.21.15.16:0/3762250141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 0x7f47cc19b6d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f47bc032ce0 tx=0x7f47bc032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:30.499 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.469+0000 7f47c8d99640 1 -- 172.21.15.16:0/3762250141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47bc042070 con 0x7f47cc1081a0 2024-09-19T09:18:30.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.469+0000 7f47c8d99640 1 -- 172.21.15.16:0/3762250141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f47bc0308c0 con 0x7f47cc1081a0 2024-09-19T09:18:30.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.469+0000 7f47d1117640 1 -- 172.21.15.16:0/3762250141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc19bea0 con 0x7f47cc1081a0 2024-09-19T09:18:30.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.469+0000 7f47c8d99640 1 -- 172.21.15.16:0/3762250141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47bc037040 con 0x7f47cc1081a0 2024-09-19T09:18:30.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.469+0000 7f47d1117640 1 -- 172.21.15.16:0/3762250141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47cc19eb90 con 0x7f47cc1081a0 2024-09-19T09:18:30.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.470+0000 7f47c8d99640 1 -- 172.21.15.16:0/3762250141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f47bc045070 con 0x7f47cc1081a0 2024-09-19T09:18:30.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.470+0000 7f47c8d99640 1 --2- 172.21.15.16:0/3762250141 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f47b8039200 0x7f47b803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:30.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.471+0000 7f47c8d99640 1 -- 172.21.15.16:0/3762250141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f47bc06c370 con 0x7f47cc1081a0 2024-09-19T09:18:30.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.471+0000 7f47c3fff640 1 --2- 172.21.15.16:0/3762250141 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f47b8039200 0x7f47b803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:30.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.471+0000 7f47d1117640 1 -- 172.21.15.16:0/3762250141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47cc108600 con 0x7f47cc1081a0 2024-09-19T09:18:30.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.471+0000 7f47c3fff640 1 --2- 172.21.15.16:0/3762250141 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f47b8039200 0x7f47b803b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f47b40098c0 tx=0x7f47b4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:30.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.476+0000 7f47c8d99640 1 -- 172.21.15.16:0/3762250141 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f47bc045520 con 0x7f47cc1081a0 2024-09-19T09:18:30.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.695+0000 7f47d1117640 1 -- 172.21.15.16:0/3762250141 --> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f47cc1030f0 con 0x7f47b8039200 2024-09-19T09:18:30.500 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.700+0000 7f47c8d99640 1 -- 172.21.15.16:0/3762250141 <== mgr.14118 v2:172.21.15.16:6800/1680178094 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f47cc1030f0 con 0x7f47b8039200 2024-09-19T09:18:30.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.702+0000 7f47d1117640 1 -- 172.21.15.16:0/3762250141 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f47b8039200 msgr2=0x7f47b803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:30.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.702+0000 7f47d1117640 1 --2- 172.21.15.16:0/3762250141 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f47b8039200 0x7f47b803b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f47b40098c0 tx=0x7f47b4006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.702+0000 7f47d1117640 1 -- 172.21.15.16:0/3762250141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 msgr2=0x7f47cc19b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:30.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.702+0000 7f47d1117640 1 --2- 172.21.15.16:0/3762250141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 0x7f47cc19b6d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f47bc032ce0 tx=0x7f47bc032d10 comp rx=0 tx=0).stop 2024-09-19T09:18:30.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.702+0000 7f47d1117640 1 -- 172.21.15.16:0/3762250141 shutdown_connections 2024-09-19T09:18:30.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.702+0000 7f47d1117640 1 --2- 172.21.15.16:0/3762250141 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f47b8039200 0x7f47b803b6c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.702+0000 7f47d1117640 1 --2- 172.21.15.16:0/3762250141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47cc1081a0 0x7f47cc19b6d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:30.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.702+0000 7f47d1117640 1 -- 172.21.15.16:0/3762250141 >> 172.21.15.16:0/3762250141 conn(0x7f47cc101e20 msgr2=0x7f47cc1029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:30.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.702+0000 7f47d1117640 1 -- 172.21.15.16:0/3762250141 shutdown_connections 2024-09-19T09:18:30.501 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:29.702+0000 7f47d1117640 1 -- 172.21.15.16:0/3762250141 wait complete. 2024-09-19T09:18:30.501 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mgr service... 2024-09-19T09:18:30.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:30 smithi016 ceph-mon[30979]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:18:30.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:30 smithi016 ceph-mon[30979]: Saving service mon spec with placement count:5 2024-09-19T09:18:30.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:30 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' 2024-09-19T09:18:32.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T09:18:32.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.114+0000 7f6eb1094640 1 Processor -- start 2024-09-19T09:18:32.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.114+0000 7f6eb1094640 1 -- start start 2024-09-19T09:18:32.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.114+0000 7f6eb1094640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 0x7f6eac10a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.114+0000 7f6eb1094640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac07bdb0 con 0x7f6eac1081c0 2024-09-19T09:18:32.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.115+0000 7f6eaad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 0x7f6eac10a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.115+0000 7f6eaad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 0x7f6eac10a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51292/0 (socket says 172.21.15.16:51292) 2024-09-19T09:18:32.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.115+0000 7f6eaad9d640 1 -- 172.21.15.16:0/1287708038 learned_addr learned my addr 172.21.15.16:0/1287708038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:32.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.115+0000 7f6eaad9d640 1 -- 172.21.15.16:0/1287708038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac10aaf0 con 0x7f6eac1081c0 2024-09-19T09:18:32.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.116+0000 7f6eaad9d640 1 --2- 172.21.15.16:0/1287708038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 0x7f6eac10a5b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009e80 tx=0x7f6e9c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239474f240c24bbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:32.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.116+0000 7f6ea9d9b640 1 -- 172.21.15.16:0/1287708038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e9c036070 con 0x7f6eac1081c0 2024-09-19T09:18:32.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.116+0000 7f6ea9d9b640 1 -- 172.21.15.16:0/1287708038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e9c0318c0 con 0x7f6eac1081c0 2024-09-19T09:18:32.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.116+0000 7f6ea9d9b640 1 -- 172.21.15.16:0/1287708038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e9c007ab0 con 0x7f6eac1081c0 2024-09-19T09:18:32.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.116+0000 7f6eb1094640 1 -- 172.21.15.16:0/1287708038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 msgr2=0x7f6eac10a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:32.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.116+0000 7f6eb1094640 1 --2- 172.21.15.16:0/1287708038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 0x7f6eac10a5b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009e80 tx=0x7f6e9c02a7a0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.117+0000 7f6eb1094640 1 -- 172.21.15.16:0/1287708038 shutdown_connections 2024-09-19T09:18:32.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.117+0000 7f6eb1094640 1 --2- 172.21.15.16:0/1287708038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 0x7f6eac10a5b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.117+0000 7f6eb1094640 1 -- 172.21.15.16:0/1287708038 >> 172.21.15.16:0/1287708038 conn(0x7f6eac101e20 msgr2=0x7f6eac104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:32.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.117+0000 7f6eb1094640 1 -- 172.21.15.16:0/1287708038 shutdown_connections 2024-09-19T09:18:32.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.117+0000 7f6eb1094640 1 -- 172.21.15.16:0/1287708038 wait complete. 2024-09-19T09:18:32.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.117+0000 7f6eb1094640 1 Processor -- start 2024-09-19T09:18:32.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.118+0000 7f6eb1094640 1 -- start start 2024-09-19T09:18:32.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.118+0000 7f6eb1094640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 0x7f6eac107a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.118+0000 7f6eb1094640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac10bad0 con 0x7f6eac1081c0 2024-09-19T09:18:32.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.118+0000 7f6eaad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 0x7f6eac107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.118+0000 7f6eaad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 0x7f6eac107a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51300/0 (socket says 172.21.15.16:51300) 2024-09-19T09:18:32.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.118+0000 7f6eaad9d640 1 -- 172.21.15.16:0/2412171541 learned_addr learned my addr 172.21.15.16:0/2412171541 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:32.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.118+0000 7f6eaad9d640 1 -- 172.21.15.16:0/2412171541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac107f80 con 0x7f6eac1081c0 2024-09-19T09:18:32.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.118+0000 7f6eaad9d640 1 --2- 172.21.15.16:0/2412171541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 0x7f6eac107a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c02acc0 tx=0x7f6e9c031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:32.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.118+0000 7f6e93fff640 1 -- 172.21.15.16:0/2412171541 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e9c036050 con 0x7f6eac1081c0 2024-09-19T09:18:32.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.119+0000 7f6e93fff640 1 -- 172.21.15.16:0/2412171541 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e9c037040 con 0x7f6eac1081c0 2024-09-19T09:18:32.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.119+0000 7f6eb1094640 1 -- 172.21.15.16:0/2412171541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eac106170 con 0x7f6eac1081c0 2024-09-19T09:18:32.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.119+0000 7f6e93fff640 1 -- 172.21.15.16:0/2412171541 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e9c03d850 con 0x7f6eac1081c0 2024-09-19T09:18:32.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.119+0000 7f6eb1094640 1 -- 172.21.15.16:0/2412171541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eac1066b0 con 0x7f6eac1081c0 2024-09-19T09:18:32.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.119+0000 7f6e93fff640 1 -- 172.21.15.16:0/2412171541 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f6e9c044020 con 0x7f6eac1081c0 2024-09-19T09:18:32.119 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.120+0000 7f6e93fff640 1 --2- 172.21.15.16:0/2412171541 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f6e940391b0 0x7f6e9403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:32.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.120+0000 7f6e93fff640 1 -- 172.21.15.16:0/2412171541 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6e9c042020 con 0x7f6eac1081c0 2024-09-19T09:18:32.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.120+0000 7f6eaa59c640 1 --2- 172.21.15.16:0/2412171541 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f6e940391b0 0x7f6e9403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:32.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.120+0000 7f6eb1094640 1 -- 172.21.15.16:0/2412171541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eac108620 con 0x7f6eac1081c0 2024-09-19T09:18:32.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.120+0000 7f6eaa59c640 1 --2- 172.21.15.16:0/2412171541 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f6e940391b0 0x7f6e9403b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00098c0 tx=0x7f6ea0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:32.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.125+0000 7f6e93fff640 1 -- 172.21.15.16:0/2412171541 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6e9c04b020 con 0x7f6eac1081c0 2024-09-19T09:18:32.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.317+0000 7f6eb1094640 1 -- 172.21.15.16:0/2412171541 --> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6eac106cf0 con 0x7f6e940391b0 2024-09-19T09:18:32.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.321+0000 7f6e93fff640 1 -- 172.21.15.16:0/2412171541 <== mgr.14118 v2:172.21.15.16:6800/1680178094 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6eac106cf0 con 0x7f6e940391b0 2024-09-19T09:18:32.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.323+0000 7f6eb1094640 1 -- 172.21.15.16:0/2412171541 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f6e940391b0 msgr2=0x7f6e9403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:32.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.323+0000 7f6eb1094640 1 --2- 172.21.15.16:0/2412171541 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f6e940391b0 0x7f6e9403b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00098c0 tx=0x7f6ea0006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.323+0000 7f6eb1094640 1 -- 172.21.15.16:0/2412171541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 msgr2=0x7f6eac107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:32.120 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.323+0000 7f6eb1094640 1 --2- 172.21.15.16:0/2412171541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 0x7f6eac107a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c02acc0 tx=0x7f6e9c031ca0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.323+0000 7f6eb1094640 1 -- 172.21.15.16:0/2412171541 shutdown_connections 2024-09-19T09:18:32.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.323+0000 7f6eb1094640 1 --2- 172.21.15.16:0/2412171541 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f6e940391b0 0x7f6e9403b670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.323+0000 7f6eb1094640 1 --2- 172.21.15.16:0/2412171541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eac1081c0 0x7f6eac107a40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:32.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.323+0000 7f6eb1094640 1 -- 172.21.15.16:0/2412171541 >> 172.21.15.16:0/2412171541 conn(0x7f6eac101e20 msgr2=0x7f6eac1029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:32.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.323+0000 7f6eb1094640 1 -- 172.21.15.16:0/2412171541 shutdown_connections 2024-09-19T09:18:32.121 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:31.323+0000 7f6eb1094640 1 -- 172.21.15.16:0/2412171541 wait complete. 2024-09-19T09:18:32.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:32 smithi016 ceph-mon[30979]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:18:32.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:32 smithi016 ceph-mon[30979]: Saving service mgr spec with placement count:2 2024-09-19T09:18:32.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:32 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' 2024-09-19T09:18:33.595 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.550+0000 7f6b61040640 1 Processor -- start 2024-09-19T09:18:33.595 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.550+0000 7f6b61040640 1 -- start start 2024-09-19T09:18:33.595 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.550+0000 7f6b61040640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 0x7f6b5c1063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:33.595 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.550+0000 7f6b61040640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c07bd90 con 0x7f6b5c105fd0 2024-09-19T09:18:33.595 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.551+0000 7f6b5ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 0x7f6b5c1063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:33.596 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.551+0000 7f6b5ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 0x7f6b5c1063d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51316/0 (socket says 172.21.15.16:51316) 2024-09-19T09:18:33.596 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.551+0000 7f6b5ad9d640 1 -- 172.21.15.16:0/3890254101 learned_addr learned my addr 172.21.15.16:0/3890254101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:33.596 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.551+0000 7f6b5ad9d640 1 -- 172.21.15.16:0/3890254101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c106910 con 0x7f6b5c105fd0 2024-09-19T09:18:33.596 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.552+0000 7f6b5ad9d640 1 --2- 172.21.15.16:0/3890254101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 0x7f6b5c1063d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009820 tx=0x7f6b5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be524f4d22772ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:33.596 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.552+0000 7f6b59d9b640 1 -- 172.21.15.16:0/3890254101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b54037070 con 0x7f6b5c105fd0 2024-09-19T09:18:33.596 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.552+0000 7f6b59d9b640 1 -- 172.21.15.16:0/3890254101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b54032900 con 0x7f6b5c105fd0 2024-09-19T09:18:33.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.552+0000 7f6b59d9b640 1 -- 172.21.15.16:0/3890254101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b54030850 con 0x7f6b5c105fd0 2024-09-19T09:18:33.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.552+0000 7f6b61040640 1 -- 172.21.15.16:0/3890254101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 msgr2=0x7f6b5c1063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:33.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.552+0000 7f6b61040640 1 --2- 172.21.15.16:0/3890254101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 0x7f6b5c1063d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009820 tx=0x7f6b5402a270 comp rx=0 tx=0).stop 2024-09-19T09:18:33.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.552+0000 7f6b61040640 1 -- 172.21.15.16:0/3890254101 shutdown_connections 2024-09-19T09:18:33.597 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.552+0000 7f6b61040640 1 --2- 172.21.15.16:0/3890254101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 0x7f6b5c1063d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:33.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.552+0000 7f6b61040640 1 -- 172.21.15.16:0/3890254101 >> 172.21.15.16:0/3890254101 conn(0x7f6b5c101e20 msgr2=0x7f6b5c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:33.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.552+0000 7f6b61040640 1 -- 172.21.15.16:0/3890254101 shutdown_connections 2024-09-19T09:18:33.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.553+0000 7f6b61040640 1 -- 172.21.15.16:0/3890254101 wait complete. 2024-09-19T09:18:33.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.553+0000 7f6b61040640 1 Processor -- start 2024-09-19T09:18:33.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.554+0000 7f6b61040640 1 -- start start 2024-09-19T09:18:33.598 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.554+0000 7f6b61040640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 0x7f6b5c19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:33.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.554+0000 7f6b61040640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c107a10 con 0x7f6b5c105fd0 2024-09-19T09:18:33.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.554+0000 7f6b5ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 0x7f6b5c19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:33.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.554+0000 7f6b5ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 0x7f6b5c19d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51330/0 (socket says 172.21.15.16:51330) 2024-09-19T09:18:33.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.554+0000 7f6b5ad9d640 1 -- 172.21.15.16:0/37026705 learned_addr learned my addr 172.21.15.16:0/37026705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:33.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.554+0000 7f6b5ad9d640 1 -- 172.21.15.16:0/37026705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c19de00 con 0x7f6b5c105fd0 2024-09-19T09:18:33.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.554+0000 7f6b5ad9d640 1 --2- 172.21.15.16:0/37026705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 0x7f6b5c19d8c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b540097f0 tx=0x7f6b54030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:33.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.555+0000 7f6b53fff640 1 -- 172.21.15.16:0/37026705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b54037040 con 0x7f6b5c105fd0 2024-09-19T09:18:33.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.555+0000 7f6b53fff640 1 -- 172.21.15.16:0/37026705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b54040070 con 0x7f6b5c105fd0 2024-09-19T09:18:33.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.555+0000 7f6b53fff640 1 -- 172.21.15.16:0/37026705 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b5403a390 con 0x7f6b5c105fd0 2024-09-19T09:18:33.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.555+0000 7f6b61040640 1 -- 172.21.15.16:0/37026705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5c19e090 con 0x7f6b5c105fd0 2024-09-19T09:18:33.600 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.555+0000 7f6b61040640 1 -- 172.21.15.16:0/37026705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b5c1a0d80 con 0x7f6b5c105fd0 2024-09-19T09:18:33.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.556+0000 7f6b53fff640 1 -- 172.21.15.16:0/37026705 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f6b5403a5e0 con 0x7f6b5c105fd0 2024-09-19T09:18:33.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.556+0000 7f6b61040640 1 -- 172.21.15.16:0/37026705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b5c1063d0 con 0x7f6b5c105fd0 2024-09-19T09:18:33.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.556+0000 7f6b53fff640 1 --2- 172.21.15.16:0/37026705 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f6b48039200 0x7f6b4803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:33.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.556+0000 7f6b53fff640 1 -- 172.21.15.16:0/37026705 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b5406c860 con 0x7f6b5c105fd0 2024-09-19T09:18:33.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.556+0000 7f6b5a59c640 1 --2- 172.21.15.16:0/37026705 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f6b48039200 0x7f6b4803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:33.601 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.557+0000 7f6b5a59c640 1 --2- 172.21.15.16:0/37026705 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f6b48039200 0x7f6b4803b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6b440098c0 tx=0x7f6b44006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:33.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.561+0000 7f6b53fff640 1 -- 172.21.15.16:0/37026705 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6b54043760 con 0x7f6b5c105fd0 2024-09-19T09:18:33.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.740+0000 7f6b61040640 1 -- 172.21.15.16:0/37026705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f6b5c196d30 con 0x7f6b5c105fd0 2024-09-19T09:18:33.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.742+0000 7f6b53fff640 1 -- 172.21.15.16:0/37026705 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f6b5403a9d0 con 0x7f6b5c105fd0 2024-09-19T09:18:33.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.744+0000 7f6b61040640 1 -- 172.21.15.16:0/37026705 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f6b48039200 msgr2=0x7f6b4803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:33.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.744+0000 7f6b61040640 1 --2- 172.21.15.16:0/37026705 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f6b48039200 0x7f6b4803b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6b440098c0 tx=0x7f6b44006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:33.602 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.744+0000 7f6b61040640 1 -- 172.21.15.16:0/37026705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 msgr2=0x7f6b5c19d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:33.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.744+0000 7f6b61040640 1 --2- 172.21.15.16:0/37026705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 0x7f6b5c19d8c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b540097f0 tx=0x7f6b54030d80 comp rx=0 tx=0).stop 2024-09-19T09:18:33.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.744+0000 7f6b61040640 1 -- 172.21.15.16:0/37026705 shutdown_connections 2024-09-19T09:18:33.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.744+0000 7f6b61040640 1 --2- 172.21.15.16:0/37026705 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f6b48039200 0x7f6b4803b6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:33.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.744+0000 7f6b61040640 1 --2- 172.21.15.16:0/37026705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b5c105fd0 0x7f6b5c19d8c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:33.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.744+0000 7f6b61040640 1 -- 172.21.15.16:0/37026705 >> 172.21.15.16:0/37026705 conn(0x7f6b5c101e20 msgr2=0x7f6b5c191bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:33.603 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.744+0000 7f6b61040640 1 -- 172.21.15.16:0/37026705 shutdown_connections 2024-09-19T09:18:33.604 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:32.744+0000 7f6b61040640 1 -- 172.21.15.16:0/37026705 wait complete. 2024-09-19T09:18:33.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:33 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/37026705' entity='client.admin' 2024-09-19T09:18:33.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:33 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' 2024-09-19T09:18:33.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:33 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' 2024-09-19T09:18:35.418 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:35 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1286638635' entity='client.admin' 2024-09-19T09:18:35.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:35 smithi016 ceph-mon[30979]: from='mgr.14118 172.21.15.16:0/4011210594' entity='mgr.x' 2024-09-19T09:18:35.426 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.967+0000 7f041c77e640 1 Processor -- start 2024-09-19T09:18:35.426 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.967+0000 7f041c77e640 1 -- start start 2024-09-19T09:18:35.426 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.968+0000 7f041c77e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 0x7f0414107b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:35.426 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.968+0000 7f041c77e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04141080d0 con 0x7f0414107700 2024-09-19T09:18:35.426 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.968+0000 7f041a51a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 0x7f0414107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:35.426 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.968+0000 7f041a51a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 0x7f0414107b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51346/0 (socket says 172.21.15.16:51346) 2024-09-19T09:18:35.426 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.968+0000 7f041a51a640 1 -- 172.21.15.16:0/4002384856 learned_addr learned my addr 172.21.15.16:0/4002384856 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:35.426 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.968+0000 7f041a51a640 1 -- 172.21.15.16:0/4002384856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0414108900 con 0x7f0414107700 2024-09-19T09:18:35.426 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.969+0000 7f041a51a640 1 --2- 172.21.15.16:0/4002384856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 0x7f0414107b00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0410009820 tx=0x7f041002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9374d1ab6749a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:35.426 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.969+0000 7f0419518640 1 -- 172.21.15.16:0/4002384856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0410037070 con 0x7f0414107700 2024-09-19T09:18:35.426 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.969+0000 7f0419518640 1 -- 172.21.15.16:0/4002384856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0410032920 con 0x7f0414107700 2024-09-19T09:18:35.426 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.969+0000 7f0419518640 1 -- 172.21.15.16:0/4002384856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0410030810 con 0x7f0414107700 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.970+0000 7f041c77e640 1 -- 172.21.15.16:0/4002384856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 msgr2=0x7f0414107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.970+0000 7f041c77e640 1 --2- 172.21.15.16:0/4002384856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 0x7f0414107b00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0410009820 tx=0x7f041002a270 comp rx=0 tx=0).stop 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.970+0000 7f041c77e640 1 -- 172.21.15.16:0/4002384856 shutdown_connections 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.970+0000 7f041c77e640 1 --2- 172.21.15.16:0/4002384856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 0x7f0414107b00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.970+0000 7f041c77e640 1 -- 172.21.15.16:0/4002384856 >> 172.21.15.16:0/4002384856 conn(0x7f0414102f30 msgr2=0x7f0414105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.970+0000 7f041c77e640 1 -- 172.21.15.16:0/4002384856 shutdown_connections 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.970+0000 7f041c77e640 1 -- 172.21.15.16:0/4002384856 wait complete. 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.971+0000 7f041c77e640 1 Processor -- start 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.971+0000 7f041c77e640 1 -- start start 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.971+0000 7f041c77e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 0x7f041419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.971+0000 7f041c77e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f041410bcb0 con 0x7f0414107700 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.971+0000 7f041a51a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 0x7f041419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.971+0000 7f041a51a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 0x7f041419b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51348/0 (socket says 172.21.15.16:51348) 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.971+0000 7f041a51a640 1 -- 172.21.15.16:0/1286638635 learned_addr learned my addr 172.21.15.16:0/1286638635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:35.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.971+0000 7f041a51a640 1 -- 172.21.15.16:0/1286638635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041419bc70 con 0x7f0414107700 2024-09-19T09:18:35.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.971+0000 7f041a51a640 1 --2- 172.21.15.16:0/1286638635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 0x7f041419b730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0410030560 tx=0x7f0410032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:35.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.972+0000 7f040b7fe640 1 -- 172.21.15.16:0/1286638635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0410037040 con 0x7f0414107700 2024-09-19T09:18:35.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.972+0000 7f040b7fe640 1 -- 172.21.15.16:0/1286638635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0410040070 con 0x7f0414107700 2024-09-19T09:18:35.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.972+0000 7f041c77e640 1 -- 172.21.15.16:0/1286638635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041419bf00 con 0x7f0414107700 2024-09-19T09:18:35.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.972+0000 7f040b7fe640 1 -- 172.21.15.16:0/1286638635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f041003a740 con 0x7f0414107700 2024-09-19T09:18:35.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.972+0000 7f041c77e640 1 -- 172.21.15.16:0/1286638635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041419ebf0 con 0x7f0414107700 2024-09-19T09:18:35.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.972+0000 7f040b7fe640 1 -- 172.21.15.16:0/1286638635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f041003b420 con 0x7f0414107700 2024-09-19T09:18:35.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.973+0000 7f041c77e640 1 -- 172.21.15.16:0/1286638635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f041410be30 con 0x7f0414107700 2024-09-19T09:18:35.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.973+0000 7f040b7fe640 1 --2- 172.21.15.16:0/1286638635 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f0400039200 0x7f040003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:35.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.973+0000 7f040b7fe640 1 -- 172.21.15.16:0/1286638635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f041006c820 con 0x7f0414107700 2024-09-19T09:18:35.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.973+0000 7f0419d19640 1 --2- 172.21.15.16:0/1286638635 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f0400039200 0x7f040003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:35.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.973+0000 7f0419d19640 1 --2- 172.21.15.16:0/1286638635 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f0400039200 0x7f040003b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f04040098c0 tx=0x7f0404006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:35.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:33.978+0000 7f040b7fe640 1 -- 172.21.15.16:0/1286638635 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f041004a200 con 0x7f0414107700 2024-09-19T09:18:35.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:34.165+0000 7f041c77e640 1 -- 172.21.15.16:0/1286638635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f0414107b80 con 0x7f0414107700 2024-09-19T09:18:35.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:34.167+0000 7f040b7fe640 1 -- 172.21.15.16:0/1286638635 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f0410031e80 con 0x7f0414107700 2024-09-19T09:18:35.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:34.170+0000 7f041c77e640 1 -- 172.21.15.16:0/1286638635 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f0400039200 msgr2=0x7f040003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:35.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:34.170+0000 7f041c77e640 1 --2- 172.21.15.16:0/1286638635 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f0400039200 0x7f040003b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f04040098c0 tx=0x7f0404006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:35.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:34.170+0000 7f041c77e640 1 -- 172.21.15.16:0/1286638635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 msgr2=0x7f041419b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:35.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:34.170+0000 7f041c77e640 1 --2- 172.21.15.16:0/1286638635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 0x7f041419b730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0410030560 tx=0x7f0410032670 comp rx=0 tx=0).stop 2024-09-19T09:18:35.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:34.170+0000 7f041c77e640 1 -- 172.21.15.16:0/1286638635 shutdown_connections 2024-09-19T09:18:35.432 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:34.170+0000 7f041c77e640 1 --2- 172.21.15.16:0/1286638635 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f0400039200 0x7f040003b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:35.432 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:34.170+0000 7f041c77e640 1 --2- 172.21.15.16:0/1286638635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0414107700 0x7f041419b730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:35.432 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:34.170+0000 7f041c77e640 1 -- 172.21.15.16:0/1286638635 >> 172.21.15.16:0/1286638635 conn(0x7f0414102f30 msgr2=0x7f0414103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:35.432 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:34.170+0000 7f041c77e640 1 -- 172.21.15.16:0/1286638635 shutdown_connections 2024-09-19T09:18:35.432 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:34.170+0000 7f041c77e640 1 -- 172.21.15.16:0/1286638635 wait complete. 2024-09-19T09:18:35.432 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling the dashboard module... 2024-09-19T09:18:36.327 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:36 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2569928987' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T09:18:36.327 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: ignoring --setuser ceph since I am not root 2024-09-19T09:18:36.327 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: ignoring --setgroup ceph since I am not root 2024-09-19T09:18:36.327 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:36.325+0000 7f450a255040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T09:18:36.614 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:36.385+0000 7f450a255040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T09:18:36.614 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:36.543+0000 7f450a255040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T09:18:36.614 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:36.613+0000 7f450a255040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T09:18:36.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.847+0000 7f0ec0fb8640 1 Processor -- start 2024-09-19T09:18:36.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.847+0000 7f0ec0fb8640 1 -- start start 2024-09-19T09:18:36.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.848+0000 7f0ec0fb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 0x7f0ebc10a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:36.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.848+0000 7f0ec0fb8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc07be30 con 0x7f0ebc108140 2024-09-19T09:18:36.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.848+0000 7f0eba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 0x7f0ebc10a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:36.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.848+0000 7f0eba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 0x7f0ebc10a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58828/0 (socket says 172.21.15.16:58828) 2024-09-19T09:18:36.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.848+0000 7f0eba59c640 1 -- 172.21.15.16:0/1971987184 learned_addr learned my addr 172.21.15.16:0/1971987184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:36.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.848+0000 7f0eba59c640 1 -- 172.21.15.16:0/1971987184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc10aa70 con 0x7f0ebc108140 2024-09-19T09:18:36.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.849+0000 7f0eba59c640 1 --2- 172.21.15.16:0/1971987184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 0x7f0ebc10a530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009820 tx=0x7f0eac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cdd00d32f66d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:36.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.849+0000 7f0eb9d9b640 1 -- 172.21.15.16:0/1971987184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eac037070 con 0x7f0ebc108140 2024-09-19T09:18:36.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.849+0000 7f0eb9d9b640 1 -- 172.21.15.16:0/1971987184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eac032920 con 0x7f0ebc108140 2024-09-19T09:18:36.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.849+0000 7f0eb9d9b640 1 -- 172.21.15.16:0/1971987184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eac030810 con 0x7f0ebc108140 2024-09-19T09:18:36.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.850+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/1971987184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 msgr2=0x7f0ebc10a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:36.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.850+0000 7f0ec0fb8640 1 --2- 172.21.15.16:0/1971987184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 0x7f0ebc10a530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009820 tx=0x7f0eac02a270 comp rx=0 tx=0).stop 2024-09-19T09:18:36.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.850+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/1971987184 shutdown_connections 2024-09-19T09:18:36.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.850+0000 7f0ec0fb8640 1 --2- 172.21.15.16:0/1971987184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 0x7f0ebc10a530 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:36.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.850+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/1971987184 >> 172.21.15.16:0/1971987184 conn(0x7f0ebc101e20 msgr2=0x7f0ebc104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:36.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.850+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/1971987184 shutdown_connections 2024-09-19T09:18:36.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.850+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/1971987184 wait complete. 2024-09-19T09:18:36.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.850+0000 7f0ec0fb8640 1 Processor -- start 2024-09-19T09:18:36.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.851+0000 7f0ec0fb8640 1 -- start start 2024-09-19T09:18:36.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.851+0000 7f0ec0fb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 0x7f0ebc1079c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:36.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.851+0000 7f0ec0fb8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc10bb50 con 0x7f0ebc108140 2024-09-19T09:18:36.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.851+0000 7f0eba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 0x7f0ebc1079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.851+0000 7f0eba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 0x7f0ebc1079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58834/0 (socket says 172.21.15.16:58834) 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.851+0000 7f0eba59c640 1 -- 172.21.15.16:0/2569928987 learned_addr learned my addr 172.21.15.16:0/2569928987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.851+0000 7f0eba59c640 1 -- 172.21.15.16:0/2569928987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc107f00 con 0x7f0ebc108140 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.851+0000 7f0eba59c640 1 --2- 172.21.15.16:0/2569928987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 0x7f0ebc1079c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0097f0 tx=0x7f0eac0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.851+0000 7f0eb37fe640 1 -- 172.21.15.16:0/2569928987 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eac037040 con 0x7f0ebc108140 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.851+0000 7f0eb37fe640 1 -- 172.21.15.16:0/2569928987 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eac030cc0 con 0x7f0ebc108140 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.851+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/2569928987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ebc1060f0 con 0x7f0ebc108140 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.852+0000 7f0eb37fe640 1 -- 172.21.15.16:0/2569928987 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eac03a6b0 con 0x7f0ebc108140 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.852+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/2569928987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ebc106630 con 0x7f0ebc108140 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.852+0000 7f0eb37fe640 1 -- 172.21.15.16:0/2569928987 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f0eac03a870 con 0x7f0ebc108140 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.853+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/2569928987 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f0ea8039200 0x7f0ea803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.853+0000 7f0eb37fe640 1 -- 172.21.15.16:0/2569928987 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0eac06c7c0 con 0x7f0ebc108140 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.853+0000 7f0eb3fff640 1 --2- 172.21.15.16:0/2569928987 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f0ea8039200 0x7f0ea803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:36.760 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.853+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/2569928987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ebc1085a0 con 0x7f0ebc108140 2024-09-19T09:18:36.761 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.853+0000 7f0eb3fff640 1 --2- 172.21.15.16:0/2569928987 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f0ea8039200 0x7f0ea803b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40098c0 tx=0x7f0ea4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:36.761 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:35.858+0000 7f0eb37fe640 1 -- 172.21.15.16:0/2569928987 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0eac02fd60 con 0x7f0ebc108140 2024-09-19T09:18:36.761 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:36.101+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/2569928987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f0ebc106d00 con 0x7f0ebc108140 2024-09-19T09:18:36.761 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:36.170+0000 7f0eb37fe640 1 -- 172.21.15.16:0/2569928987 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f0eac044920 con 0x7f0ebc108140 2024-09-19T09:18:36.761 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:36.171+0000 7f0eb37fe640 1 -- 172.21.15.16:0/2569928987 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f0eac044400 con 0x7f0ebc108140 2024-09-19T09:18:36.761 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:36.173+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/2569928987 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f0ea8039200 msgr2=0x7f0ea803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:36.761 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:36.173+0000 7f0ec0fb8640 1 --2- 172.21.15.16:0/2569928987 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f0ea8039200 0x7f0ea803b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40098c0 tx=0x7f0ea4006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:36.761 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:36.173+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/2569928987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 msgr2=0x7f0ebc1079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:36.761 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:36.173+0000 7f0ec0fb8640 1 --2- 172.21.15.16:0/2569928987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 0x7f0ebc1079c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0097f0 tx=0x7f0eac0327b0 comp rx=0 tx=0).stop 2024-09-19T09:18:36.761 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:36.173+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/2569928987 shutdown_connections 2024-09-19T09:18:36.761 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:36.173+0000 7f0ec0fb8640 1 --2- 172.21.15.16:0/2569928987 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7f0ea8039200 0x7f0ea803b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:36.761 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:36.173+0000 7f0ec0fb8640 1 --2- 172.21.15.16:0/2569928987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ebc108140 0x7f0ebc1079c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:36.762 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:36.173+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/2569928987 >> 172.21.15.16:0/2569928987 conn(0x7f0ebc101e20 msgr2=0x7f0ebc1029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:36.762 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:36.173+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/2569928987 shutdown_connections 2024-09-19T09:18:36.762 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:36.173+0000 7f0ec0fb8640 1 -- 172.21.15.16:0/2569928987 wait complete. 2024-09-19T09:18:36.878 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:36.770+0000 7f450a255040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:18:37.171 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:36.923+0000 7f450a255040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T09:18:37.483 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:37 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2569928987' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T09:18:37.483 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:37 smithi016 ceph-mon[30979]: mgrmap e9: x(active, since 19s) 2024-09-19T09:18:37.483 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:37 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:37.258+0000 7f450a255040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T09:18:37.483 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:37 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:37.325+0000 7f450a255040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T09:18:37.753 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:37 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:37.522+0000 7f450a255040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T09:18:38.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:18:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T09:18:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T09:18:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T09:18:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T09:18:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:18:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.275+0000 7fb5d1e37640 1 Processor -- start 2024-09-19T09:18:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.276+0000 7fb5d1e37640 1 -- start start 2024-09-19T09:18:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.276+0000 7fb5d1e37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 0x7fb5cc10a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:38.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.276+0000 7fb5d1e37640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc07bd70 con 0x7fb5cc108200 2024-09-19T09:18:38.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.276+0000 7fb5cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 0x7fb5cc10a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:38.108 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.276+0000 7fb5cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 0x7fb5cc10a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58856/0 (socket says 172.21.15.16:58856) 2024-09-19T09:18:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.276+0000 7fb5cb7fe640 1 -- 172.21.15.16:0/803125258 learned_addr learned my addr 172.21.15.16:0/803125258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.276+0000 7fb5cb7fe640 1 -- 172.21.15.16:0/803125258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5cc10ab30 con 0x7fb5cc108200 2024-09-19T09:18:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.277+0000 7fb5cb7fe640 1 --2- 172.21.15.16:0/803125258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 0x7fb5cc10a5f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009820 tx=0x7fb5bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33197c4e4ffad1e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.277+0000 7fb5caffd640 1 -- 172.21.15.16:0/803125258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5bc037070 con 0x7fb5cc108200 2024-09-19T09:18:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.277+0000 7fb5caffd640 1 -- 172.21.15.16:0/803125258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5bc032920 con 0x7fb5cc108200 2024-09-19T09:18:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.277+0000 7fb5caffd640 1 -- 172.21.15.16:0/803125258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5bc030810 con 0x7fb5cc108200 2024-09-19T09:18:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.278+0000 7fb5d1e37640 1 -- 172.21.15.16:0/803125258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 msgr2=0x7fb5cc10a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.278+0000 7fb5d1e37640 1 --2- 172.21.15.16:0/803125258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 0x7fb5cc10a5f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009820 tx=0x7fb5bc02a270 comp rx=0 tx=0).stop 2024-09-19T09:18:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.278+0000 7fb5d1e37640 1 -- 172.21.15.16:0/803125258 shutdown_connections 2024-09-19T09:18:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.278+0000 7fb5d1e37640 1 --2- 172.21.15.16:0/803125258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 0x7fb5cc10a5f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.278+0000 7fb5d1e37640 1 -- 172.21.15.16:0/803125258 >> 172.21.15.16:0/803125258 conn(0x7fb5cc101e20 msgr2=0x7fb5cc104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.278+0000 7fb5d1e37640 1 -- 172.21.15.16:0/803125258 shutdown_connections 2024-09-19T09:18:38.109 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.278+0000 7fb5d1e37640 1 -- 172.21.15.16:0/803125258 wait complete. 2024-09-19T09:18:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.279+0000 7fb5d1e37640 1 Processor -- start 2024-09-19T09:18:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.279+0000 7fb5d1e37640 1 -- start start 2024-09-19T09:18:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.279+0000 7fb5d1e37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 0x7fb5cc19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.279+0000 7fb5d1e37640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc10bb50 con 0x7fb5cc108200 2024-09-19T09:18:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.279+0000 7fb5cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 0x7fb5cc19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.279+0000 7fb5cb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 0x7fb5cc19b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58860/0 (socket says 172.21.15.16:58860) 2024-09-19T09:18:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.279+0000 7fb5cb7fe640 1 -- 172.21.15.16:0/3311106971 learned_addr learned my addr 172.21.15.16:0/3311106971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.280+0000 7fb5cb7fe640 1 -- 172.21.15.16:0/3311106971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5cc19bc50 con 0x7fb5cc108200 2024-09-19T09:18:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.280+0000 7fb5cb7fe640 1 --2- 172.21.15.16:0/3311106971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 0x7fb5cc19b710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc030380 tx=0x7fb5bc032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.280+0000 7fb5c97fa640 1 -- 172.21.15.16:0/3311106971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5bc037040 con 0x7fb5cc108200 2024-09-19T09:18:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.280+0000 7fb5d1e37640 1 -- 172.21.15.16:0/3311106971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5cc19bee0 con 0x7fb5cc108200 2024-09-19T09:18:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.280+0000 7fb5c97fa640 1 -- 172.21.15.16:0/3311106971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5bc040070 con 0x7fb5cc108200 2024-09-19T09:18:38.110 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.280+0000 7fb5c97fa640 1 -- 172.21.15.16:0/3311106971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5bc03a7b0 con 0x7fb5cc108200 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.280+0000 7fb5d1e37640 1 -- 172.21.15.16:0/3311106971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5cc19ebd0 con 0x7fb5cc108200 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.281+0000 7fb5c97fa640 1 -- 172.21.15.16:0/3311106971 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fb5bc03aa30 con 0x7fb5cc108200 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.281+0000 7fb5c97fa640 1 --2- 172.21.15.16:0/3311106971 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fb5b8039200 0x7fb5b803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.281+0000 7fb5c97fa640 1 -- 172.21.15.16:0/3311106971 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb5bc06c930 con 0x7fb5cc108200 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.281+0000 7fb5c3fff640 1 -- 172.21.15.16:0/3311106971 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fb5b8039200 msgr2=0x7fb5b803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1680178094 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.281+0000 7fb5c3fff640 1 --2- 172.21.15.16:0/3311106971 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fb5b8039200 0x7fb5b803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.281+0000 7fb5d1e37640 1 -- 172.21.15.16:0/3311106971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5cc108660 con 0x7fb5cc108200 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.286+0000 7fb5c97fa640 1 -- 172.21.15.16:0/3311106971 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb5bc03bc00 con 0x7fb5cc108200 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.482+0000 7fb5c3fff640 1 -- 172.21.15.16:0/3311106971 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fb5b8039200 msgr2=0x7fb5b803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1680178094 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.482+0000 7fb5c3fff640 1 --2- 172.21.15.16:0/3311106971 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fb5b8039200 0x7fb5b803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.509+0000 7fb5d1e37640 1 -- 172.21.15.16:0/3311106971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb5cc19f0b0 con 0x7fb5cc108200 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.510+0000 7fb5c97fa640 1 -- 172.21.15.16:0/3311106971 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fb5bc041b40 con 0x7fb5cc108200 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.512+0000 7fb5d1e37640 1 -- 172.21.15.16:0/3311106971 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fb5b8039200 msgr2=0x7fb5b803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:18:38.111 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.512+0000 7fb5d1e37640 1 --2- 172.21.15.16:0/3311106971 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fb5b8039200 0x7fb5b803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.512+0000 7fb5d1e37640 1 -- 172.21.15.16:0/3311106971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 msgr2=0x7fb5cc19b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.512+0000 7fb5d1e37640 1 --2- 172.21.15.16:0/3311106971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 0x7fb5cc19b710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc030380 tx=0x7fb5bc032670 comp rx=0 tx=0).stop 2024-09-19T09:18:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.512+0000 7fb5d1e37640 1 -- 172.21.15.16:0/3311106971 shutdown_connections 2024-09-19T09:18:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.512+0000 7fb5d1e37640 1 --2- 172.21.15.16:0/3311106971 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7fb5b8039200 0x7fb5b803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.512+0000 7fb5d1e37640 1 --2- 172.21.15.16:0/3311106971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5cc108200 0x7fb5cc19b710 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.512+0000 7fb5d1e37640 1 -- 172.21.15.16:0/3311106971 >> 172.21.15.16:0/3311106971 conn(0x7fb5cc101e20 msgr2=0x7fb5cc102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.512+0000 7fb5d1e37640 1 -- 172.21.15.16:0/3311106971 shutdown_connections 2024-09-19T09:18:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:37.512+0000 7fb5d1e37640 1 -- 172.21.15.16:0/3311106971 wait complete. 2024-09-19T09:18:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-19T09:18:38.112 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 9... 2024-09-19T09:18:38.452 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:38 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3311106971' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T09:18:38.739 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:38 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:38.461+0000 7f450a255040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T09:18:38.739 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:38 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:38.533+0000 7f450a255040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T09:18:39.401 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:39 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:39.280+0000 7f450a255040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T09:18:39.753 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:39 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:39.401+0000 7f450a255040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T09:18:39.753 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:39 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:39.549+0000 7f450a255040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:18:39.753 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:39 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:39.613+0000 7f450a255040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T09:18:40.068 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:39 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:39.769+0000 7f450a255040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T09:18:40.069 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:39 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:39.839+0000 7f450a255040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T09:18:40.069 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:39 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:39.902+0000 7f450a255040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T09:18:40.069 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:40 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:40.068+0000 7f450a255040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T09:18:40.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:40 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:40.135+0000 7f450a255040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T09:18:40.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:40 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:40.205+0000 7f450a255040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T09:18:40.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:40 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:40.291+0000 7f450a255040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T09:18:40.354 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:40 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:40.353+0000 7f450a255040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T09:18:40.752 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:40 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:40.415+0000 7f450a255040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T09:18:41.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-mon[30979]: Active manager daemon x restarted 2024-09-19T09:18:41.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-mon[30979]: Activating manager daemon x 2024-09-19T09:18:41.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-mon[30979]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T09:18:41.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-mon[30979]: mgrmap e10: x(active, starting, since 0.00337928s) 2024-09-19T09:18:41.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:18:41.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T09:18:41.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T09:18:41.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T09:18:41.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T09:18:41.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-mon[30979]: Manager daemon x is now available 2024-09-19T09:18:41.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:18:41.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T09:18:41.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T09:18:41.497 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:41 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:41.241+0000 7f450a255040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T09:18:43.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-19T09:18:43.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T09:18:43.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T09:18:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-19T09:18:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.532+0000 7ff8d16db640 1 Processor -- start 2024-09-19T09:18:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.532+0000 7ff8d16db640 1 -- start start 2024-09-19T09:18:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.532+0000 7ff8d16db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 0x7ff8cc10a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.532+0000 7ff8d16db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc07be30 con 0x7ff8cc108180 2024-09-19T09:18:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.532+0000 7ff8caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 0x7ff8cc10a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.532+0000 7ff8caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 0x7ff8cc10a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58876/0 (socket says 172.21.15.16:58876) 2024-09-19T09:18:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.532+0000 7ff8caffd640 1 -- 172.21.15.16:0/2053343324 learned_addr learned my addr 172.21.15.16:0/2053343324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.533+0000 7ff8caffd640 1 -- 172.21.15.16:0/2053343324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc10aab0 con 0x7ff8cc108180 2024-09-19T09:18:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.533+0000 7ff8caffd640 1 --2- 172.21.15.16:0/2053343324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 0x7ff8cc10a570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009820 tx=0x7ff8bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc0f892e4fe129 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.533+0000 7ff8ca7fc640 1 -- 172.21.15.16:0/2053343324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8bc037070 con 0x7ff8cc108180 2024-09-19T09:18:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.534+0000 7ff8ca7fc640 1 -- 172.21.15.16:0/2053343324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8bc032920 con 0x7ff8cc108180 2024-09-19T09:18:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.534+0000 7ff8ca7fc640 1 -- 172.21.15.16:0/2053343324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8bc030810 con 0x7ff8cc108180 2024-09-19T09:18:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.534+0000 7ff8d16db640 1 -- 172.21.15.16:0/2053343324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 msgr2=0x7ff8cc10a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.534+0000 7ff8d16db640 1 --2- 172.21.15.16:0/2053343324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 0x7ff8cc10a570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009820 tx=0x7ff8bc02a270 comp rx=0 tx=0).stop 2024-09-19T09:18:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.534+0000 7ff8d16db640 1 -- 172.21.15.16:0/2053343324 shutdown_connections 2024-09-19T09:18:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.534+0000 7ff8d16db640 1 --2- 172.21.15.16:0/2053343324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 0x7ff8cc10a570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.534+0000 7ff8d16db640 1 -- 172.21.15.16:0/2053343324 >> 172.21.15.16:0/2053343324 conn(0x7ff8cc101e20 msgr2=0x7ff8cc104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.534+0000 7ff8d16db640 1 -- 172.21.15.16:0/2053343324 shutdown_connections 2024-09-19T09:18:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.534+0000 7ff8d16db640 1 -- 172.21.15.16:0/2053343324 wait complete. 2024-09-19T09:18:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.535+0000 7ff8d16db640 1 Processor -- start 2024-09-19T09:18:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.535+0000 7ff8d16db640 1 -- start start 2024-09-19T09:18:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.535+0000 7ff8d16db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 0x7ff8cc19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.535+0000 7ff8d16db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc10bb90 con 0x7ff8cc108180 2024-09-19T09:18:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.535+0000 7ff8caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 0x7ff8cc19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:43.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.535+0000 7ff8caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 0x7ff8cc19da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58884/0 (socket says 172.21.15.16:58884) 2024-09-19T09:18:43.224 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.535+0000 7ff8caffd640 1 -- 172.21.15.16:0/3428204158 learned_addr learned my addr 172.21.15.16:0/3428204158 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:43.224 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.535+0000 7ff8caffd640 1 -- 172.21.15.16:0/3428204158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc19df50 con 0x7ff8cc108180 2024-09-19T09:18:43.224 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.536+0000 7ff8caffd640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 0x7ff8cc19da10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc030420 tx=0x7ff8bc032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:43.224 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.536+0000 7ff8c8ff9640 1 -- 172.21.15.16:0/3428204158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8bc037040 con 0x7ff8cc108180 2024-09-19T09:18:43.224 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.536+0000 7ff8c8ff9640 1 -- 172.21.15.16:0/3428204158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8bc040070 con 0x7ff8cc108180 2024-09-19T09:18:43.224 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.536+0000 7ff8d16db640 1 -- 172.21.15.16:0/3428204158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc19e1e0 con 0x7ff8cc108180 2024-09-19T09:18:43.224 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.536+0000 7ff8c8ff9640 1 -- 172.21.15.16:0/3428204158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8bc03a6e0 con 0x7ff8cc108180 2024-09-19T09:18:43.224 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.536+0000 7ff8d16db640 1 -- 172.21.15.16:0/3428204158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc1a0ed0 con 0x7ff8cc108180 2024-09-19T09:18:43.224 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.537+0000 7ff8c8ff9640 1 -- 172.21.15.16:0/3428204158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7ff8bc03bb20 con 0x7ff8cc108180 2024-09-19T09:18:43.224 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.537+0000 7ff8c8ff9640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7ff8b8039260 0x7ff8b803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:43.224 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.537+0000 7ff8c8ff9640 1 -- 172.21.15.16:0/3428204158 --> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff8b803be30 con 0x7ff8b8039260 2024-09-19T09:18:43.224 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.537+0000 7ff8c8ff9640 1 -- 172.21.15.16:0/3428204158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff8bc06c8c0 con 0x7ff8cc108180 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.537+0000 7ff8c3fff640 1 -- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7ff8b8039260 msgr2=0x7ff8b803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1680178094 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.537+0000 7ff8c3fff640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7ff8b8039260 0x7ff8b803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.738+0000 7ff8c3fff640 1 -- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7ff8b8039260 msgr2=0x7ff8b803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1680178094 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:38.738+0000 7ff8c3fff640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7ff8b8039260 0x7ff8b803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:39.139+0000 7ff8c3fff640 1 -- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7ff8b8039260 msgr2=0x7ff8b803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1680178094 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:39.139+0000 7ff8c3fff640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7ff8b8039260 0x7ff8b803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:39.940+0000 7ff8c3fff640 1 -- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7ff8b8039260 msgr2=0x7ff8b803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1680178094 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:39.940+0000 7ff8c3fff640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7ff8b8039260 0x7ff8b803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:41.245+0000 7ff8c8ff9640 1 -- 172.21.15.16:0/3428204158 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7ff8bc06b370 con 0x7ff8cc108180 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:41.245+0000 7ff8c8ff9640 1 -- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7ff8b8039260 msgr2=0x7ff8b803b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:41.245+0000 7ff8c8ff9640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7ff8b8039260 0x7ff8b803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.246+0000 7ff8c8ff9640 1 -- 172.21.15.16:0/3428204158 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7ff8bc03b420 con 0x7ff8cc108180 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.247+0000 7ff8c8ff9640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8b803cd20 0x7ff8b803f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:43.225 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.247+0000 7ff8c8ff9640 1 -- 172.21.15.16:0/3428204158 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff8bc006810 con 0x7ff8b803cd20 2024-09-19T09:18:43.226 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.247+0000 7ff8c3fff640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8b803cd20 0x7ff8b803f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:43.226 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.248+0000 7ff8c3fff640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8b803cd20 0x7ff8b803f110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4003980 tx=0x7ff8b40092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:43.226 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.249+0000 7ff8c8ff9640 1 -- 172.21.15.16:0/3428204158 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff8bc006810 con 0x7ff8b803cd20 2024-09-19T09:18:43.226 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.255+0000 7ff8d16db640 1 -- 172.21.15.16:0/3428204158 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff8cc1085e0 con 0x7ff8b803cd20 2024-09-19T09:18:43.226 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.256+0000 7ff8c8ff9640 1 -- 172.21.15.16:0/3428204158 <== mgr.14150 v2:172.21.15.16:6800/4287557979 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ff8cc1085e0 con 0x7ff8b803cd20 2024-09-19T09:18:43.226 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.256+0000 7ff8d16db640 1 -- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8b803cd20 msgr2=0x7ff8b803f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:43.226 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.256+0000 7ff8d16db640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8b803cd20 0x7ff8b803f110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4003980 tx=0x7ff8b40092b0 comp rx=0 tx=0).stop 2024-09-19T09:18:43.226 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.256+0000 7ff8d16db640 1 -- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 msgr2=0x7ff8cc19da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:43.226 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.256+0000 7ff8d16db640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 0x7ff8cc19da10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc030420 tx=0x7ff8bc032670 comp rx=0 tx=0).stop 2024-09-19T09:18:43.226 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.256+0000 7ff8d16db640 1 -- 172.21.15.16:0/3428204158 shutdown_connections 2024-09-19T09:18:43.226 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.256+0000 7ff8d16db640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8b803cd20 0x7ff8b803f110 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:43.226 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.256+0000 7ff8d16db640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:6800/1680178094,v1:172.21.15.16:6801/1680178094] conn(0x7ff8b8039260 0x7ff8b803b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:43.226 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.256+0000 7ff8d16db640 1 --2- 172.21.15.16:0/3428204158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc108180 0x7ff8cc19da10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:43.227 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.256+0000 7ff8d16db640 1 -- 172.21.15.16:0/3428204158 >> 172.21.15.16:0/3428204158 conn(0x7ff8cc101e20 msgr2=0x7ff8cc102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:43.227 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.256+0000 7ff8d16db640 1 -- 172.21.15.16:0/3428204158 shutdown_connections 2024-09-19T09:18:43.227 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:42.256+0000 7ff8d16db640 1 -- 172.21.15.16:0/3428204158 wait complete. 2024-09-19T09:18:43.227 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 9 is available 2024-09-19T09:18:43.227 INFO:teuthology.orchestra.run.smithi016.stdout:Generating a dashboard self-signed certificate... 2024-09-19T09:18:43.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-mon[30979]: mgrmap e11: x(active, since 1.00592s) 2024-09-19T09:18:43.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:43.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-mon[30979]: [19/Sep/2024:09:18:43] ENGINE Bus STARTING 2024-09-19T09:18:43.503 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:43.417+0000 7f446fd1e640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:09:18:43] ENGINE Error in HTTPServer.serve 2024-09-19T09:18:43.503 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: Traceback (most recent call last): 2024-09-19T09:18:43.504 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T09:18:43.504 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: self._connections.run(self.expiration_interval) 2024-09-19T09:18:43.504 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T09:18:43.504 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: self._run(expiration_interval) 2024-09-19T09:18:43.504 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T09:18:43.504 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T09:18:43.504 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T09:18:43.504 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T09:18:43.504 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T09:18:43.504 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: s = self.context.wrap_socket( 2024-09-19T09:18:43.504 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T09:18:43.504 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: return self.sslsocket_class._create( 2024-09-19T09:18:43.505 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T09:18:43.505 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: self.do_handshake() 2024-09-19T09:18:43.505 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T09:18:43.505 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: self._sslobj.do_handshake() 2024-09-19T09:18:43.505 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T09:18:43.505 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:18:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:18:44.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T09:18:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.699+0000 7f7c0e772640 1 Processor -- start 2024-09-19T09:18:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.699+0000 7f7c0e772640 1 -- start start 2024-09-19T09:18:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.699+0000 7f7c0e772640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 0x7f7c0807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.699+0000 7f7c0e772640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0807c870 con 0x7f7c0807de40 2024-09-19T09:18:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.700+0000 7f7c07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 0x7f7c0807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.700+0000 7f7c07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 0x7f7c0807c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58946/0 (socket says 172.21.15.16:58946) 2024-09-19T09:18:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.700+0000 7f7c07fff640 1 -- 172.21.15.16:0/1062206650 learned_addr learned my addr 172.21.15.16:0/1062206650 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.700+0000 7f7c07fff640 1 -- 172.21.15.16:0/1062206650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0807c9f0 con 0x7f7c0807de40 2024-09-19T09:18:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.701+0000 7f7c07fff640 1 --2- 172.21.15.16:0/1062206650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 0x7f7c0807c2a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009820 tx=0x7f7bf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bec052ad8e0eae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.701+0000 7f7c06ffd640 1 -- 172.21.15.16:0/1062206650 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf8037070 con 0x7f7c0807de40 2024-09-19T09:18:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.701+0000 7f7c06ffd640 1 -- 172.21.15.16:0/1062206650 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf8032920 con 0x7f7c0807de40 2024-09-19T09:18:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.701+0000 7f7c06ffd640 1 -- 172.21.15.16:0/1062206650 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf8030810 con 0x7f7c0807de40 2024-09-19T09:18:44.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.701+0000 7f7c0e772640 1 -- 172.21.15.16:0/1062206650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 msgr2=0x7f7c0807c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:44.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.701+0000 7f7c0e772640 1 --2- 172.21.15.16:0/1062206650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 0x7f7c0807c2a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009820 tx=0x7f7bf802a270 comp rx=0 tx=0).stop 2024-09-19T09:18:44.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.701+0000 7f7c0e772640 1 -- 172.21.15.16:0/1062206650 shutdown_connections 2024-09-19T09:18:44.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.701+0000 7f7c0e772640 1 --2- 172.21.15.16:0/1062206650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 0x7f7c0807c2a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:44.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.701+0000 7f7c0e772640 1 -- 172.21.15.16:0/1062206650 >> 172.21.15.16:0/1062206650 conn(0x7f7c08102f30 msgr2=0x7f7c08105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:44.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.701+0000 7f7c0e772640 1 -- 172.21.15.16:0/1062206650 shutdown_connections 2024-09-19T09:18:44.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.701+0000 7f7c0e772640 1 -- 172.21.15.16:0/1062206650 wait complete. 2024-09-19T09:18:44.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.702+0000 7f7c0e772640 1 Processor -- start 2024-09-19T09:18:44.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.702+0000 7f7c0e772640 1 -- start start 2024-09-19T09:18:44.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.702+0000 7f7c0e772640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 0x7f7c0819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:44.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.702+0000 7f7c0e772640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0810ddc0 con 0x7f7c0807de40 2024-09-19T09:18:44.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.702+0000 7f7c07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 0x7f7c0819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:44.634 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.703+0000 7f7c07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 0x7f7c0819d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58948/0 (socket says 172.21.15.16:58948) 2024-09-19T09:18:44.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.703+0000 7f7c07fff640 1 -- 172.21.15.16:0/976898199 learned_addr learned my addr 172.21.15.16:0/976898199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:44.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.703+0000 7f7c07fff640 1 -- 172.21.15.16:0/976898199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0819de80 con 0x7f7c0807de40 2024-09-19T09:18:44.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.703+0000 7f7c07fff640 1 --2- 172.21.15.16:0/976898199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 0x7f7c0819d940 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8030560 tx=0x7f7bf8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:44.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.703+0000 7f7c057fa640 1 -- 172.21.15.16:0/976898199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf8037040 con 0x7f7c0807de40 2024-09-19T09:18:44.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.703+0000 7f7c057fa640 1 -- 172.21.15.16:0/976898199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf8040070 con 0x7f7c0807de40 2024-09-19T09:18:44.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.703+0000 7f7c0e772640 1 -- 172.21.15.16:0/976898199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0819e110 con 0x7f7c0807de40 2024-09-19T09:18:44.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.703+0000 7f7c057fa640 1 -- 172.21.15.16:0/976898199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf803a740 con 0x7f7c0807de40 2024-09-19T09:18:44.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.703+0000 7f7c0e772640 1 -- 172.21.15.16:0/976898199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c081a0e00 con 0x7f7c0807de40 2024-09-19T09:18:44.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.704+0000 7f7c057fa640 1 -- 172.21.15.16:0/976898199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f7bf803a8e0 con 0x7f7c0807de40 2024-09-19T09:18:44.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.704+0000 7f7c0e772640 1 -- 172.21.15.16:0/976898199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c0807c320 con 0x7f7c0807de40 2024-09-19T09:18:44.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.704+0000 7f7c057fa640 1 --2- 172.21.15.16:0/976898199 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7bf0039260 0x7f7bf003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:44.635 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.704+0000 7f7c057fa640 1 -- 172.21.15.16:0/976898199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7bf806cbb0 con 0x7f7c0807de40 2024-09-19T09:18:44.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.704+0000 7f7c077fe640 1 --2- 172.21.15.16:0/976898199 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7bf0039260 0x7f7bf003b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:44.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.705+0000 7f7c077fe640 1 --2- 172.21.15.16:0/976898199 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7bf0039260 0x7f7bf003b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0098c0 tx=0x7f7bfc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:44.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.709+0000 7f7c057fa640 1 -- 172.21.15.16:0/976898199 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7bf8044c70 con 0x7f7c0807de40 2024-09-19T09:18:44.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:43.933+0000 7f7c0e772640 1 -- 172.21.15.16:0/976898199 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f7c081041b0 con 0x7f7bf0039260 2024-09-19T09:18:44.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:44.028+0000 7f7c057fa640 1 -- 172.21.15.16:0/976898199 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f7c081041b0 con 0x7f7bf0039260 2024-09-19T09:18:44.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:44.030+0000 7f7c0e772640 1 -- 172.21.15.16:0/976898199 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7bf0039260 msgr2=0x7f7bf003b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:44.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:44.030+0000 7f7c0e772640 1 --2- 172.21.15.16:0/976898199 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7bf0039260 0x7f7bf003b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0098c0 tx=0x7f7bfc006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:44.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:44.030+0000 7f7c0e772640 1 -- 172.21.15.16:0/976898199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 msgr2=0x7f7c0819d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:44.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:44.030+0000 7f7c0e772640 1 --2- 172.21.15.16:0/976898199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 0x7f7c0819d940 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8030560 tx=0x7f7bf8032670 comp rx=0 tx=0).stop 2024-09-19T09:18:44.636 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:44.030+0000 7f7c0e772640 1 -- 172.21.15.16:0/976898199 shutdown_connections 2024-09-19T09:18:44.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:44.030+0000 7f7c0e772640 1 --2- 172.21.15.16:0/976898199 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7bf0039260 0x7f7bf003b720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:44.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:44.030+0000 7f7c0e772640 1 --2- 172.21.15.16:0/976898199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0807de40 0x7f7c0819d940 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:44.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:44.030+0000 7f7c0e772640 1 -- 172.21.15.16:0/976898199 >> 172.21.15.16:0/976898199 conn(0x7f7c08102f30 msgr2=0x7f7c08103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:44.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:44.030+0000 7f7c0e772640 1 -- 172.21.15.16:0/976898199 shutdown_connections 2024-09-19T09:18:44.637 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:44.030+0000 7f7c0e772640 1 -- 172.21.15.16:0/976898199 wait complete. 2024-09-19T09:18:44.637 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial admin user... 2024-09-19T09:18:44.953 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:44 smithi016 ceph-mon[30979]: [19/Sep/2024:09:18:43] ENGINE Serving on http://172.21.15.16:8765 2024-09-19T09:18:44.953 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:44 smithi016 ceph-mon[30979]: [19/Sep/2024:09:18:43] ENGINE Serving on https://172.21.15.16:7150 2024-09-19T09:18:44.953 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:44 smithi016 ceph-mon[30979]: [19/Sep/2024:09:18:43] ENGINE Bus STARTED 2024-09-19T09:18:44.953 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:44 smithi016 ceph-mon[30979]: [19/Sep/2024:09:18:43] ENGINE Error in HTTPServer.serve 2024-09-19T09:18:44.953 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-19T09:18:44.953 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T09:18:44.953 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-19T09:18:44.953 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T09:18:44.953 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-19T09:18:44.954 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T09:18:44.954 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T09:18:44.954 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T09:18:44.954 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T09:18:44.954 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T09:18:44.954 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-19T09:18:44.954 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T09:18:44.954 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-19T09:18:44.954 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T09:18:44.954 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-19T09:18:44.954 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T09:18:44.954 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-19T09:18:44.955 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T09:18:44.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:44 smithi016 ceph-mon[30979]: mgrmap e12: x(active, since 2s) 2024-09-19T09:18:44.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:44 smithi016 ceph-mon[30979]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:18:44.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:44 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:44.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:44 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:44.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:44 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:46.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ELOSQnYrZanIpQ.qX2ofvu.ZE12lD/ony8w.7FpSXk8AjZzDVyU5K", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726737525, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T09:18:46.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.037+0000 7f4db5205640 1 Processor -- start 2024-09-19T09:18:46.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.037+0000 7f4db5205640 1 -- start start 2024-09-19T09:18:46.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.037+0000 7f4db5205640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 0x7f4db010a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.037+0000 7f4db5205640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4db007bdf0 con 0x7f4db0108180 2024-09-19T09:18:46.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.038+0000 7f4daed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 0x7f4db010a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.038+0000 7f4daed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 0x7f4db010a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58950/0 (socket says 172.21.15.16:58950) 2024-09-19T09:18:46.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.038+0000 7f4daed9d640 1 -- 172.21.15.16:0/1258647660 learned_addr learned my addr 172.21.15.16:0/1258647660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.038+0000 7f4daed9d640 1 -- 172.21.15.16:0/1258647660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db010aab0 con 0x7f4db0108180 2024-09-19T09:18:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.039+0000 7f4daed9d640 1 --2- 172.21.15.16:0/1258647660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 0x7f4db010a570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009820 tx=0x7f4da002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8588d97820d99934 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.039+0000 7f4dadd9b640 1 -- 172.21.15.16:0/1258647660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da0037070 con 0x7f4db0108180 2024-09-19T09:18:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.039+0000 7f4dadd9b640 1 -- 172.21.15.16:0/1258647660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da0032920 con 0x7f4db0108180 2024-09-19T09:18:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.039+0000 7f4dadd9b640 1 -- 172.21.15.16:0/1258647660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da0030810 con 0x7f4db0108180 2024-09-19T09:18:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.040+0000 7f4db5205640 1 -- 172.21.15.16:0/1258647660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 msgr2=0x7f4db010a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.040+0000 7f4db5205640 1 --2- 172.21.15.16:0/1258647660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 0x7f4db010a570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009820 tx=0x7f4da002a270 comp rx=0 tx=0).stop 2024-09-19T09:18:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.040+0000 7f4db5205640 1 -- 172.21.15.16:0/1258647660 shutdown_connections 2024-09-19T09:18:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.040+0000 7f4db5205640 1 --2- 172.21.15.16:0/1258647660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 0x7f4db010a570 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.040+0000 7f4db5205640 1 -- 172.21.15.16:0/1258647660 >> 172.21.15.16:0/1258647660 conn(0x7f4db0101e20 msgr2=0x7f4db0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.040+0000 7f4db5205640 1 -- 172.21.15.16:0/1258647660 shutdown_connections 2024-09-19T09:18:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.040+0000 7f4db5205640 1 -- 172.21.15.16:0/1258647660 wait complete. 2024-09-19T09:18:46.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.041+0000 7f4db5205640 1 Processor -- start 2024-09-19T09:18:46.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.041+0000 7f4db5205640 1 -- start start 2024-09-19T09:18:46.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.042+0000 7f4db5205640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 0x7f4db019b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.042+0000 7f4db5205640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4db010bb30 con 0x7f4db0108180 2024-09-19T09:18:46.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.042+0000 7f4daed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 0x7f4db019b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.042+0000 7f4daed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 0x7f4db019b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58960/0 (socket says 172.21.15.16:58960) 2024-09-19T09:18:46.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.042+0000 7f4daed9d640 1 -- 172.21.15.16:0/3954495469 learned_addr learned my addr 172.21.15.16:0/3954495469 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:46.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.042+0000 7f4daed9d640 1 -- 172.21.15.16:0/3954495469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db019bc20 con 0x7f4db0108180 2024-09-19T09:18:46.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.042+0000 7f4daed9d640 1 --2- 172.21.15.16:0/3954495469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 0x7f4db019b6e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4da00097f0 tx=0x7f4da00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:46.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.042+0000 7f4d97fff640 1 -- 172.21.15.16:0/3954495469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da0037040 con 0x7f4db0108180 2024-09-19T09:18:46.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.042+0000 7f4d97fff640 1 -- 172.21.15.16:0/3954495469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da0030cc0 con 0x7f4db0108180 2024-09-19T09:18:46.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.043+0000 7f4db5205640 1 -- 172.21.15.16:0/3954495469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db019beb0 con 0x7f4db0108180 2024-09-19T09:18:46.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.043+0000 7f4d97fff640 1 -- 172.21.15.16:0/3954495469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da003a6b0 con 0x7f4db0108180 2024-09-19T09:18:46.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.043+0000 7f4db5205640 1 -- 172.21.15.16:0/3954495469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db019eba0 con 0x7f4db0108180 2024-09-19T09:18:46.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.044+0000 7f4d97fff640 1 -- 172.21.15.16:0/3954495469 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f4da003a8a0 con 0x7f4db0108180 2024-09-19T09:18:46.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.044+0000 7f4db5205640 1 -- 172.21.15.16:0/3954495469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db01085e0 con 0x7f4db0108180 2024-09-19T09:18:46.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.044+0000 7f4d97fff640 1 --2- 172.21.15.16:0/3954495469 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4d98039200 0x7f4d9803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:46.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.044+0000 7f4d97fff640 1 -- 172.21.15.16:0/3954495469 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4da006c900 con 0x7f4db0108180 2024-09-19T09:18:46.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.044+0000 7f4dae59c640 1 --2- 172.21.15.16:0/3954495469 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4d98039200 0x7f4d9803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:46.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.045+0000 7f4dae59c640 1 --2- 172.21.15.16:0/3954495469 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4d98039200 0x7f4d9803b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4da40098c0 tx=0x7f4da4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:46.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.049+0000 7f4d97fff640 1 -- 172.21.15.16:0/3954495469 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4da003ace0 con 0x7f4db0108180 2024-09-19T09:18:46.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.251+0000 7f4db5205640 1 -- 172.21.15.16:0/3954495469 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f4db0103180 con 0x7f4d98039200 2024-09-19T09:18:46.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.500+0000 7f4d97fff640 1 -- 172.21.15.16:0/3954495469 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f4db0103180 con 0x7f4d98039200 2024-09-19T09:18:46.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.502+0000 7f4db5205640 1 -- 172.21.15.16:0/3954495469 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4d98039200 msgr2=0x7f4d9803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:46.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.502+0000 7f4db5205640 1 --2- 172.21.15.16:0/3954495469 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4d98039200 0x7f4d9803b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4da40098c0 tx=0x7f4da4006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.502+0000 7f4db5205640 1 -- 172.21.15.16:0/3954495469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 msgr2=0x7f4db019b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:46.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.502+0000 7f4db5205640 1 --2- 172.21.15.16:0/3954495469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 0x7f4db019b6e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4da00097f0 tx=0x7f4da00327b0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.502+0000 7f4db5205640 1 -- 172.21.15.16:0/3954495469 shutdown_connections 2024-09-19T09:18:46.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.502+0000 7f4db5205640 1 --2- 172.21.15.16:0/3954495469 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4d98039200 0x7f4d9803b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.502+0000 7f4db5205640 1 --2- 172.21.15.16:0/3954495469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db0108180 0x7f4db019b6e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:46.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.502+0000 7f4db5205640 1 -- 172.21.15.16:0/3954495469 >> 172.21.15.16:0/3954495469 conn(0x7f4db0101e20 msgr2=0x7f4db0102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:46.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.503+0000 7f4db5205640 1 -- 172.21.15.16:0/3954495469 shutdown_connections 2024-09-19T09:18:46.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:45.503+0000 7f4db5205640 1 -- 172.21.15.16:0/3954495469 wait complete. 2024-09-19T09:18:46.285 INFO:teuthology.orchestra.run.smithi016.stdout:Fetching dashboard port number... 2024-09-19T09:18:46.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:46 smithi016 ceph-mon[30979]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:18:46.662 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:47.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:47 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1557237251' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T09:18:47.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T09:18:47.883 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.096+0000 7f5b964af640 1 Processor -- start 2024-09-19T09:18:47.883 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.096+0000 7f5b964af640 1 -- start start 2024-09-19T09:18:47.883 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.096+0000 7f5b964af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 0x7f5b90107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:47.883 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.096+0000 7f5b964af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b90108150 con 0x7f5b90107780 2024-09-19T09:18:47.883 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.097+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 0x7f5b90107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:47.884 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.097+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 0x7f5b90107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46454/0 (socket says 172.21.15.16:46454) 2024-09-19T09:18:47.884 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.097+0000 7f5b8ffff640 1 -- 172.21.15.16:0/4129824823 learned_addr learned my addr 172.21.15.16:0/4129824823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:47.884 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.097+0000 7f5b8ffff640 1 -- 172.21.15.16:0/4129824823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b90108980 con 0x7f5b90107780 2024-09-19T09:18:47.884 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.098+0000 7f5b8ffff640 1 --2- 172.21.15.16:0/4129824823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 0x7f5b90107b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009820 tx=0x7f5b8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186971e803c4ffc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:47.884 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.098+0000 7f5b8f7fe640 1 -- 172.21.15.16:0/4129824823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b80037070 con 0x7f5b90107780 2024-09-19T09:18:47.885 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.098+0000 7f5b8f7fe640 1 -- 172.21.15.16:0/4129824823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b80032920 con 0x7f5b90107780 2024-09-19T09:18:47.885 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.098+0000 7f5b8f7fe640 1 -- 172.21.15.16:0/4129824823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b80030810 con 0x7f5b90107780 2024-09-19T09:18:47.885 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.099+0000 7f5b964af640 1 -- 172.21.15.16:0/4129824823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 msgr2=0x7f5b90107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:47.885 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.099+0000 7f5b964af640 1 --2- 172.21.15.16:0/4129824823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 0x7f5b90107b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009820 tx=0x7f5b8002a270 comp rx=0 tx=0).stop 2024-09-19T09:18:47.885 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.099+0000 7f5b964af640 1 -- 172.21.15.16:0/4129824823 shutdown_connections 2024-09-19T09:18:47.885 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.099+0000 7f5b964af640 1 --2- 172.21.15.16:0/4129824823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 0x7f5b90107b80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:47.886 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.099+0000 7f5b964af640 1 -- 172.21.15.16:0/4129824823 >> 172.21.15.16:0/4129824823 conn(0x7f5b90102f30 msgr2=0x7f5b90105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:47.886 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.099+0000 7f5b964af640 1 -- 172.21.15.16:0/4129824823 shutdown_connections 2024-09-19T09:18:47.886 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.099+0000 7f5b964af640 1 -- 172.21.15.16:0/4129824823 wait complete. 2024-09-19T09:18:47.886 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.099+0000 7f5b964af640 1 Processor -- start 2024-09-19T09:18:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.099+0000 7f5b964af640 1 -- start start 2024-09-19T09:18:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.099+0000 7f5b964af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 0x7f5b9007daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.099+0000 7f5b964af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9010bd30 con 0x7f5b90107780 2024-09-19T09:18:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.100+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 0x7f5b9007daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.100+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 0x7f5b9007daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46468/0 (socket says 172.21.15.16:46468) 2024-09-19T09:18:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.100+0000 7f5b8ffff640 1 -- 172.21.15.16:0/1557237251 learned_addr learned my addr 172.21.15.16:0/1557237251 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.100+0000 7f5b8ffff640 1 -- 172.21.15.16:0/1557237251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9007e030 con 0x7f5b90107780 2024-09-19T09:18:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.100+0000 7f5b8ffff640 1 --2- 172.21.15.16:0/1557237251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 0x7f5b9007daf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5b80030560 tx=0x7f5b80032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.100+0000 7f5b8dffb640 1 -- 172.21.15.16:0/1557237251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b80037040 con 0x7f5b90107780 2024-09-19T09:18:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.100+0000 7f5b8dffb640 1 -- 172.21.15.16:0/1557237251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b80040070 con 0x7f5b90107780 2024-09-19T09:18:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.100+0000 7f5b964af640 1 -- 172.21.15.16:0/1557237251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9007c220 con 0x7f5b90107780 2024-09-19T09:18:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.100+0000 7f5b8dffb640 1 -- 172.21.15.16:0/1557237251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b8003a740 con 0x7f5b90107780 2024-09-19T09:18:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.100+0000 7f5b964af640 1 -- 172.21.15.16:0/1557237251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9007c760 con 0x7f5b90107780 2024-09-19T09:18:47.888 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.101+0000 7f5b8dffb640 1 -- 172.21.15.16:0/1557237251 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f5b8003b420 con 0x7f5b90107780 2024-09-19T09:18:47.888 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.101+0000 7f5b964af640 1 -- 172.21.15.16:0/1557237251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b90107c00 con 0x7f5b90107780 2024-09-19T09:18:47.888 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.102+0000 7f5b8dffb640 1 --2- 172.21.15.16:0/1557237251 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5b7c039200 0x7f5b7c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:47.888 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.102+0000 7f5b8dffb640 1 -- 172.21.15.16:0/1557237251 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5b8006c9c0 con 0x7f5b90107780 2024-09-19T09:18:47.889 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.102+0000 7f5b87fff640 1 --2- 172.21.15.16:0/1557237251 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5b7c039200 0x7f5b7c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:47.889 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.102+0000 7f5b87fff640 1 --2- 172.21.15.16:0/1557237251 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5b7c039200 0x7f5b7c03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5b780098c0 tx=0x7f5b78006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:47.889 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.106+0000 7f5b8dffb640 1 -- 172.21.15.16:0/1557237251 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5b8006ee20 con 0x7f5b90107780 2024-09-19T09:18:47.889 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.285+0000 7f5b964af640 1 -- 172.21.15.16:0/1557237251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f5b9007cf00 con 0x7f5b90107780 2024-09-19T09:18:47.889 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.285+0000 7f5b8dffb640 1 -- 172.21.15.16:0/1557237251 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f5b8004a200 con 0x7f5b90107780 2024-09-19T09:18:47.889 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.287+0000 7f5b964af640 1 -- 172.21.15.16:0/1557237251 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5b7c039200 msgr2=0x7f5b7c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:47.889 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.287+0000 7f5b964af640 1 --2- 172.21.15.16:0/1557237251 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5b7c039200 0x7f5b7c03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5b780098c0 tx=0x7f5b78006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:47.889 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.287+0000 7f5b964af640 1 -- 172.21.15.16:0/1557237251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 msgr2=0x7f5b9007daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:47.889 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.287+0000 7f5b964af640 1 --2- 172.21.15.16:0/1557237251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 0x7f5b9007daf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5b80030560 tx=0x7f5b80032670 comp rx=0 tx=0).stop 2024-09-19T09:18:47.889 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.287+0000 7f5b964af640 1 -- 172.21.15.16:0/1557237251 shutdown_connections 2024-09-19T09:18:47.890 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.287+0000 7f5b964af640 1 --2- 172.21.15.16:0/1557237251 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5b7c039200 0x7f5b7c03b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:47.890 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.288+0000 7f5b964af640 1 --2- 172.21.15.16:0/1557237251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b90107780 0x7f5b9007daf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:47.890 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.288+0000 7f5b964af640 1 -- 172.21.15.16:0/1557237251 >> 172.21.15.16:0/1557237251 conn(0x7f5b90102f30 msgr2=0x7f5b90103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:47.890 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.288+0000 7f5b964af640 1 -- 172.21.15.16:0/1557237251 shutdown_connections 2024-09-19T09:18:47.890 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:47.288+0000 7f5b964af640 1 -- 172.21.15.16:0/1557237251 wait complete. 2024-09-19T09:18:47.891 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T09:18:47.892 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-19T09:18:47.896 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T09:18:47.896 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-19T09:18:47.896 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-19T09:18:47.896 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T09:18:47.898 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph Dashboard is now available at: 2024-09-19T09:18:47.898 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:18:47.898 INFO:teuthology.orchestra.run.smithi016.stdout: URL: https://smithi016.front.sepia.ceph.com:8443/ 2024-09-19T09:18:47.899 INFO:teuthology.orchestra.run.smithi016.stdout: User: admin 2024-09-19T09:18:47.899 INFO:teuthology.orchestra.run.smithi016.stdout: Password: w9ftvkajx4 2024-09-19T09:18:47.899 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:18:47.900 INFO:teuthology.orchestra.run.smithi016.stdout:Saving cluster configuration to /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config directory 2024-09-19T09:18:48.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:48 smithi016 ceph-mon[30979]: mgrmap e13: x(active, since 6s) 2024-09-19T09:18:49.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.333+0000 7fb7c322e640 1 Processor -- start 2024-09-19T09:18:49.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.333+0000 7fb7c322e640 1 -- start start 2024-09-19T09:18:49.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.333+0000 7fb7c322e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 0x7fb7bc107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:49.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.333+0000 7fb7c322e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc108150 con 0x7fb7bc107780 2024-09-19T09:18:49.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.333+0000 7fb7c0fca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 0x7fb7bc107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:49.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.333+0000 7fb7c0fca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 0x7fb7bc107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46470/0 (socket says 172.21.15.16:46470) 2024-09-19T09:18:49.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.333+0000 7fb7c0fca640 1 -- 172.21.15.16:0/3733621401 learned_addr learned my addr 172.21.15.16:0/3733621401 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:49.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.333+0000 7fb7c0fca640 1 -- 172.21.15.16:0/3733621401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7bc108980 con 0x7fb7bc107780 2024-09-19T09:18:49.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.334+0000 7fb7c0fca640 1 --2- 172.21.15.16:0/3733621401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 0x7fb7bc107b80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009820 tx=0x7fb7b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e35330ab480254a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:49.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.334+0000 7fb7bb7fe640 1 -- 172.21.15.16:0/3733621401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7b0037070 con 0x7fb7bc107780 2024-09-19T09:18:49.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.334+0000 7fb7bb7fe640 1 -- 172.21.15.16:0/3733621401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b0032920 con 0x7fb7bc107780 2024-09-19T09:18:49.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.334+0000 7fb7bb7fe640 1 -- 172.21.15.16:0/3733621401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7b0030810 con 0x7fb7bc107780 2024-09-19T09:18:49.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.335+0000 7fb7c322e640 1 -- 172.21.15.16:0/3733621401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 msgr2=0x7fb7bc107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:49.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.335+0000 7fb7c322e640 1 --2- 172.21.15.16:0/3733621401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 0x7fb7bc107b80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009820 tx=0x7fb7b002a270 comp rx=0 tx=0).stop 2024-09-19T09:18:49.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.335+0000 7fb7c322e640 1 -- 172.21.15.16:0/3733621401 shutdown_connections 2024-09-19T09:18:49.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.335+0000 7fb7c322e640 1 --2- 172.21.15.16:0/3733621401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 0x7fb7bc107b80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:49.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.335+0000 7fb7c322e640 1 -- 172.21.15.16:0/3733621401 >> 172.21.15.16:0/3733621401 conn(0x7fb7bc102f30 msgr2=0x7fb7bc105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:49.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.335+0000 7fb7c322e640 1 -- 172.21.15.16:0/3733621401 shutdown_connections 2024-09-19T09:18:49.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.335+0000 7fb7c322e640 1 -- 172.21.15.16:0/3733621401 wait complete. 2024-09-19T09:18:49.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.335+0000 7fb7c322e640 1 Processor -- start 2024-09-19T09:18:49.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.336+0000 7fb7c322e640 1 -- start start 2024-09-19T09:18:49.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.336+0000 7fb7c322e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 0x7fb7bc19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:49.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.336+0000 7fb7c322e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc10bd30 con 0x7fb7bc107780 2024-09-19T09:18:49.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.336+0000 7fb7c0fca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 0x7fb7bc19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:49.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.336+0000 7fb7c0fca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 0x7fb7bc19d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46480/0 (socket says 172.21.15.16:46480) 2024-09-19T09:18:49.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.336+0000 7fb7c0fca640 1 -- 172.21.15.16:0/2936982599 learned_addr learned my addr 172.21.15.16:0/2936982599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:49.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.336+0000 7fb7c0fca640 1 -- 172.21.15.16:0/2936982599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7bc19deb0 con 0x7fb7bc107780 2024-09-19T09:18:49.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.336+0000 7fb7c0fca640 1 --2- 172.21.15.16:0/2936982599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 0x7fb7bc19d970 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0030560 tx=0x7fb7b0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:49.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.336+0000 7fb7b9ffb640 1 -- 172.21.15.16:0/2936982599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7b0037040 con 0x7fb7bc107780 2024-09-19T09:18:49.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.337+0000 7fb7b9ffb640 1 -- 172.21.15.16:0/2936982599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b0040070 con 0x7fb7bc107780 2024-09-19T09:18:49.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.337+0000 7fb7c322e640 1 -- 172.21.15.16:0/2936982599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7bc19e140 con 0x7fb7bc107780 2024-09-19T09:18:49.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.337+0000 7fb7b9ffb640 1 -- 172.21.15.16:0/2936982599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7b003a740 con 0x7fb7bc107780 2024-09-19T09:18:49.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.337+0000 7fb7c322e640 1 -- 172.21.15.16:0/2936982599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7bc1a0e30 con 0x7fb7bc107780 2024-09-19T09:18:49.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.338+0000 7fb7b9ffb640 1 -- 172.21.15.16:0/2936982599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fb7b003a9b0 con 0x7fb7bc107780 2024-09-19T09:18:49.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.338+0000 7fb7c322e640 1 -- 172.21.15.16:0/2936982599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7bc107c00 con 0x7fb7bc107780 2024-09-19T09:18:49.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.338+0000 7fb7b9ffb640 1 --2- 172.21.15.16:0/2936982599 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7ac0392a0 0x7fb7ac03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:49.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.338+0000 7fb7b9ffb640 1 -- 172.21.15.16:0/2936982599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb7b006cab0 con 0x7fb7bc107780 2024-09-19T09:18:49.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.338+0000 7fb7bbfff640 1 --2- 172.21.15.16:0/2936982599 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7ac0392a0 0x7fb7ac03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:49.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.338+0000 7fb7bbfff640 1 --2- 172.21.15.16:0/2936982599 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7ac0392a0 0x7fb7ac03b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80098c0 tx=0x7fb7a8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:49.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.343+0000 7fb7b9ffb640 1 -- 172.21.15.16:0/2936982599 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb7b0039950 con 0x7fb7bc107780 2024-09-19T09:18:49.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.611+0000 7fb7c322e640 1 -- 172.21.15.16:0/2936982599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fb7bc1a1420 con 0x7fb7bc107780 2024-09-19T09:18:49.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.614+0000 7fb7b9ffb640 1 -- 172.21.15.16:0/2936982599 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fb7b003adf0 con 0x7fb7bc107780 2024-09-19T09:18:49.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T09:18:49.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.616+0000 7fb7c322e640 1 -- 172.21.15.16:0/2936982599 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7ac0392a0 msgr2=0x7fb7ac03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:49.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.616+0000 7fb7c322e640 1 --2- 172.21.15.16:0/2936982599 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7ac0392a0 0x7fb7ac03b760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80098c0 tx=0x7fb7a8006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:49.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.616+0000 7fb7c322e640 1 -- 172.21.15.16:0/2936982599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 msgr2=0x7fb7bc19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:49.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.616+0000 7fb7c322e640 1 --2- 172.21.15.16:0/2936982599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 0x7fb7bc19d970 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0030560 tx=0x7fb7b0032670 comp rx=0 tx=0).stop 2024-09-19T09:18:49.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.616+0000 7fb7c322e640 1 -- 172.21.15.16:0/2936982599 shutdown_connections 2024-09-19T09:18:49.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.616+0000 7fb7c322e640 1 --2- 172.21.15.16:0/2936982599 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7ac0392a0 0x7fb7ac03b760 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:49.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.616+0000 7fb7c322e640 1 --2- 172.21.15.16:0/2936982599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc107780 0x7fb7bc19d970 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:49.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.617+0000 7fb7c322e640 1 -- 172.21.15.16:0/2936982599 >> 172.21.15.16:0/2936982599 conn(0x7fb7bc102f30 msgr2=0x7fb7bc103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:49.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.617+0000 7fb7c322e640 1 -- 172.21.15.16:0/2936982599 shutdown_connections 2024-09-19T09:18:49.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-19T09:18:48.617+0000 7fb7c322e640 1 -- 172.21.15.16:0/2936982599 wait complete. 2024-09-19T09:18:49.305 INFO:teuthology.orchestra.run.smithi016.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T09:18:49.305 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:18:49.305 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:18:49.305 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:18:49.305 INFO:teuthology.orchestra.run.smithi016.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T09:18:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:18:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T09:18:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:18:49.306 INFO:teuthology.orchestra.run.smithi016.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T09:18:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:18:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: ceph telemetry on 2024-09-19T09:18:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:18:49.306 INFO:teuthology.orchestra.run.smithi016.stdout:For more information see: 2024-09-19T09:18:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:18:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T09:18:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:18:49.306 INFO:teuthology.orchestra.run.smithi016.stdout:Bootstrap complete. 2024-09-19T09:18:49.344 INFO:tasks.cephadm:Fetching config... 2024-09-19T09:18:49.345 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:18:49.345 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T09:18:49.364 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T09:18:49.364 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:18:49.364 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T09:18:49.420 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T09:18:49.421 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:18:49.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T09:18:49.486 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T09:18:49.486 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:18:49.487 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T09:18:49.543 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T09:18:49.543 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuHle617f3RXOILWbfJhgf0L8Xhfr2UqtJVK408oKQMn7I4t6Z2F/OhcaxF3OtvfJ/s3yJKqTMI3vUdBYng95srxt0SE8bQzTsQlvRpRZa69D4k6qAC9oqE5X8Q66lZezws/7/5fOf6etvwizQPlzwQA14g+9JTo2rdCpndH7DEkwIYSrRUCUKMUAH/mRGS/zCJfWPmB5BUqXhwTUpiH8SvdwUKi3hswz6DaxB1/KHjTOqSTnAek2zfDVkAYcD/1ABjVSib+eUPA0Oabxp61VCF7gkSSskwRF1Bt9UI8K78jMXkSVLKJq5nQ/ydf6wCpUaRt+13PdcyFVSW/DU1ZSlHjWaCqiDa4mvEwRcTmbEBHAlDyUdFvCKySSoEh/FQwna1/dfXo+T1wWWLaVklDzuowB9s3w8NTfKkGzIZPaSnNsSUxoA5UxV9eVCy3EwzdYIg6nzj7E8slEvVNglslSiGCnBKvffFlp/ZRYHm3jU2tgbh9tuKnVwU0xjj6lsBeU= ceph-ee1ab676-7667-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T09:18:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuHle617f3RXOILWbfJhgf0L8Xhfr2UqtJVK408oKQMn7I4t6Z2F/OhcaxF3OtvfJ/s3yJKqTMI3vUdBYng95srxt0SE8bQzTsQlvRpRZa69D4k6qAC9oqE5X8Q66lZezws/7/5fOf6etvwizQPlzwQA14g+9JTo2rdCpndH7DEkwIYSrRUCUKMUAH/mRGS/zCJfWPmB5BUqXhwTUpiH8SvdwUKi3hswz6DaxB1/KHjTOqSTnAek2zfDVkAYcD/1ABjVSib+eUPA0Oabxp61VCF7gkSSskwRF1Bt9UI8K78jMXkSVLKJq5nQ/ydf6wCpUaRt+13PdcyFVSW/DU1ZSlHjWaCqiDa4mvEwRcTmbEBHAlDyUdFvCKySSoEh/FQwna1/dfXo+T1wWWLaVklDzuowB9s3w8NTfKkGzIZPaSnNsSUxoA5UxV9eVCy3EwzdYIg6nzj7E8slEvVNglslSiGCnBKvffFlp/ZRYHm3jU2tgbh9tuKnVwU0xjj6lsBeU= ceph-ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:18:49.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuHle617f3RXOILWbfJhgf0L8Xhfr2UqtJVK408oKQMn7I4t6Z2F/OhcaxF3OtvfJ/s3yJKqTMI3vUdBYng95srxt0SE8bQzTsQlvRpRZa69D4k6qAC9oqE5X8Q66lZezws/7/5fOf6etvwizQPlzwQA14g+9JTo2rdCpndH7DEkwIYSrRUCUKMUAH/mRGS/zCJfWPmB5BUqXhwTUpiH8SvdwUKi3hswz6DaxB1/KHjTOqSTnAek2zfDVkAYcD/1ABjVSib+eUPA0Oabxp61VCF7gkSSskwRF1Bt9UI8K78jMXkSVLKJq5nQ/ydf6wCpUaRt+13PdcyFVSW/DU1ZSlHjWaCqiDa4mvEwRcTmbEBHAlDyUdFvCKySSoEh/FQwna1/dfXo+T1wWWLaVklDzuowB9s3w8NTfKkGzIZPaSnNsSUxoA5UxV9eVCy3EwzdYIg6nzj7E8slEvVNglslSiGCnBKvffFlp/ZRYHm3jU2tgbh9tuKnVwU0xjj6lsBeU= ceph-ee1ab676-7667-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T09:18:49.672 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuHle617f3RXOILWbfJhgf0L8Xhfr2UqtJVK408oKQMn7I4t6Z2F/OhcaxF3OtvfJ/s3yJKqTMI3vUdBYng95srxt0SE8bQzTsQlvRpRZa69D4k6qAC9oqE5X8Q66lZezws/7/5fOf6etvwizQPlzwQA14g+9JTo2rdCpndH7DEkwIYSrRUCUKMUAH/mRGS/zCJfWPmB5BUqXhwTUpiH8SvdwUKi3hswz6DaxB1/KHjTOqSTnAek2zfDVkAYcD/1ABjVSib+eUPA0Oabxp61VCF7gkSSskwRF1Bt9UI8K78jMXkSVLKJq5nQ/ydf6wCpUaRt+13PdcyFVSW/DU1ZSlHjWaCqiDa4mvEwRcTmbEBHAlDyUdFvCKySSoEh/FQwna1/dfXo+T1wWWLaVklDzuowB9s3w8NTfKkGzIZPaSnNsSUxoA5UxV9eVCy3EwzdYIg6nzj7E8slEvVNglslSiGCnBKvffFlp/ZRYHm3jU2tgbh9tuKnVwU0xjj6lsBeU= ceph-ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:18:49.685 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuHle617f3RXOILWbfJhgf0L8Xhfr2UqtJVK408oKQMn7I4t6Z2F/OhcaxF3OtvfJ/s3yJKqTMI3vUdBYng95srxt0SE8bQzTsQlvRpRZa69D4k6qAC9oqE5X8Q66lZezws/7/5fOf6etvwizQPlzwQA14g+9JTo2rdCpndH7DEkwIYSrRUCUKMUAH/mRGS/zCJfWPmB5BUqXhwTUpiH8SvdwUKi3hswz6DaxB1/KHjTOqSTnAek2zfDVkAYcD/1ABjVSib+eUPA0Oabxp61VCF7gkSSskwRF1Bt9UI8K78jMXkSVLKJq5nQ/ydf6wCpUaRt+13PdcyFVSW/DU1ZSlHjWaCqiDa4mvEwRcTmbEBHAlDyUdFvCKySSoEh/FQwna1/dfXo+T1wWWLaVklDzuowB9s3w8NTfKkGzIZPaSnNsSUxoA5UxV9eVCy3EwzdYIg6nzj7E8slEvVNglslSiGCnBKvffFlp/ZRYHm3jU2tgbh9tuKnVwU0xjj6lsBeU= ceph-ee1ab676-7667-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T09:18:49.727 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuHle617f3RXOILWbfJhgf0L8Xhfr2UqtJVK408oKQMn7I4t6Z2F/OhcaxF3OtvfJ/s3yJKqTMI3vUdBYng95srxt0SE8bQzTsQlvRpRZa69D4k6qAC9oqE5X8Q66lZezws/7/5fOf6etvwizQPlzwQA14g+9JTo2rdCpndH7DEkwIYSrRUCUKMUAH/mRGS/zCJfWPmB5BUqXhwTUpiH8SvdwUKi3hswz6DaxB1/KHjTOqSTnAek2zfDVkAYcD/1ABjVSib+eUPA0Oabxp61VCF7gkSSskwRF1Bt9UI8K78jMXkSVLKJq5nQ/ydf6wCpUaRt+13PdcyFVSW/DU1ZSlHjWaCqiDa4mvEwRcTmbEBHAlDyUdFvCKySSoEh/FQwna1/dfXo+T1wWWLaVklDzuowB9s3w8NTfKkGzIZPaSnNsSUxoA5UxV9eVCy3EwzdYIg6nzj7E8slEvVNglslSiGCnBKvffFlp/ZRYHm3jU2tgbh9tuKnVwU0xjj6lsBeU= ceph-ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:18:49.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T09:18:49.821 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:49 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2936982599' entity='client.admin' 2024-09-19T09:18:49.990 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.419+0000 7f447d763640 1 -- 172.21.15.16:0/3728626390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4478100540 msgr2=0x7f4478100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.419+0000 7f447d763640 1 --2- 172.21.15.16:0/3728626390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4478100540 0x7f4478100940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4468009880 tx=0x7f446802a5d0 comp rx=0 tx=0).stop 2024-09-19T09:18:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.419+0000 7f447d763640 1 -- 172.21.15.16:0/3728626390 shutdown_connections 2024-09-19T09:18:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.419+0000 7f447d763640 1 --2- 172.21.15.16:0/3728626390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4478100540 0x7f4478100940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.419+0000 7f447d763640 1 -- 172.21.15.16:0/3728626390 >> 172.21.15.16:0/3728626390 conn(0x7f44780fbcd0 msgr2=0x7f44780fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.419+0000 7f447d763640 1 -- 172.21.15.16:0/3728626390 shutdown_connections 2024-09-19T09:18:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.419+0000 7f447d763640 1 -- 172.21.15.16:0/3728626390 wait complete. 2024-09-19T09:18:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.420+0000 7f447d763640 1 Processor -- start 2024-09-19T09:18:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.420+0000 7f447d763640 1 -- start start 2024-09-19T09:18:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.420+0000 7f447d763640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4478100540 0x7f4478195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.420+0000 7f447d763640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f44781065f0 con 0x7f4478100540 2024-09-19T09:18:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.420+0000 7f4476ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4478100540 0x7f4478195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.420+0000 7f4476ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4478100540 0x7f4478195c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46490/0 (socket says 172.21.15.16:46490) 2024-09-19T09:18:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.420+0000 7f4476ffd640 1 -- 172.21.15.16:0/3588806953 learned_addr learned my addr 172.21.15.16:0/3588806953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.421+0000 7f4476ffd640 1 -- 172.21.15.16:0/3588806953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44781961a0 con 0x7f4478100540 2024-09-19T09:18:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.421+0000 7f4476ffd640 1 --2- 172.21.15.16:0/3588806953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4478100540 0x7f4478195c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4468004f60 tx=0x7f44680048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.421+0000 7f4474ff9640 1 -- 172.21.15.16:0/3588806953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4468046070 con 0x7f4478100540 2024-09-19T09:18:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.421+0000 7f4474ff9640 1 -- 172.21.15.16:0/3588806953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4468004cc0 con 0x7f4478100540 2024-09-19T09:18:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.421+0000 7f447d763640 1 -- 172.21.15.16:0/3588806953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4478196430 con 0x7f4478100540 2024-09-19T09:18:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.421+0000 7f4474ff9640 1 -- 172.21.15.16:0/3588806953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4468035070 con 0x7f4478100540 2024-09-19T09:18:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.421+0000 7f447d763640 1 -- 172.21.15.16:0/3588806953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44780696b0 con 0x7f4478100540 2024-09-19T09:18:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.422+0000 7f4474ff9640 1 -- 172.21.15.16:0/3588806953 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f4468049070 con 0x7f4478100540 2024-09-19T09:18:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.422+0000 7f4474ff9640 1 --2- 172.21.15.16:0/3588806953 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f44640392a0 0x7f446403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.422+0000 7f447d763640 1 -- 172.21.15.16:0/3588806953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4478106d70 con 0x7f4478100540 2024-09-19T09:18:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.422+0000 7f4474ff9640 1 -- 172.21.15.16:0/3588806953 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f446803e070 con 0x7f4478100540 2024-09-19T09:18:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.422+0000 7f446e7fc640 1 --2- 172.21.15.16:0/3588806953 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f44640392a0 0x7f446403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.423+0000 7f446e7fc640 1 --2- 172.21.15.16:0/3588806953 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f44640392a0 0x7f446403b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f44600098c0 tx=0x7f4460006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.428+0000 7f4474ff9640 1 -- 172.21.15.16:0/3588806953 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f44680303a0 con 0x7f4478100540 2024-09-19T09:18:50.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.609+0000 7f447d763640 1 -- 172.21.15.16:0/3588806953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f447806a2a0 con 0x7f4478100540 2024-09-19T09:18:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.612+0000 7f4474ff9640 1 -- 172.21.15.16:0/3588806953 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f4468035210 con 0x7f4478100540 2024-09-19T09:18:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.614+0000 7f447d763640 1 -- 172.21.15.16:0/3588806953 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f44640392a0 msgr2=0x7f446403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.614+0000 7f447d763640 1 --2- 172.21.15.16:0/3588806953 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f44640392a0 0x7f446403b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f44600098c0 tx=0x7f4460006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.614+0000 7f447d763640 1 -- 172.21.15.16:0/3588806953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4478100540 msgr2=0x7f4478195c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.614+0000 7f447d763640 1 --2- 172.21.15.16:0/3588806953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4478100540 0x7f4478195c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4468004f60 tx=0x7f44680048f0 comp rx=0 tx=0).stop 2024-09-19T09:18:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.614+0000 7f447d763640 1 -- 172.21.15.16:0/3588806953 shutdown_connections 2024-09-19T09:18:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.614+0000 7f447d763640 1 --2- 172.21.15.16:0/3588806953 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f44640392a0 0x7f446403b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.614+0000 7f447d763640 1 --2- 172.21.15.16:0/3588806953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4478100540 0x7f4478195c60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.614+0000 7f447d763640 1 -- 172.21.15.16:0/3588806953 >> 172.21.15.16:0/3588806953 conn(0x7f44780fbcd0 msgr2=0x7f44780fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.614+0000 7f447d763640 1 -- 172.21.15.16:0/3588806953 shutdown_connections 2024-09-19T09:18:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:50.614+0000 7f447d763640 1 -- 172.21.15.16:0/3588806953 wait complete. 2024-09-19T09:18:51.210 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T09:18:51.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T09:18:51.699 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:51.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:51 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3588806953' entity='client.admin' 2024-09-19T09:18:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.188+0000 7f936b8b4640 1 -- 172.21.15.16:0/3985111522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9364100540 msgr2=0x7f9364100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.188+0000 7f936b8b4640 1 --2- 172.21.15.16:0/3985111522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9364100540 0x7f9364100940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9360009880 tx=0x7f936002a5d0 comp rx=0 tx=0).stop 2024-09-19T09:18:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.188+0000 7f936b8b4640 1 -- 172.21.15.16:0/3985111522 shutdown_connections 2024-09-19T09:18:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.188+0000 7f936b8b4640 1 --2- 172.21.15.16:0/3985111522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9364100540 0x7f9364100940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:52.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.188+0000 7f936b8b4640 1 -- 172.21.15.16:0/3985111522 >> 172.21.15.16:0/3985111522 conn(0x7f93640fbcd0 msgr2=0x7f93640fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.188+0000 7f936b8b4640 1 -- 172.21.15.16:0/3985111522 shutdown_connections 2024-09-19T09:18:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.188+0000 7f936b8b4640 1 -- 172.21.15.16:0/3985111522 wait complete. 2024-09-19T09:18:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.189+0000 7f936b8b4640 1 Processor -- start 2024-09-19T09:18:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.189+0000 7f936b8b4640 1 -- start start 2024-09-19T09:18:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.189+0000 7f936b8b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9364100540 0x7f936406a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.189+0000 7f936b8b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93641065f0 con 0x7f9364100540 2024-09-19T09:18:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.189+0000 7f9369650640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9364100540 0x7f936406a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.189+0000 7f9369650640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9364100540 0x7f936406a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46502/0 (socket says 172.21.15.16:46502) 2024-09-19T09:18:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.189+0000 7f9369650640 1 -- 172.21.15.16:0/3262256728 learned_addr learned my addr 172.21.15.16:0/3262256728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.189+0000 7f9369650640 1 -- 172.21.15.16:0/3262256728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9364199af0 con 0x7f9364100540 2024-09-19T09:18:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.190+0000 7f9369650640 1 --2- 172.21.15.16:0/3262256728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9364100540 0x7f936406a130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9360004f60 tx=0x7f93600048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.190+0000 7f935a7fc640 1 -- 172.21.15.16:0/3262256728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f936003f070 con 0x7f9364100540 2024-09-19T09:18:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.190+0000 7f935a7fc640 1 -- 172.21.15.16:0/3262256728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9360004d40 con 0x7f9364100540 2024-09-19T09:18:52.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.190+0000 7f936b8b4640 1 -- 172.21.15.16:0/3262256728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9364199cc0 con 0x7f9364100540 2024-09-19T09:18:52.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.190+0000 7f935a7fc640 1 -- 172.21.15.16:0/3262256728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9360035030 con 0x7f9364100540 2024-09-19T09:18:52.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.190+0000 7f936b8b4640 1 -- 172.21.15.16:0/3262256728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936419a060 con 0x7f9364100540 2024-09-19T09:18:52.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.191+0000 7f935a7fc640 1 -- 172.21.15.16:0/3262256728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f9360042070 con 0x7f9364100540 2024-09-19T09:18:52.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.191+0000 7f936b8b4640 1 -- 172.21.15.16:0/3262256728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9364106d70 con 0x7f9364100540 2024-09-19T09:18:52.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.191+0000 7f935a7fc640 1 --2- 172.21.15.16:0/3262256728 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f93500392a0 0x7f935003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:52.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.191+0000 7f935a7fc640 1 -- 172.21.15.16:0/3262256728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f936006d0e0 con 0x7f9364100540 2024-09-19T09:18:52.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.191+0000 7f9368e4f640 1 --2- 172.21.15.16:0/3262256728 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f93500392a0 0x7f935003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:52.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.192+0000 7f9368e4f640 1 --2- 172.21.15.16:0/3262256728 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f93500392a0 0x7f935003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f93540098c0 tx=0x7f9354006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:52.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.196+0000 7f935a7fc640 1 -- 172.21.15.16:0/3262256728 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9360003de0 con 0x7f9364100540 2024-09-19T09:18:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.376+0000 7f936b8b4640 1 -- 172.21.15.16:0/3262256728 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f936402da50 con 0x7f93500392a0 2024-09-19T09:18:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.379+0000 7f935a7fc640 1 -- 172.21.15.16:0/3262256728 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f936402da50 con 0x7f93500392a0 2024-09-19T09:18:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.381+0000 7f936b8b4640 1 -- 172.21.15.16:0/3262256728 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f93500392a0 msgr2=0x7f935003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.381+0000 7f936b8b4640 1 --2- 172.21.15.16:0/3262256728 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f93500392a0 0x7f935003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f93540098c0 tx=0x7f9354006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.381+0000 7f936b8b4640 1 -- 172.21.15.16:0/3262256728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9364100540 msgr2=0x7f936406a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.381+0000 7f936b8b4640 1 --2- 172.21.15.16:0/3262256728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9364100540 0x7f936406a130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9360004f60 tx=0x7f93600048f0 comp rx=0 tx=0).stop 2024-09-19T09:18:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.381+0000 7f936b8b4640 1 -- 172.21.15.16:0/3262256728 shutdown_connections 2024-09-19T09:18:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.381+0000 7f936b8b4640 1 --2- 172.21.15.16:0/3262256728 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f93500392a0 0x7f935003b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.381+0000 7f936b8b4640 1 --2- 172.21.15.16:0/3262256728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9364100540 0x7f936406a130 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.381+0000 7f936b8b4640 1 -- 172.21.15.16:0/3262256728 >> 172.21.15.16:0/3262256728 conn(0x7f93640fbcd0 msgr2=0x7f93640fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.382+0000 7f936b8b4640 1 -- 172.21.15.16:0/3262256728 shutdown_connections 2024-09-19T09:18:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:52.382+0000 7f936b8b4640 1 -- 172.21.15.16:0/3262256728 wait complete. 2024-09-19T09:18:52.923 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:52 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:52.923 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:52 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:52.923 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:52 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:18:52.924 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:52 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:52.924 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:52 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:18:52.924 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:52 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:52.924 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:52 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:52.924 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:52 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:18:52.924 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:52 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:18:52.924 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:52 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:18:53.272 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi097 2024-09-19T09:18:53.273 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-19T09:18:53.273 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.conf 2024-09-19T09:18:53.293 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-19T09:18:53.293 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:18:53.352 INFO:tasks.cephadm:Adding host smithi097 to orchestrator... 2024-09-19T09:18:53.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch host add smithi097 2024-09-19T09:18:53.599 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:53.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:53 smithi016 ceph-mon[30979]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:18:53.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:53 smithi016 ceph-mon[30979]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-19T09:18:53.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:53 smithi016 ceph-mon[30979]: Updating smithi016:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:18:53.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:53 smithi016 ceph-mon[30979]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:18:53.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:53 smithi016 ceph-mon[30979]: Updating smithi016:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T09:18:53.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:53.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:53.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.061+0000 7f26fffa1640 1 -- 172.21.15.16:0/1218418261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f81027b0 msgr2=0x7f26f8102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.061+0000 7f26fffa1640 1 --2- 172.21.15.16:0/1218418261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f81027b0 0x7f26f8102b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009880 tx=0x7f26f402a670 comp rx=0 tx=0).stop 2024-09-19T09:18:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.061+0000 7f26fffa1640 1 -- 172.21.15.16:0/1218418261 shutdown_connections 2024-09-19T09:18:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.061+0000 7f26fffa1640 1 --2- 172.21.15.16:0/1218418261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f81027b0 0x7f26f8102b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.061+0000 7f26fffa1640 1 -- 172.21.15.16:0/1218418261 >> 172.21.15.16:0/1218418261 conn(0x7f26f80fc1b0 msgr2=0x7f26f80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.061+0000 7f26fffa1640 1 -- 172.21.15.16:0/1218418261 shutdown_connections 2024-09-19T09:18:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.061+0000 7f26fffa1640 1 -- 172.21.15.16:0/1218418261 wait complete. 2024-09-19T09:18:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.062+0000 7f26fffa1640 1 Processor -- start 2024-09-19T09:18:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.062+0000 7f26fffa1640 1 -- start start 2024-09-19T09:18:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.062+0000 7f26fffa1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f81027b0 0x7f26f81971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.062+0000 7f26fffa1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26f810bcb0 con 0x7f26f81027b0 2024-09-19T09:18:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.062+0000 7f26fdd3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f81027b0 0x7f26f81971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.062+0000 7f26fdd3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f81027b0 0x7f26f81971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46514/0 (socket says 172.21.15.16:46514) 2024-09-19T09:18:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.062+0000 7f26fdd3d640 1 -- 172.21.15.16:0/2287219879 learned_addr learned my addr 172.21.15.16:0/2287219879 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.062+0000 7f26fdd3d640 1 -- 172.21.15.16:0/2287219879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f8197700 con 0x7f26f81027b0 2024-09-19T09:18:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.062+0000 7f26fdd3d640 1 --2- 172.21.15.16:0/2287219879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f81027b0 0x7f26f81971c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009230 tx=0x7f26f4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.063+0000 7f26eeffd640 1 -- 172.21.15.16:0/2287219879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26f403f070 con 0x7f26f81027b0 2024-09-19T09:18:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.063+0000 7f26fffa1640 1 -- 172.21.15.16:0/2287219879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f8197990 con 0x7f26f81027b0 2024-09-19T09:18:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.063+0000 7f26eeffd640 1 -- 172.21.15.16:0/2287219879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f403bc80 con 0x7f26f81027b0 2024-09-19T09:18:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.063+0000 7f26eeffd640 1 -- 172.21.15.16:0/2287219879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26f4035030 con 0x7f26f81027b0 2024-09-19T09:18:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.063+0000 7f26fffa1640 1 -- 172.21.15.16:0/2287219879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f8197df0 con 0x7f26f81027b0 2024-09-19T09:18:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.063+0000 7f26eeffd640 1 -- 172.21.15.16:0/2287219879 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f26f4042070 con 0x7f26f81027b0 2024-09-19T09:18:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.064+0000 7f26eeffd640 1 --2- 172.21.15.16:0/2287219879 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f26e40392a0 0x7f26e403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.064+0000 7f26eeffd640 1 -- 172.21.15.16:0/2287219879 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f26f406cfd0 con 0x7f26f81027b0 2024-09-19T09:18:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.064+0000 7f26fffa1640 1 -- 172.21.15.16:0/2287219879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f806a750 con 0x7f26f81027b0 2024-09-19T09:18:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.064+0000 7f26fd53c640 1 --2- 172.21.15.16:0/2287219879 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f26e40392a0 0x7f26e403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.064+0000 7f26fd53c640 1 --2- 172.21.15.16:0/2287219879 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f26e40392a0 0x7f26e403b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f26e80098c0 tx=0x7f26e8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:54.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.069+0000 7f26eeffd640 1 -- 172.21.15.16:0/2287219879 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f26f404c350 con 0x7f26f81027b0 2024-09-19T09:18:54.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:54.248+0000 7f26fffa1640 1 -- 172.21.15.16:0/2287219879 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}) -- 0x7f26f8198520 con 0x7f26e40392a0 2024-09-19T09:18:55.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:55 smithi016 ceph-mon[30979]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:18:56.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:56 smithi016 ceph-mon[30979]: Deploying cephadm binary to smithi097 2024-09-19T09:18:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:58.087+0000 7f26eeffd640 1 -- 172.21.15.16:0/2287219879 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f26f8198520 con 0x7f26e40392a0 2024-09-19T09:18:58.089 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi097' with addr '172.21.15.97' 2024-09-19T09:18:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:58.090+0000 7f26fffa1640 1 -- 172.21.15.16:0/2287219879 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f26e40392a0 msgr2=0x7f26e403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:58.090+0000 7f26fffa1640 1 --2- 172.21.15.16:0/2287219879 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f26e40392a0 0x7f26e403b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f26e80098c0 tx=0x7f26e8006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:58.090+0000 7f26fffa1640 1 -- 172.21.15.16:0/2287219879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f81027b0 msgr2=0x7f26f81971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:58.090+0000 7f26fffa1640 1 --2- 172.21.15.16:0/2287219879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f81027b0 0x7f26f81971c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009230 tx=0x7f26f4003d40 comp rx=0 tx=0).stop 2024-09-19T09:18:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:58.090+0000 7f26fffa1640 1 -- 172.21.15.16:0/2287219879 shutdown_connections 2024-09-19T09:18:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:58.090+0000 7f26fffa1640 1 --2- 172.21.15.16:0/2287219879 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f26e40392a0 0x7f26e403b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:58.090+0000 7f26fffa1640 1 --2- 172.21.15.16:0/2287219879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f81027b0 0x7f26f81971c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:58.090+0000 7f26fffa1640 1 -- 172.21.15.16:0/2287219879 >> 172.21.15.16:0/2287219879 conn(0x7f26f80fc1b0 msgr2=0x7f26f8105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:58.090+0000 7f26fffa1640 1 -- 172.21.15.16:0/2287219879 shutdown_connections 2024-09-19T09:18:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:58.090+0000 7f26fffa1640 1 -- 172.21.15.16:0/2287219879 wait complete. 2024-09-19T09:18:58.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T09:18:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:18:59.343 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:59 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:59.343 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:59 smithi016 ceph-mon[30979]: Added host smithi097 2024-09-19T09:18:59.344 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:59 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:18:59.344 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:59 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:59.344 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:18:59 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:18:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.426+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/193940688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e4102750 msgr2=0x7fd4e4102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.426+0000 7fd4eb3d9640 1 --2- 172.21.15.16:0/193940688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e4102750 0x7fd4e4102b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009880 tx=0x7fd4e002a670 comp rx=0 tx=0).stop 2024-09-19T09:18:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.426+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/193940688 shutdown_connections 2024-09-19T09:18:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.426+0000 7fd4eb3d9640 1 --2- 172.21.15.16:0/193940688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e4102750 0x7fd4e4102b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.426+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/193940688 >> 172.21.15.16:0/193940688 conn(0x7fd4e40fc1b0 msgr2=0x7fd4e40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.426+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/193940688 shutdown_connections 2024-09-19T09:18:59.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.426+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/193940688 wait complete. 2024-09-19T09:18:59.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.427+0000 7fd4eb3d9640 1 Processor -- start 2024-09-19T09:18:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.427+0000 7fd4eb3d9640 1 -- start start 2024-09-19T09:18:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.427+0000 7fd4eb3d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e4102750 0x7fd4e41971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.427+0000 7fd4eb3d9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e410bcc0 con 0x7fd4e4102750 2024-09-19T09:18:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.427+0000 7fd4e9175640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e4102750 0x7fd4e41971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.427+0000 7fd4e9175640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e4102750 0x7fd4e41971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34996/0 (socket says 172.21.15.16:34996) 2024-09-19T09:18:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.427+0000 7fd4e9175640 1 -- 172.21.15.16:0/3289420471 learned_addr learned my addr 172.21.15.16:0/3289420471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:18:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.428+0000 7fd4e9175640 1 -- 172.21.15.16:0/3289420471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e4197710 con 0x7fd4e4102750 2024-09-19T09:18:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.428+0000 7fd4e9175640 1 --2- 172.21.15.16:0/3289420471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e4102750 0x7fd4e41971d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009230 tx=0x7fd4e0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.428+0000 7fd4da7fc640 1 -- 172.21.15.16:0/3289420471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4e003f070 con 0x7fd4e4102750 2024-09-19T09:18:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.428+0000 7fd4da7fc640 1 -- 172.21.15.16:0/3289420471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e003bc80 con 0x7fd4e4102750 2024-09-19T09:18:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.428+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/3289420471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e41979a0 con 0x7fd4e4102750 2024-09-19T09:18:59.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.428+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/3289420471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e4197e00 con 0x7fd4e4102750 2024-09-19T09:18:59.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.428+0000 7fd4da7fc640 1 -- 172.21.15.16:0/3289420471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4e0035030 con 0x7fd4e4102750 2024-09-19T09:18:59.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.429+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/3289420471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c8005290 con 0x7fd4e4102750 2024-09-19T09:18:59.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.431+0000 7fd4da7fc640 1 -- 172.21.15.16:0/3289420471 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd4e0042070 con 0x7fd4e4102750 2024-09-19T09:18:59.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.431+0000 7fd4da7fc640 1 --2- 172.21.15.16:0/3289420471 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4d00392a0 0x7fd4d003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:18:59.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.431+0000 7fd4da7fc640 1 -- 172.21.15.16:0/3289420471 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd4e006d420 con 0x7fd4e4102750 2024-09-19T09:18:59.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.431+0000 7fd4e8974640 1 --2- 172.21.15.16:0/3289420471 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4d00392a0 0x7fd4d003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:18:59.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.432+0000 7fd4e8974640 1 --2- 172.21.15.16:0/3289420471 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4d00392a0 0x7fd4d003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40098c0 tx=0x7fd4d4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:18:59.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.435+0000 7fd4da7fc640 1 -- 172.21.15.16:0/3289420471 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd4e00314e0 con 0x7fd4e4102750 2024-09-19T09:18:59.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.619+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/3289420471 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4c8000bc0 con 0x7fd4d00392a0 2024-09-19T09:18:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.620+0000 7fd4da7fc640 1 -- 172.21.15.16:0/3289420471 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fd4c8000bc0 con 0x7fd4d00392a0 2024-09-19T09:18:59.621 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:18:59.621 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}] 2024-09-19T09:18:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.622+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/3289420471 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4d00392a0 msgr2=0x7fd4d003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.622+0000 7fd4eb3d9640 1 --2- 172.21.15.16:0/3289420471 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4d00392a0 0x7fd4d003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40098c0 tx=0x7fd4d4006db0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.622+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/3289420471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e4102750 msgr2=0x7fd4e41971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:18:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.622+0000 7fd4eb3d9640 1 --2- 172.21.15.16:0/3289420471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e4102750 0x7fd4e41971d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009230 tx=0x7fd4e0003d40 comp rx=0 tx=0).stop 2024-09-19T09:18:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.622+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/3289420471 shutdown_connections 2024-09-19T09:18:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.622+0000 7fd4eb3d9640 1 --2- 172.21.15.16:0/3289420471 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4d00392a0 0x7fd4d003b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.623+0000 7fd4eb3d9640 1 --2- 172.21.15.16:0/3289420471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e4102750 0x7fd4e41971d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:18:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.623+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/3289420471 >> 172.21.15.16:0/3289420471 conn(0x7fd4e40fc1b0 msgr2=0x7fd4e4105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:18:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.623+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/3289420471 shutdown_connections 2024-09-19T09:18:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:18:59.623+0000 7fd4eb3d9640 1 -- 172.21.15.16:0/3289420471 wait complete. 2024-09-19T09:19:00.227 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2024-09-19T09:19:00.227 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-19T09:19:00.227 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2024-09-19T09:19:00.246 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-19T09:19:00.247 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:19:00.305 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2024-09-19T09:19:00.305 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch host add smithi142 2024-09-19T09:19:00.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:00 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:00.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:00 smithi016 ceph-mon[30979]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:19:00.554 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:00.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.982+0000 7f813fac4640 1 -- 172.21.15.16:0/3357257011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8138102750 msgr2=0x7f8138102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.982+0000 7f813fac4640 1 --2- 172.21.15.16:0/3357257011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8138102750 0x7f8138102b30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8134009880 tx=0x7f813402a670 comp rx=0 tx=0).stop 2024-09-19T09:19:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.982+0000 7f813fac4640 1 -- 172.21.15.16:0/3357257011 shutdown_connections 2024-09-19T09:19:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.982+0000 7f813fac4640 1 --2- 172.21.15.16:0/3357257011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8138102750 0x7f8138102b30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.982+0000 7f813fac4640 1 -- 172.21.15.16:0/3357257011 >> 172.21.15.16:0/3357257011 conn(0x7f81380fc1b0 msgr2=0x7f81380fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.982+0000 7f813fac4640 1 -- 172.21.15.16:0/3357257011 shutdown_connections 2024-09-19T09:19:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.982+0000 7f813fac4640 1 -- 172.21.15.16:0/3357257011 wait complete. 2024-09-19T09:19:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.983+0000 7f813fac4640 1 Processor -- start 2024-09-19T09:19:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.983+0000 7f813fac4640 1 -- start start 2024-09-19T09:19:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.983+0000 7f813fac4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8138102750 0x7f81381971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.983+0000 7f813fac4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f813810bcc0 con 0x7f8138102750 2024-09-19T09:19:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.983+0000 7f813d860640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8138102750 0x7f81381971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.983+0000 7f813d860640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8138102750 0x7f81381971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35018/0 (socket says 172.21.15.16:35018) 2024-09-19T09:19:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.983+0000 7f813d860640 1 -- 172.21.15.16:0/2478431023 learned_addr learned my addr 172.21.15.16:0/2478431023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:19:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.984+0000 7f813d860640 1 -- 172.21.15.16:0/2478431023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8138197700 con 0x7f8138102750 2024-09-19T09:19:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.984+0000 7f813d860640 1 --2- 172.21.15.16:0/2478431023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8138102750 0x7f81381971c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8134009230 tx=0x7f8134003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.984+0000 7f812effd640 1 -- 172.21.15.16:0/2478431023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f813403f070 con 0x7f8138102750 2024-09-19T09:19:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.984+0000 7f813fac4640 1 -- 172.21.15.16:0/2478431023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8138197990 con 0x7f8138102750 2024-09-19T09:19:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.984+0000 7f812effd640 1 -- 172.21.15.16:0/2478431023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813403bd40 con 0x7f8138102750 2024-09-19T09:19:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.984+0000 7f812effd640 1 -- 172.21.15.16:0/2478431023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8134035030 con 0x7f8138102750 2024-09-19T09:19:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.984+0000 7f813fac4640 1 -- 172.21.15.16:0/2478431023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8138197df0 con 0x7f8138102750 2024-09-19T09:19:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.985+0000 7f812effd640 1 -- 172.21.15.16:0/2478431023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f8134042070 con 0x7f8138102750 2024-09-19T09:19:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.985+0000 7f812effd640 1 --2- 172.21.15.16:0/2478431023 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f81240392a0 0x7f812403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.985+0000 7f813fac4640 1 -- 172.21.15.16:0/2478431023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f813806a7b0 con 0x7f8138102750 2024-09-19T09:19:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.985+0000 7f812effd640 1 -- 172.21.15.16:0/2478431023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f813406d020 con 0x7f8138102750 2024-09-19T09:19:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.985+0000 7f813d05f640 1 --2- 172.21.15.16:0/2478431023 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f81240392a0 0x7f812403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.986+0000 7f813d05f640 1 --2- 172.21.15.16:0/2478431023 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f81240392a0 0x7f812403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f81280098c0 tx=0x7f8128006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:00.990+0000 7f812effd640 1 -- 172.21.15.16:0/2478431023 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8134042b40 con 0x7f8138102750 2024-09-19T09:19:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:01.173+0000 7f813fac4640 1 -- 172.21.15.16:0/2478431023 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}) -- 0x7f8138198580 con 0x7f81240392a0 2024-09-19T09:19:01.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:01 smithi016 ceph-mon[30979]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:19:03.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:03 smithi016 ceph-mon[30979]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:03.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:03 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:03.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:03 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:03.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:03 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:03.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:03 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:03.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:03 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:19:03.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:03 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:03.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:03 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:03.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:03 smithi016 ceph-mon[30979]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-19T09:19:03.651 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:03 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:03.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:03 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:03.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:03 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:04 smithi016 ceph-mon[30979]: Updating smithi097:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:19:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:04 smithi016 ceph-mon[30979]: Deploying cephadm binary to smithi142 2024-09-19T09:19:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:04 smithi016 ceph-mon[30979]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:19:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:04 smithi016 ceph-mon[30979]: Updating smithi097:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T09:19:05.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:05.160+0000 7f812effd640 1 -- 172.21.15.16:0/2478431023 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f8138198580 con 0x7f81240392a0 2024-09-19T09:19:05.161 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi142' with addr '172.21.15.142' 2024-09-19T09:19:05.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:05.162+0000 7f813fac4640 1 -- 172.21.15.16:0/2478431023 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f81240392a0 msgr2=0x7f812403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:05.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:05.162+0000 7f813fac4640 1 --2- 172.21.15.16:0/2478431023 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f81240392a0 0x7f812403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f81280098c0 tx=0x7f8128006db0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:05.162+0000 7f813fac4640 1 -- 172.21.15.16:0/2478431023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8138102750 msgr2=0x7f81381971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:05.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:05.162+0000 7f813fac4640 1 --2- 172.21.15.16:0/2478431023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8138102750 0x7f81381971c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8134009230 tx=0x7f8134003d80 comp rx=0 tx=0).stop 2024-09-19T09:19:05.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:05.162+0000 7f813fac4640 1 -- 172.21.15.16:0/2478431023 shutdown_connections 2024-09-19T09:19:05.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:05.162+0000 7f813fac4640 1 --2- 172.21.15.16:0/2478431023 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f81240392a0 0x7f812403b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:05.162+0000 7f813fac4640 1 --2- 172.21.15.16:0/2478431023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8138102750 0x7f81381971c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:05.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:05.162+0000 7f813fac4640 1 -- 172.21.15.16:0/2478431023 >> 172.21.15.16:0/2478431023 conn(0x7f81380fc1b0 msgr2=0x7f8138103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:05.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:05.162+0000 7f813fac4640 1 -- 172.21.15.16:0/2478431023 shutdown_connections 2024-09-19T09:19:05.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:05.162+0000 7f813fac4640 1 -- 172.21.15.16:0/2478431023 wait complete. 2024-09-19T09:19:05.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:05 smithi016 ceph-mon[30979]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:05.813 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T09:19:06.067 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:06.481 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:06.481 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:06 smithi016 ceph-mon[30979]: Added host smithi142 2024-09-19T09:19:06.481 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:06.481 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:06.481 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.480+0000 7fc402606640 1 -- 172.21.15.16:0/779726789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3fc102730 msgr2=0x7fc3fc102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.480+0000 7fc402606640 1 --2- 172.21.15.16:0/779726789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3fc102730 0x7fc3fc102b10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009810 tx=0x7fc3ec02a540 comp rx=0 tx=0).stop 2024-09-19T09:19:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.480+0000 7fc402606640 1 -- 172.21.15.16:0/779726789 shutdown_connections 2024-09-19T09:19:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.480+0000 7fc402606640 1 --2- 172.21.15.16:0/779726789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3fc102730 0x7fc3fc102b10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.480+0000 7fc402606640 1 -- 172.21.15.16:0/779726789 >> 172.21.15.16:0/779726789 conn(0x7fc3fc077310 msgr2=0x7fc3fc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.480+0000 7fc402606640 1 -- 172.21.15.16:0/779726789 shutdown_connections 2024-09-19T09:19:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.480+0000 7fc402606640 1 -- 172.21.15.16:0/779726789 wait complete. 2024-09-19T09:19:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.481+0000 7fc402606640 1 Processor -- start 2024-09-19T09:19:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.481+0000 7fc402606640 1 -- start start 2024-09-19T09:19:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.481+0000 7fc402606640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3fc102730 0x7fc3fc1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.481+0000 7fc402606640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc10bc80 con 0x7fc3fc102730 2024-09-19T09:19:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.481+0000 7fc3fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3fc102730 0x7fc3fc1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.481+0000 7fc3fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3fc102730 0x7fc3fc1971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41868/0 (socket says 172.21.15.16:41868) 2024-09-19T09:19:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.482+0000 7fc3fbfff640 1 -- 172.21.15.16:0/2676316321 learned_addr learned my addr 172.21.15.16:0/2676316321 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:19:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.482+0000 7fc3fbfff640 1 -- 172.21.15.16:0/2676316321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3fc1976f0 con 0x7fc3fc102730 2024-09-19T09:19:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.482+0000 7fc3fbfff640 1 --2- 172.21.15.16:0/2676316321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3fc102730 0x7fc3fc1971b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0045e0 tx=0x7fc3ec004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.482+0000 7fc3f97fa640 1 -- 172.21.15.16:0/2676316321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3ec030e70 con 0x7fc3fc102730 2024-09-19T09:19:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.482+0000 7fc402606640 1 -- 172.21.15.16:0/2676316321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3fc197980 con 0x7fc3fc102730 2024-09-19T09:19:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.482+0000 7fc3f97fa640 1 -- 172.21.15.16:0/2676316321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3ec0318e0 con 0x7fc3fc102730 2024-09-19T09:19:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.482+0000 7fc3f97fa640 1 -- 172.21.15.16:0/2676316321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3ec0049c0 con 0x7fc3fc102730 2024-09-19T09:19:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.482+0000 7fc402606640 1 -- 172.21.15.16:0/2676316321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3fc197de0 con 0x7fc3fc102730 2024-09-19T09:19:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.483+0000 7fc3f97fa640 1 -- 172.21.15.16:0/2676316321 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fc3ec039040 con 0x7fc3fc102730 2024-09-19T09:19:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.483+0000 7fc402606640 1 -- 172.21.15.16:0/2676316321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3fc069f90 con 0x7fc3fc102730 2024-09-19T09:19:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.483+0000 7fc3f97fa640 1 --2- 172.21.15.16:0/2676316321 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc3e40392a0 0x7fc3e403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.483+0000 7fc3f97fa640 1 -- 172.21.15.16:0/2676316321 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc3ec06d640 con 0x7fc3fc102730 2024-09-19T09:19:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.484+0000 7fc3fb7fe640 1 --2- 172.21.15.16:0/2676316321 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc3e40392a0 0x7fc3e403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.484+0000 7fc3fb7fe640 1 --2- 172.21.15.16:0/2676316321 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc3e40392a0 0x7fc3e403b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00098c0 tx=0x7fc3f0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.488+0000 7fc3f97fa640 1 -- 172.21.15.16:0/2676316321 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc3ec040030 con 0x7fc3fc102730 2024-09-19T09:19:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.670+0000 7fc402606640 1 -- 172.21.15.16:0/2676316321 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3fc198570 con 0x7fc3e40392a0 2024-09-19T09:19:06.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.672+0000 7fc3f97fa640 1 -- 172.21.15.16:0/2676316321 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fc3fc198570 con 0x7fc3e40392a0 2024-09-19T09:19:06.673 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:19:06.673 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2024-09-19T09:19:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.674+0000 7fc402606640 1 -- 172.21.15.16:0/2676316321 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc3e40392a0 msgr2=0x7fc3e403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.674+0000 7fc402606640 1 --2- 172.21.15.16:0/2676316321 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc3e40392a0 0x7fc3e403b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00098c0 tx=0x7fc3f0006db0 comp rx=0 tx=0).stop 2024-09-19T09:19:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.674+0000 7fc402606640 1 -- 172.21.15.16:0/2676316321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3fc102730 msgr2=0x7fc3fc1971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.674+0000 7fc402606640 1 --2- 172.21.15.16:0/2676316321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3fc102730 0x7fc3fc1971b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0045e0 tx=0x7fc3ec004610 comp rx=0 tx=0).stop 2024-09-19T09:19:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.674+0000 7fc402606640 1 -- 172.21.15.16:0/2676316321 shutdown_connections 2024-09-19T09:19:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.674+0000 7fc402606640 1 --2- 172.21.15.16:0/2676316321 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc3e40392a0 0x7fc3e403b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.674+0000 7fc402606640 1 --2- 172.21.15.16:0/2676316321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3fc102730 0x7fc3fc1971b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.674+0000 7fc402606640 1 -- 172.21.15.16:0/2676316321 >> 172.21.15.16:0/2676316321 conn(0x7fc3fc077310 msgr2=0x7fc3fc1034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.674+0000 7fc402606640 1 -- 172.21.15.16:0/2676316321 shutdown_connections 2024-09-19T09:19:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:06.674+0000 7fc402606640 1 -- 172.21.15.16:0/2676316321 wait complete. 2024-09-19T09:19:07.293 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:07 smithi016 ceph-mon[30979]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:07.294 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:07.324 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T09:19:07.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T09:19:07.575 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:08.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.020+0000 7f00ced13640 1 -- 172.21.15.16:0/1566882181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c8107660 msgr2=0x7f00c8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:08.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.020+0000 7f00ced13640 1 --2- 172.21.15.16:0/1566882181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c8107660 0x7f00c8107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009880 tx=0x7f00b802a5d0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.020+0000 7f00ced13640 1 -- 172.21.15.16:0/1566882181 shutdown_connections 2024-09-19T09:19:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.020+0000 7f00ced13640 1 --2- 172.21.15.16:0/1566882181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c8107660 0x7f00c8107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.020+0000 7f00ced13640 1 -- 172.21.15.16:0/1566882181 >> 172.21.15.16:0/1566882181 conn(0x7f00c80fd290 msgr2=0x7f00c80ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.020+0000 7f00ced13640 1 -- 172.21.15.16:0/1566882181 shutdown_connections 2024-09-19T09:19:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.020+0000 7f00ced13640 1 -- 172.21.15.16:0/1566882181 wait complete. 2024-09-19T09:19:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.021+0000 7f00ced13640 1 Processor -- start 2024-09-19T09:19:08.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.021+0000 7f00ced13640 1 -- start start 2024-09-19T09:19:08.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.021+0000 7f00ced13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c8107660 0x7f00c81970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.021+0000 7f00ced13640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00c810bc90 con 0x7f00c8107660 2024-09-19T09:19:08.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.021+0000 7f00ccaaf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c8107660 0x7f00c81970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.021+0000 7f00ccaaf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c8107660 0x7f00c81970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41896/0 (socket says 172.21.15.16:41896) 2024-09-19T09:19:08.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.021+0000 7f00ccaaf640 1 -- 172.21.15.16:0/2701294602 learned_addr learned my addr 172.21.15.16:0/2701294602 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:19:08.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.022+0000 7f00ccaaf640 1 -- 172.21.15.16:0/2701294602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c8197600 con 0x7f00c8107660 2024-09-19T09:19:08.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.022+0000 7f00ccaaf640 1 --2- 172.21.15.16:0/2701294602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c8107660 0x7f00c81970c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009230 tx=0x7f00b8003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:08.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.022+0000 7f00c5ffb640 1 -- 172.21.15.16:0/2701294602 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00b803f070 con 0x7f00c8107660 2024-09-19T09:19:08.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.022+0000 7f00c5ffb640 1 -- 172.21.15.16:0/2701294602 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b803bd40 con 0x7f00c8107660 2024-09-19T09:19:08.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.022+0000 7f00c5ffb640 1 -- 172.21.15.16:0/2701294602 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00b8035030 con 0x7f00c8107660 2024-09-19T09:19:08.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.022+0000 7f00ced13640 1 -- 172.21.15.16:0/2701294602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c8197890 con 0x7f00c8107660 2024-09-19T09:19:08.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.022+0000 7f00ced13640 1 -- 172.21.15.16:0/2701294602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c819b990 con 0x7f00c8107660 2024-09-19T09:19:08.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.023+0000 7f00c5ffb640 1 -- 172.21.15.16:0/2701294602 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f00b8042070 con 0x7f00c8107660 2024-09-19T09:19:08.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.023+0000 7f00ced13640 1 -- 172.21.15.16:0/2701294602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c8102cd0 con 0x7f00c8107660 2024-09-19T09:19:08.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.023+0000 7f00c5ffb640 1 --2- 172.21.15.16:0/2701294602 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f00b4039300 0x7f00b403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:08.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.023+0000 7f00c5ffb640 1 -- 172.21.15.16:0/2701294602 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f00b806cf00 con 0x7f00c8107660 2024-09-19T09:19:08.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.023+0000 7f00c7fff640 1 --2- 172.21.15.16:0/2701294602 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f00b4039300 0x7f00b403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:08.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.024+0000 7f00c7fff640 1 --2- 172.21.15.16:0/2701294602 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f00b4039300 0x7f00b403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f00b00098c0 tx=0x7f00b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:08.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.028+0000 7f00c5ffb640 1 -- 172.21.15.16:0/2701294602 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f00b80412c0 con 0x7f00c8107660 2024-09-19T09:19:08.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:08.211+0000 7f00ced13640 1 -- 172.21.15.16:0/2701294602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f00c8107ac0 con 0x7f00c8107660 2024-09-19T09:19:08.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:08 smithi016 ceph-mon[30979]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:19:09.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:09.167+0000 7f00c5ffb640 1 -- 172.21.15.16:0/2701294602 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f00b803b860 con 0x7f00c8107660 2024-09-19T09:19:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-09-19T09:19:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:09.171+0000 7f00ced13640 1 -- 172.21.15.16:0/2701294602 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f00b4039300 msgr2=0x7f00b403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:09.171+0000 7f00ced13640 1 --2- 172.21.15.16:0/2701294602 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f00b4039300 0x7f00b403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f00b00098c0 tx=0x7f00b0006db0 comp rx=0 tx=0).stop 2024-09-19T09:19:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:09.171+0000 7f00ced13640 1 -- 172.21.15.16:0/2701294602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c8107660 msgr2=0x7f00c81970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:09.171+0000 7f00ced13640 1 --2- 172.21.15.16:0/2701294602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c8107660 0x7f00c81970c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009230 tx=0x7f00b8003d80 comp rx=0 tx=0).stop 2024-09-19T09:19:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:09.171+0000 7f00ced13640 1 -- 172.21.15.16:0/2701294602 shutdown_connections 2024-09-19T09:19:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:09.171+0000 7f00ced13640 1 --2- 172.21.15.16:0/2701294602 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f00b4039300 0x7f00b403b7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:09.171+0000 7f00ced13640 1 --2- 172.21.15.16:0/2701294602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c8107660 0x7f00c81970c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:09.171+0000 7f00ced13640 1 -- 172.21.15.16:0/2701294602 >> 172.21.15.16:0/2701294602 conn(0x7f00c80fd290 msgr2=0x7f00c80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:09.171+0000 7f00ced13640 1 -- 172.21.15.16:0/2701294602 shutdown_connections 2024-09-19T09:19:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:09.171+0000 7f00ced13640 1 -- 172.21.15.16:0/2701294602 wait complete. 2024-09-19T09:19:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:09 smithi016 ceph-mon[30979]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:09 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2701294602' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T09:19:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:09 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:09 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:09 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:09 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:09 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:19:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:09 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:09.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:09 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:09.864 INFO:tasks.cephadm:Adding mon.a on smithi016 2024-09-19T09:19:09.864 INFO:tasks.cephadm:Adding mon.b on smithi097 2024-09-19T09:19:09.864 INFO:tasks.cephadm:Adding mon.c on smithi142 2024-09-19T09:19:09.865 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi016:172.21.15.16=a;smithi097:172.21.15.97=b;smithi142:172.21.15.142=c' 2024-09-19T09:19:10.110 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:19:10.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.464+0000 7f9ac0c45640 1 -- 172.21.15.142:0/546075122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc106560 msgr2=0x7f9abc106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:10.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.464+0000 7f9ac0c45640 1 --2- 172.21.15.142:0/546075122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc106560 0x7f9abc106940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009880 tx=0x7f9aac02a5d0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.464+0000 7f9ac0c45640 1 -- 172.21.15.142:0/546075122 shutdown_connections 2024-09-19T09:19:10.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.464+0000 7f9ac0c45640 1 --2- 172.21.15.142:0/546075122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc106560 0x7f9abc106940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.464+0000 7f9ac0c45640 1 -- 172.21.15.142:0/546075122 >> 172.21.15.142:0/546075122 conn(0x7f9abc0fc1b0 msgr2=0x7f9abc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:10.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.465+0000 7f9ac0c45640 1 -- 172.21.15.142:0/546075122 shutdown_connections 2024-09-19T09:19:10.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.465+0000 7f9ac0c45640 1 -- 172.21.15.142:0/546075122 wait complete. 2024-09-19T09:19:10.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.465+0000 7f9ac0c45640 1 Processor -- start 2024-09-19T09:19:10.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.465+0000 7f9ac0c45640 1 -- start start 2024-09-19T09:19:10.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.466+0000 7f9ac0c45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc106560 0x7f9abc197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.466+0000 7f9ac0c45640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc10ab90 con 0x7f9abc106560 2024-09-19T09:19:10.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.466+0000 7f9aba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc106560 0x7f9abc197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.466+0000 7f9aba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc106560 0x7f9abc197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.142:56564/0 (socket says 172.21.15.142:56564) 2024-09-19T09:19:10.470 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.466+0000 7f9aba59c640 1 -- 172.21.15.142:0/3701135392 learned_addr learned my addr 172.21.15.142:0/3701135392 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-19T09:19:10.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.466+0000 7f9aba59c640 1 -- 172.21.15.142:0/3701135392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc1976d0 con 0x7f9abc106560 2024-09-19T09:19:10.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.467+0000 7f9aba59c640 1 --2- 172.21.15.142:0/3701135392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc106560 0x7f9abc197190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009230 tx=0x7f9aac003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:10.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.467+0000 7f9ab3fff640 1 -- 172.21.15.142:0/3701135392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aac03f070 con 0x7f9abc106560 2024-09-19T09:19:10.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.467+0000 7f9ac0c45640 1 -- 172.21.15.142:0/3701135392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9abc197960 con 0x7f9abc106560 2024-09-19T09:19:10.471 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.467+0000 7f9ab3fff640 1 -- 172.21.15.142:0/3701135392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aac03bc80 con 0x7f9abc106560 2024-09-19T09:19:10.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.467+0000 7f9ab3fff640 1 -- 172.21.15.142:0/3701135392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aac035030 con 0x7f9abc106560 2024-09-19T09:19:10.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.467+0000 7f9ac0c45640 1 -- 172.21.15.142:0/3701135392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9abc197dc0 con 0x7f9abc106560 2024-09-19T09:19:10.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.468+0000 7f9ab3fff640 1 -- 172.21.15.142:0/3701135392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f9aac042070 con 0x7f9abc106560 2024-09-19T09:19:10.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.468+0000 7f9ac0c45640 1 -- 172.21.15.142:0/3701135392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9abc101bd0 con 0x7f9abc106560 2024-09-19T09:19:10.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.468+0000 7f9ab3fff640 1 --2- 172.21.15.142:0/3701135392 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9aa80392a0 0x7f9aa803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:10.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.468+0000 7f9ab3fff640 1 -- 172.21.15.142:0/3701135392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9aac06d2c0 con 0x7f9abc106560 2024-09-19T09:19:10.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.469+0000 7f9ab1d9b640 1 --2- 172.21.15.142:0/3701135392 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9aa80392a0 0x7f9aa803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:10.472 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.469+0000 7f9ab1d9b640 1 --2- 172.21.15.142:0/3701135392 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9aa80392a0 0x7f9aa803b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40098c0 tx=0x7f9aa4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:10.474 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.473+0000 7f9ab3fff640 1 -- 172.21.15.142:0/3701135392 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9aac04c350 con 0x7f9abc106560 2024-09-19T09:19:10.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:10 smithi016 ceph-mon[30979]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-19T09:19:10.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:10 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2701294602' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T09:19:10.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:10 smithi016 ceph-mon[30979]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:19:10.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:10 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:10.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:10 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:10.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:10 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:10.650 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.649+0000 7f9ac0c45640 1 -- 172.21.15.142:0/3701135392 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi097:172.21.15.97=b;smithi142:172.21.15.142=c", "target": ["mon-mgr", ""]}) -- 0x7f9abc1985b0 con 0x7f9aa80392a0 2024-09-19T09:19:10.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.655+0000 7f9ab3fff640 1 -- 172.21.15.142:0/3701135392 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9abc1985b0 con 0x7f9aa80392a0 2024-09-19T09:19:10.656 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mon update... 2024-09-19T09:19:10.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.657+0000 7f9ac0c45640 1 -- 172.21.15.142:0/3701135392 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9aa80392a0 msgr2=0x7f9aa803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:10.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.657+0000 7f9ac0c45640 1 --2- 172.21.15.142:0/3701135392 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9aa80392a0 0x7f9aa803b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40098c0 tx=0x7f9aa4006db0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.657+0000 7f9ac0c45640 1 -- 172.21.15.142:0/3701135392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc106560 msgr2=0x7f9abc197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:10.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.657+0000 7f9ac0c45640 1 --2- 172.21.15.142:0/3701135392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc106560 0x7f9abc197190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009230 tx=0x7f9aac003d40 comp rx=0 tx=0).stop 2024-09-19T09:19:10.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.657+0000 7f9ac0c45640 1 -- 172.21.15.142:0/3701135392 shutdown_connections 2024-09-19T09:19:10.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.657+0000 7f9ac0c45640 1 --2- 172.21.15.142:0/3701135392 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9aa80392a0 0x7f9aa803b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.657+0000 7f9ac0c45640 1 --2- 172.21.15.142:0/3701135392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc106560 0x7f9abc197190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:10.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.657+0000 7f9ac0c45640 1 -- 172.21.15.142:0/3701135392 >> 172.21.15.142:0/3701135392 conn(0x7f9abc0fc1b0 msgr2=0x7f9abc069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:10.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.657+0000 7f9ac0c45640 1 -- 172.21.15.142:0/3701135392 shutdown_connections 2024-09-19T09:19:10.658 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:10.657+0000 7f9ac0c45640 1 -- 172.21.15.142:0/3701135392 wait complete. 2024-09-19T09:19:11.253 DEBUG:teuthology.orchestra.run.smithi097:mon.b> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T09:19:11.255 DEBUG:teuthology.orchestra.run.smithi142:mon.c> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T09:19:11.261 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T09:19:11.261 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T09:19:11.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:11 smithi016 ceph-mon[30979]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:11.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:11 smithi016 ceph-mon[30979]: Updating smithi142:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:19:11.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:11 smithi016 ceph-mon[30979]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:19:11.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:11 smithi016 ceph-mon[30979]: Updating smithi142:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T09:19:11.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:11.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:11.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:11.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:11.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:11.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:19:11.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:12 smithi016 ceph-mon[30979]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi097:172.21.15.97=b;smithi142:172.21.15.142=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:19:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:12 smithi016 ceph-mon[30979]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi097:172.21.15.97=b;smithi142:172.21.15.142=c;count:3 2024-09-19T09:19:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:12 smithi016 ceph-mon[30979]: Deploying daemon mon.c on smithi142 2024-09-19T09:19:12.858 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:19:13.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.495+0000 7fa8f34ad640 1 -- 172.21.15.142:0/1444260304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ec107650 msgr2=0x7fa8ec107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:13.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.495+0000 7fa8f34ad640 1 --2- 172.21.15.142:0/1444260304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ec107650 0x7fa8ec107a30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009880 tx=0x7fa8e802a5d0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.495+0000 7fa8f34ad640 1 -- 172.21.15.142:0/1444260304 shutdown_connections 2024-09-19T09:19:13.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.495+0000 7fa8f34ad640 1 --2- 172.21.15.142:0/1444260304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ec107650 0x7fa8ec107a30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.495+0000 7fa8f34ad640 1 -- 172.21.15.142:0/1444260304 >> 172.21.15.142:0/1444260304 conn(0x7fa8ec0fd260 msgr2=0x7fa8ec0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:13.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.495+0000 7fa8f34ad640 1 -- 172.21.15.142:0/1444260304 shutdown_connections 2024-09-19T09:19:13.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.495+0000 7fa8f34ad640 1 -- 172.21.15.142:0/1444260304 wait complete. 2024-09-19T09:19:13.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.496+0000 7fa8f34ad640 1 Processor -- start 2024-09-19T09:19:13.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.496+0000 7fa8f34ad640 1 -- start start 2024-09-19T09:19:13.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.496+0000 7fa8f34ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ec107650 0x7fa8ec197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:13.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.496+0000 7fa8f34ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec10bc80 con 0x7fa8ec107650 2024-09-19T09:19:13.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.496+0000 7fa8f1249640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ec107650 0x7fa8ec197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:13.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.496+0000 7fa8f1249640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ec107650 0x7fa8ec197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.142:41624/0 (socket says 172.21.15.142:41624) 2024-09-19T09:19:13.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.496+0000 7fa8f1249640 1 -- 172.21.15.142:0/151613798 learned_addr learned my addr 172.21.15.142:0/151613798 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-19T09:19:13.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.497+0000 7fa8f1249640 1 -- 172.21.15.142:0/151613798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec197660 con 0x7fa8ec107650 2024-09-19T09:19:13.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.497+0000 7fa8f1249640 1 --2- 172.21.15.142:0/151613798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ec107650 0x7fa8ec197120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009230 tx=0x7fa8e8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:13.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.497+0000 7fa8e27fc640 1 -- 172.21.15.142:0/151613798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8e803f070 con 0x7fa8ec107650 2024-09-19T09:19:13.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.497+0000 7fa8f34ad640 1 -- 172.21.15.142:0/151613798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ec1978f0 con 0x7fa8ec107650 2024-09-19T09:19:13.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.497+0000 7fa8e27fc640 1 -- 172.21.15.142:0/151613798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e803bc80 con 0x7fa8ec107650 2024-09-19T09:19:13.499 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.497+0000 7fa8e27fc640 1 -- 172.21.15.142:0/151613798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8e8035030 con 0x7fa8ec107650 2024-09-19T09:19:13.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.497+0000 7fa8f34ad640 1 -- 172.21.15.142:0/151613798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ec19b9f0 con 0x7fa8ec107650 2024-09-19T09:19:13.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.498+0000 7fa8e27fc640 1 -- 172.21.15.142:0/151613798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fa8e8042070 con 0x7fa8ec107650 2024-09-19T09:19:13.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.498+0000 7fa8f34ad640 1 -- 172.21.15.142:0/151613798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8ec102cc0 con 0x7fa8ec107650 2024-09-19T09:19:13.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.498+0000 7fa8e27fc640 1 --2- 172.21.15.142:0/151613798 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa8d80392a0 0x7fa8d803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:13.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.498+0000 7fa8e27fc640 1 -- 172.21.15.142:0/151613798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa8e806d0b0 con 0x7fa8ec107650 2024-09-19T09:19:13.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.498+0000 7fa8f0a48640 1 --2- 172.21.15.142:0/151613798 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa8d80392a0 0x7fa8d803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:13.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.499+0000 7fa8f0a48640 1 --2- 172.21.15.142:0/151613798 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa8d80392a0 0x7fa8d803b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0098c0 tx=0x7fa8dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:13.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:13 smithi016 ceph-mon[30979]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:13.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.503+0000 7fa8e27fc640 1 -- 172.21.15.142:0/151613798 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa8e8031e20 con 0x7fa8ec107650 2024-09-19T09:19:13.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.746+0000 7fa8f34ad640 1 -- 172.21.15.142:0/151613798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa8ec198480 con 0x7fa8ec107650 2024-09-19T09:19:13.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.747+0000 7fa8e27fc640 1 -- 172.21.15.142:0/151613798 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fa8e8031570 con 0x7fa8ec107650 2024-09-19T09:19:13.748 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:19:13.748 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","modified":"2024-09-19T09:17:41.219429Z","created":"2024-09-19T09:17:41.219429Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T09:19:13.748 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-09-19T09:19:13.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.749+0000 7fa8f34ad640 1 -- 172.21.15.142:0/151613798 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa8d80392a0 msgr2=0x7fa8d803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:13.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.749+0000 7fa8f34ad640 1 --2- 172.21.15.142:0/151613798 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa8d80392a0 0x7fa8d803b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0098c0 tx=0x7fa8dc006db0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.749+0000 7fa8f34ad640 1 -- 172.21.15.142:0/151613798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ec107650 msgr2=0x7fa8ec197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:13.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.749+0000 7fa8f34ad640 1 --2- 172.21.15.142:0/151613798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ec107650 0x7fa8ec197120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009230 tx=0x7fa8e8003d40 comp rx=0 tx=0).stop 2024-09-19T09:19:13.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.749+0000 7fa8f34ad640 1 -- 172.21.15.142:0/151613798 shutdown_connections 2024-09-19T09:19:13.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.749+0000 7fa8f34ad640 1 --2- 172.21.15.142:0/151613798 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa8d80392a0 0x7fa8d803b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.749+0000 7fa8f34ad640 1 --2- 172.21.15.142:0/151613798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8ec107650 0x7fa8ec197120 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:13.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.749+0000 7fa8f34ad640 1 -- 172.21.15.142:0/151613798 >> 172.21.15.142:0/151613798 conn(0x7fa8ec0fd260 msgr2=0x7fa8ec0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:13.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.749+0000 7fa8f34ad640 1 -- 172.21.15.142:0/151613798 shutdown_connections 2024-09-19T09:19:13.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:13.749+0000 7fa8f34ad640 1 -- 172.21.15.142:0/151613798 wait complete. 2024-09-19T09:19:14.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:14 smithi016 ceph-mon[30979]: from='client.? 172.21.15.142:0/151613798' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:19:15.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:15 smithi142 systemd[1]: Started Ceph mon.c for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:19:15.517 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T09:19:15.518 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T09:19:15.763 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:19:20.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:19:20.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: mon.a calling monitor election 2024-09-19T09:19:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: mon.c calling monitor election 2024-09-19T09:19:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T09:19:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: monmap epoch 2 2024-09-19T09:19:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: fsid ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:19:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: last_changed 2024-09-19T09:19:15.092613+0000 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: created 2024-09-19T09:17:41.219429+0000 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: min_mon_release 19 (squid) 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: election_strategy: 1 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: fsmap 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: mgrmap e13: x(active, since 38s) 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: overall HEALTH_OK 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:19:20.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:20.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.615+0000 7f6380b1a640 1 -- 172.21.15.142:0/49951929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368003530 msgr2=0x7f63680059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:20.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.615+0000 7f6380b1a640 1 --2- 172.21.15.142:0/49951929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368003530 0x7f63680059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f637c1016a0 tx=0x7f636c02c490 comp rx=0 tx=0).stop 2024-09-19T09:19:20.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.616+0000 7f6380b1a640 1 -- 172.21.15.142:0/49951929 shutdown_connections 2024-09-19T09:19:20.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.616+0000 7f6380b1a640 1 --2- 172.21.15.142:0/49951929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368003530 0x7f63680059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:20.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.616+0000 7f6380b1a640 1 -- 172.21.15.142:0/49951929 >> 172.21.15.142:0/49951929 conn(0x7f637c0fc220 msgr2=0x7f637c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:20.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.616+0000 7f6380b1a640 1 -- 172.21.15.142:0/49951929 shutdown_connections 2024-09-19T09:19:20.617 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.616+0000 7f6380b1a640 1 -- 172.21.15.142:0/49951929 wait complete. 2024-09-19T09:19:20.618 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:19:20.618 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:20.618 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: mon.a calling monitor election 2024-09-19T09:19:20.618 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:20.618 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:20.618 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:20.618 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: mon.c calling monitor election 2024-09-19T09:19:20.618 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:20.618 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: monmap epoch 2 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: fsid ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: last_changed 2024-09-19T09:19:15.092613+0000 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: created 2024-09-19T09:17:41.219429+0000 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: min_mon_release 19 (squid) 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: election_strategy: 1 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: fsmap 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:19:20.619 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: mgrmap e13: x(active, since 38s) 2024-09-19T09:19:20.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: overall HEALTH_OK 2024-09-19T09:19:20.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:20.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:20.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:20.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:19:20.620 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:20.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.617+0000 7f6380b1a640 1 Processor -- start 2024-09-19T09:19:20.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.617+0000 7f6380b1a640 1 -- start start 2024-09-19T09:19:20.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.617+0000 7f6380b1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368003530 0x7f637c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:20.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.617+0000 7f6380b1a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f637c1977d0 0x7f637c19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:20.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.617+0000 7f6380b1a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f637c108670 con 0x7f6368003530 2024-09-19T09:19:20.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.617+0000 7f6380b1a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f637c1084f0 con 0x7f637c1977d0 2024-09-19T09:19:20.622 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.618+0000 7f6379d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f637c1977d0 0x7f637c19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:20.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.618+0000 7f6379d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f637c1977d0 0x7f637c19bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45088/0 (socket says 172.21.15.142:45088) 2024-09-19T09:19:20.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.618+0000 7f6379d9b640 1 -- 172.21.15.142:0/1349224922 learned_addr learned my addr 172.21.15.142:0/1349224922 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-19T09:19:20.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.618+0000 7f637a59c640 1 --2- 172.21.15.142:0/1349224922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368003530 0x7f637c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:20.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.618+0000 7f6379d9b640 1 -- 172.21.15.142:0/1349224922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f637c1977d0 msgr2=0x7f637c19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T09:19:20.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.618+0000 7f6379d9b640 1 -- 172.21.15.142:0/1349224922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f637c1977d0 msgr2=0x7f637c19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T09:19:20.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.618+0000 7f6379d9b640 1 --2- 172.21.15.142:0/1349224922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f637c1977d0 0x7f637c19bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T09:19:20.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.618+0000 7f6379d9b640 1 --2- 172.21.15.142:0/1349224922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f637c1977d0 0x7f637c19bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:19:20.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.618+0000 7f637a59c640 1 -- 172.21.15.142:0/1349224922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f637c1977d0 msgr2=0x7f637c19bb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:19:20.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.618+0000 7f637a59c640 1 --2- 172.21.15.142:0/1349224922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f637c1977d0 0x7f637c19bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:20.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.618+0000 7f637a59c640 1 -- 172.21.15.142:0/1349224922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637c197df0 con 0x7f6368003530 2024-09-19T09:19:20.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.618+0000 7f637a59c640 1 --2- 172.21.15.142:0/1349224922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368003530 0x7f637c197290 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f636c02ca80 tx=0x7f636c0339a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:20.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.619+0000 7f63737fe640 1 -- 172.21.15.142:0/1349224922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f636c0329e0 con 0x7f6368003530 2024-09-19T09:19:20.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.619+0000 7f63737fe640 1 -- 172.21.15.142:0/1349224922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f636c005520 con 0x7f6368003530 2024-09-19T09:19:20.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.619+0000 7f6380b1a640 1 -- 172.21.15.142:0/1349224922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637c19c0a0 con 0x7f6368003530 2024-09-19T09:19:20.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.619+0000 7f63737fe640 1 -- 172.21.15.142:0/1349224922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f636c004bc0 con 0x7f6368003530 2024-09-19T09:19:20.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.619+0000 7f6380b1a640 1 -- 172.21.15.142:0/1349224922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f637c19c630 con 0x7f6368003530 2024-09-19T09:19:20.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.620+0000 7f63737fe640 1 -- 172.21.15.142:0/1349224922 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f636c043020 con 0x7f6368003530 2024-09-19T09:19:20.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.620+0000 7f6380b1a640 1 -- 172.21.15.142:0/1349224922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f637c101ca0 con 0x7f6368003530 2024-09-19T09:19:20.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.620+0000 7f63737fe640 1 --2- 172.21.15.142:0/1349224922 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f635c039090 0x7f635c03b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:20.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.620+0000 7f63737fe640 1 -- 172.21.15.142:0/1349224922 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f636c041020 con 0x7f6368003530 2024-09-19T09:19:20.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.620+0000 7f6379d9b640 1 --2- 172.21.15.142:0/1349224922 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f635c039090 0x7f635c03b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:20.624 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.621+0000 7f6379d9b640 1 --2- 172.21.15.142:0/1349224922 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f635c039090 0x7f635c03b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6364009100 tx=0x7f6364008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:20.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.626+0000 7f63737fe640 1 -- 172.21.15.142:0/1349224922 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f636c04a020 con 0x7f6368003530 2024-09-19T09:19:20.861 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.860+0000 7f6380b1a640 1 -- 172.21.15.142:0/1349224922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f637c000900 con 0x7f6368003530 2024-09-19T09:19:20.862 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.861+0000 7f63737fe640 1 -- 172.21.15.142:0/1349224922 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f636c03a940 con 0x7f6368003530 2024-09-19T09:19:20.862 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:19:20.862 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":2,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","modified":"2024-09-19T09:19:15.092613Z","created":"2024-09-19T09:17:41.219429Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T09:19:20.862 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 2 2024-09-19T09:19:20.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.863+0000 7f6380b1a640 1 -- 172.21.15.142:0/1349224922 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f635c039090 msgr2=0x7f635c03b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:20.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.863+0000 7f6380b1a640 1 --2- 172.21.15.142:0/1349224922 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f635c039090 0x7f635c03b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6364009100 tx=0x7f6364008040 comp rx=0 tx=0).stop 2024-09-19T09:19:20.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.863+0000 7f6380b1a640 1 -- 172.21.15.142:0/1349224922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368003530 msgr2=0x7f637c197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:20.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.863+0000 7f6380b1a640 1 --2- 172.21.15.142:0/1349224922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368003530 0x7f637c197290 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f636c02ca80 tx=0x7f636c0339a0 comp rx=0 tx=0).stop 2024-09-19T09:19:20.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.863+0000 7f6380b1a640 1 -- 172.21.15.142:0/1349224922 shutdown_connections 2024-09-19T09:19:20.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.863+0000 7f6380b1a640 1 --2- 172.21.15.142:0/1349224922 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f635c039090 0x7f635c03b550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:20.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.863+0000 7f6380b1a640 1 --2- 172.21.15.142:0/1349224922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f637c1977d0 0x7f637c19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:20.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.863+0000 7f6380b1a640 1 --2- 172.21.15.142:0/1349224922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368003530 0x7f637c197290 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:20.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.863+0000 7f6380b1a640 1 -- 172.21.15.142:0/1349224922 >> 172.21.15.142:0/1349224922 conn(0x7f637c0fc220 msgr2=0x7f637c06a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:20.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.863+0000 7f6380b1a640 1 -- 172.21.15.142:0/1349224922 shutdown_connections 2024-09-19T09:19:20.865 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:20.863+0000 7f6380b1a640 1 -- 172.21.15.142:0/1349224922 wait complete. 2024-09-19T09:19:21.441 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:21 smithi142 ceph-mon[35032]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:21.441 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:21 smithi142 ceph-mon[35032]: Deploying daemon mon.b on smithi097 2024-09-19T09:19:21.441 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:21 smithi142 ceph-mon[35032]: from='client.? 172.21.15.142:0/1349224922' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:19:21.441 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:21.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:21 smithi016 ceph-mon[30979]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:21.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:21 smithi016 ceph-mon[30979]: Deploying daemon mon.b on smithi097 2024-09-19T09:19:21.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:21 smithi016 ceph-mon[30979]: from='client.? 172.21.15.142:0/1349224922' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:19:21.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:22.468 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T09:19:22.469 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T09:19:22.717 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:19:23.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.149+0000 7fb38831d640 1 -- 172.21.15.142:0/3186327536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb380107660 msgr2=0x7fb380107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:23.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.149+0000 7fb38831d640 1 --2- 172.21.15.142:0/3186327536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb380107660 0x7fb380107a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb370009880 tx=0x7fb37002a5d0 comp rx=0 tx=0).stop 2024-09-19T09:19:23.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.149+0000 7fb38831d640 1 -- 172.21.15.142:0/3186327536 shutdown_connections 2024-09-19T09:19:23.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.149+0000 7fb38831d640 1 --2- 172.21.15.142:0/3186327536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb380107660 0x7fb380107a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:23.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.149+0000 7fb38831d640 1 -- 172.21.15.142:0/3186327536 >> 172.21.15.142:0/3186327536 conn(0x7fb3800fd320 msgr2=0x7fb3800ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:23.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.149+0000 7fb38831d640 1 -- 172.21.15.142:0/3186327536 shutdown_connections 2024-09-19T09:19:23.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.149+0000 7fb38831d640 1 -- 172.21.15.142:0/3186327536 wait complete. 2024-09-19T09:19:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb38831d640 1 Processor -- start 2024-09-19T09:19:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb38831d640 1 -- start start 2024-09-19T09:19:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb38831d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb380107660 0x7fb3801973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb38831d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb380197900 0x7fb38019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb38831d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb38010bd20 con 0x7fb380197900 2024-09-19T09:19:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb38831d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb38010bba0 con 0x7fb380107660 2024-09-19T09:19:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb3860b9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb380107660 0x7fb3801973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb3860b9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb380107660 0x7fb3801973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47560/0 (socket says 172.21.15.142:47560) 2024-09-19T09:19:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb3860b9640 1 -- 172.21.15.142:0/2414000332 learned_addr learned my addr 172.21.15.142:0/2414000332 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-19T09:19:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb3858b8640 1 --2- 172.21.15.142:0/2414000332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb380197900 0x7fb38019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb3860b9640 1 -- 172.21.15.142:0/2414000332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb380107660 msgr2=0x7fb3801973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T09:19:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb3860b9640 1 -- 172.21.15.142:0/2414000332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb380107660 msgr2=0x7fb3801973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T09:19:23.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb3860b9640 1 --2- 172.21.15.142:0/2414000332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb380107660 0x7fb3801973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T09:19:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.150+0000 7fb3860b9640 1 --2- 172.21.15.142:0/2414000332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb380107660 0x7fb3801973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:19:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.151+0000 7fb3858b8640 1 -- 172.21.15.142:0/2414000332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb380107660 msgr2=0x7fb3801973c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:19:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.151+0000 7fb3858b8640 1 --2- 172.21.15.142:0/2414000332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb380107660 0x7fb3801973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.151+0000 7fb3858b8640 1 -- 172.21.15.142:0/2414000332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb380197f20 con 0x7fb380197900 2024-09-19T09:19:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.151+0000 7fb3858b8640 1 --2- 172.21.15.142:0/2414000332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb380197900 0x7fb38019bc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00b3d0 tx=0x7fb37c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.151+0000 7fb3777fe640 1 -- 172.21.15.142:0/2414000332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb37c011630 con 0x7fb380197900 2024-09-19T09:19:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.151+0000 7fb38831d640 1 -- 172.21.15.142:0/2414000332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38019c1d0 con 0x7fb380197900 2024-09-19T09:19:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.151+0000 7fb3777fe640 1 -- 172.21.15.142:0/2414000332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb37c011cb0 con 0x7fb380197900 2024-09-19T09:19:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.151+0000 7fb3777fe640 1 -- 172.21.15.142:0/2414000332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb37c010df0 con 0x7fb380197900 2024-09-19T09:19:23.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.151+0000 7fb38831d640 1 -- 172.21.15.142:0/2414000332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38019c710 con 0x7fb380197900 2024-09-19T09:19:23.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.152+0000 7fb38831d640 1 -- 172.21.15.142:0/2414000332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb380102da0 con 0x7fb380197900 2024-09-19T09:19:23.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.152+0000 7fb3777fe640 1 -- 172.21.15.142:0/2414000332 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fb37c018020 con 0x7fb380197900 2024-09-19T09:19:23.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.152+0000 7fb3777fe640 1 --2- 172.21.15.142:0/2414000332 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb36c039040 0x7fb36c03b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:23.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.152+0000 7fb3777fe640 1 -- 172.21.15.142:0/2414000332 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb37c04bf30 con 0x7fb380197900 2024-09-19T09:19:23.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.153+0000 7fb3860b9640 1 --2- 172.21.15.142:0/2414000332 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb36c039040 0x7fb36c03b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:23.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.153+0000 7fb3860b9640 1 --2- 172.21.15.142:0/2414000332 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb36c039040 0x7fb36c03b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb3700096c0 tx=0x7fb370005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:23.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.157+0000 7fb3777fe640 1 -- 172.21.15.142:0/2414000332 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb37c01f020 con 0x7fb380197900 2024-09-19T09:19:23.387 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.386+0000 7fb38831d640 1 -- 172.21.15.142:0/2414000332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb38019ca20 con 0x7fb380197900 2024-09-19T09:19:23.388 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:23 smithi142 ceph-mon[35032]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:23.388 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.387+0000 7fb3777fe640 1 -- 172.21.15.142:0/2414000332 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fb37c0172c0 con 0x7fb380197900 2024-09-19T09:19:23.389 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:19:23.389 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":2,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","modified":"2024-09-19T09:19:15.092613Z","created":"2024-09-19T09:17:41.219429Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T09:19:23.389 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 2 2024-09-19T09:19:23.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.389+0000 7fb38831d640 1 -- 172.21.15.142:0/2414000332 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb36c039040 msgr2=0x7fb36c03b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:23.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.389+0000 7fb38831d640 1 --2- 172.21.15.142:0/2414000332 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb36c039040 0x7fb36c03b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb3700096c0 tx=0x7fb370005a40 comp rx=0 tx=0).stop 2024-09-19T09:19:23.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.390+0000 7fb38831d640 1 -- 172.21.15.142:0/2414000332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb380197900 msgr2=0x7fb38019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:23.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.390+0000 7fb38831d640 1 --2- 172.21.15.142:0/2414000332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb380197900 0x7fb38019bc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00b3d0 tx=0x7fb37c00b890 comp rx=0 tx=0).stop 2024-09-19T09:19:23.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.390+0000 7fb38831d640 1 -- 172.21.15.142:0/2414000332 shutdown_connections 2024-09-19T09:19:23.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.390+0000 7fb38831d640 1 --2- 172.21.15.142:0/2414000332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb380197900 0x7fb38019bc90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:23.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.390+0000 7fb38831d640 1 --2- 172.21.15.142:0/2414000332 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb36c039040 0x7fb36c03b500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:23.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.390+0000 7fb38831d640 1 --2- 172.21.15.142:0/2414000332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb380107660 0x7fb3801973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:23.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.390+0000 7fb38831d640 1 -- 172.21.15.142:0/2414000332 >> 172.21.15.142:0/2414000332 conn(0x7fb3800fd320 msgr2=0x7fb3800fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:23.391 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.390+0000 7fb38831d640 1 -- 172.21.15.142:0/2414000332 shutdown_connections 2024-09-19T09:19:23.392 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:23.390+0000 7fb38831d640 1 -- 172.21.15.142:0/2414000332 wait complete. 2024-09-19T09:19:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:23 smithi016 ceph-mon[30979]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:24.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:23 smithi097 systemd[1]: Started Ceph mon.b for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:19:24.968 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T09:19:24.968 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T09:19:25.218 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:19:28.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.887+0000 7feaec5f7640 1 -- 172.21.15.142:0/610319259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead0005740 msgr2=0x7fead0007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:28.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.887+0000 7feaec5f7640 1 --2- 172.21.15.142:0/610319259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead0005740 0x7fead0007bd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7feae41016c0 tx=0x7fead402c150 comp rx=0 tx=0).stop 2024-09-19T09:19:28.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.887+0000 7feaec5f7640 1 -- 172.21.15.142:0/610319259 shutdown_connections 2024-09-19T09:19:28.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.887+0000 7feaec5f7640 1 --2- 172.21.15.142:0/610319259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead0005740 0x7fead0007bd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:28.889 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.887+0000 7feaec5f7640 1 -- 172.21.15.142:0/610319259 >> 172.21.15.142:0/610319259 conn(0x7feae40fc280 msgr2=0x7feae40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:28.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.888+0000 7feaec5f7640 1 -- 172.21.15.142:0/610319259 shutdown_connections 2024-09-19T09:19:28.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.888+0000 7feaec5f7640 1 -- 172.21.15.142:0/610319259 wait complete. 2024-09-19T09:19:28.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.888+0000 7feaec5f7640 1 Processor -- start 2024-09-19T09:19:28.890 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.888+0000 7feaec5f7640 1 -- start start 2024-09-19T09:19:28.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaec5f7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fead0005740 0x7feae41994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:28.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaec5f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4199a20 0x7feae419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:28.891 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaec5f7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae419e2f0 0x7feae419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:28.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaec5f7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feae41075c0 con 0x7feae4199a20 2024-09-19T09:19:28.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaec5f7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feae4107440 con 0x7feae419e2f0 2024-09-19T09:19:28.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaec5f7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feae4107740 con 0x7fead0005740 2024-09-19T09:19:28.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaea393640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fead0005740 0x7feae41994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:28.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaeab94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae419e2f0 0x7feae419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:28.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feae9b92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4199a20 0x7feae419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:28.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaeab94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae419e2f0 0x7feae419e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.142:32954/0 (socket says 172.21.15.142:32954) 2024-09-19T09:19:28.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaeab94640 1 -- 172.21.15.142:0/3351153224 learned_addr learned my addr 172.21.15.142:0/3351153224 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-19T09:19:28.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaea393640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fead0005740 0x7feae41994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47572/0 (socket says 172.21.15.142:47572) 2024-09-19T09:19:28.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaeab94640 1 -- 172.21.15.142:0/3351153224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae419e2f0 msgr2=0x7feae419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-19T09:19:28.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaeab94640 1 -- 172.21.15.142:0/3351153224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae419e2f0 msgr2=0x7feae419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T09:19:28.892 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaeab94640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae419e2f0 0x7feae419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T09:19:28.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaeab94640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae419e2f0 0x7feae419e7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:19:28.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaea393640 1 -- 172.21.15.142:0/3351153224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fead0005740 msgr2=0x7feae41994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T09:19:28.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaea393640 1 -- 172.21.15.142:0/3351153224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fead0005740 msgr2=0x7feae41994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T09:19:28.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaea393640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fead0005740 0x7feae41994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T09:19:28.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.889+0000 7feaea393640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fead0005740 0x7feae41994e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:19:28.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.890+0000 7feae9b92640 1 -- 172.21.15.142:0/3351153224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fead0005740 msgr2=0x7feae41994e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:19:28.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.890+0000 7feae9b92640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fead0005740 0x7feae41994e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:28.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.890+0000 7feae9b92640 1 -- 172.21.15.142:0/3351153224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae419e2f0 msgr2=0x7feae419e7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:19:28.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.890+0000 7feae9b92640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae419e2f0 0x7feae419e7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:28.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.890+0000 7feae9b92640 1 -- 172.21.15.142:0/3351153224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae41a6060 con 0x7feae4199a20 2024-09-19T09:19:28.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.890+0000 7feae9b92640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4199a20 0x7feae419ddb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feae000b450 tx=0x7feae000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:28.893 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.890+0000 7feadb7fe640 1 -- 172.21.15.142:0/3351153224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0004230 con 0x7feae4199a20 2024-09-19T09:19:28.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.890+0000 7feadb7fe640 1 -- 172.21.15.142:0/3351153224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae00043d0 con 0x7feae4199a20 2024-09-19T09:19:28.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.890+0000 7feaec5f7640 1 -- 172.21.15.142:0/3351153224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae41a6290 con 0x7feae4199a20 2024-09-19T09:19:28.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.890+0000 7feadb7fe640 1 -- 172.21.15.142:0/3351153224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0003cb0 con 0x7feae4199a20 2024-09-19T09:19:28.894 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.890+0000 7feaec5f7640 1 -- 172.21.15.142:0/3351153224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae41a67a0 con 0x7feae4199a20 2024-09-19T09:19:28.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.891+0000 7feadb7fe640 1 -- 172.21.15.142:0/3351153224 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7feae000f050 con 0x7feae4199a20 2024-09-19T09:19:28.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.892+0000 7feadb7fe640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7feac8039730 0x7feac803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:28.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.892+0000 7feadb7fe640 1 -- 172.21.15.142:0/3351153224 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7feae004d180 con 0x7feae4199a20 2024-09-19T09:19:28.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.892+0000 7feaec5f7640 1 -- 172.21.15.142:0/3351153224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae41086a0 con 0x7feae4199a20 2024-09-19T09:19:28.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.892+0000 7feaea393640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7feac8039730 0x7feac803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:28.895 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.892+0000 7feaea393640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7feac8039730 0x7feac803bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fead4005dc0 tx=0x7fead4005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:28.897 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:28.897+0000 7feadb7fe640 1 -- 172.21.15.142:0/3351153224 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feae000d040 con 0x7feae4199a20 2024-09-19T09:19:29.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: mon.c calling monitor election 2024-09-19T09:19:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:19:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: mon.a calling monitor election 2024-09-19T09:19:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T09:19:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: monmap epoch 3 2024-09-19T09:19:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: fsid ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:19:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: last_changed 2024-09-19T09:19:23.745204+0000 2024-09-19T09:19:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: created 2024-09-19T09:17:41.219429+0000 2024-09-19T09:19:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: min_mon_release 19 (squid) 2024-09-19T09:19:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: election_strategy: 1 2024-09-19T09:19:29.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-19T09:19:29.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-19T09:19:29.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-09-19T09:19:29.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: fsmap 2024-09-19T09:19:29.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:19:29.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: mgrmap e13: x(active, since 47s) 2024-09-19T09:19:29.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: overall HEALTH_OK 2024-09-19T09:19:29.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:29.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:29.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:29.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:29.144 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.143+0000 7feaec5f7640 1 -- 172.21.15.142:0/3351153224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7feae4105010 con 0x7feae4199a20 2024-09-19T09:19:29.145 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.144+0000 7feadb7fe640 1 -- 172.21.15.142:0/3351153224 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7feae0011d00 con 0x7feae4199a20 2024-09-19T09:19:29.145 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:19:29.145 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":3,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","modified":"2024-09-19T09:19:23.745204Z","created":"2024-09-19T09:17:41.219429Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T09:19:29.145 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 3 2024-09-19T09:19:29.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.147+0000 7feaec5f7640 1 -- 172.21.15.142:0/3351153224 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7feac8039730 msgr2=0x7feac803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:29.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.147+0000 7feaec5f7640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7feac8039730 0x7feac803bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fead4005dc0 tx=0x7fead4005d10 comp rx=0 tx=0).stop 2024-09-19T09:19:29.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.147+0000 7feaec5f7640 1 -- 172.21.15.142:0/3351153224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4199a20 msgr2=0x7feae419ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:29.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.147+0000 7feaec5f7640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4199a20 0x7feae419ddb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feae000b450 tx=0x7feae000b910 comp rx=0 tx=0).stop 2024-09-19T09:19:29.148 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.147+0000 7feaec5f7640 1 -- 172.21.15.142:0/3351153224 shutdown_connections 2024-09-19T09:19:29.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.147+0000 7feaec5f7640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feae419e2f0 0x7feae419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:29.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.147+0000 7feaec5f7640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4199a20 0x7feae419ddb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:29.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.147+0000 7feaec5f7640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7feac8039730 0x7feac803bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:29.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.147+0000 7feaec5f7640 1 --2- 172.21.15.142:0/3351153224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fead0005740 0x7feae41994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:29.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.147+0000 7feaec5f7640 1 -- 172.21.15.142:0/3351153224 >> 172.21.15.142:0/3351153224 conn(0x7feae40fc280 msgr2=0x7feae40fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:29.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.147+0000 7feaec5f7640 1 -- 172.21.15.142:0/3351153224 shutdown_connections 2024-09-19T09:19:29.149 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:29.147+0000 7feaec5f7640 1 -- 172.21.15.142:0/3351153224 wait complete. 2024-09-19T09:19:29.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:29.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: mon.c calling monitor election 2024-09-19T09:19:29.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: mon.a calling monitor election 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: monmap epoch 3 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: fsid ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: last_changed 2024-09-19T09:19:23.745204+0000 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: created 2024-09-19T09:17:41.219429+0000 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: min_mon_release 19 (squid) 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: election_strategy: 1 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: fsmap 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: mgrmap e13: x(active, since 47s) 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: overall HEALTH_OK 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:29.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:29.725 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T09:19:29.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T09:19:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:30.015 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:29 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:30.015 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:29 smithi142 ceph-mon[35032]: from='client.? 172.21.15.142:0/3351153224' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:19:30.015 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:29 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:30.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:29 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:30.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:29 smithi016 ceph-mon[30979]: from='client.? 172.21.15.142:0/3351153224' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:19:30.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:29 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.429+0000 7fa24a517640 1 -- 172.21.15.16:0/1764324709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa244102730 msgr2=0x7fa244102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.429+0000 7fa24a517640 1 --2- 172.21.15.16:0/1764324709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa244102730 0x7fa244102b10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa234009810 tx=0x7fa23402a540 comp rx=0 tx=0).stop 2024-09-19T09:19:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.429+0000 7fa24a517640 1 -- 172.21.15.16:0/1764324709 shutdown_connections 2024-09-19T09:19:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.429+0000 7fa24a517640 1 --2- 172.21.15.16:0/1764324709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa244102730 0x7fa244102b10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.429+0000 7fa24a517640 1 -- 172.21.15.16:0/1764324709 >> 172.21.15.16:0/1764324709 conn(0x7fa244077310 msgr2=0x7fa2440fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.429+0000 7fa24a517640 1 -- 172.21.15.16:0/1764324709 shutdown_connections 2024-09-19T09:19:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.429+0000 7fa24a517640 1 -- 172.21.15.16:0/1764324709 wait complete. 2024-09-19T09:19:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.430+0000 7fa24a517640 1 Processor -- start 2024-09-19T09:19:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.430+0000 7fa24a517640 1 -- start start 2024-09-19T09:19:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa24a517640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa244100390 0x7fa244100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa24a517640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa244100cb0 0x7fa2441a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa24a517640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2441a3c70 0x7fa2441a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa24a517640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa24410bc30 con 0x7fa2441a3c70 2024-09-19T09:19:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa24a517640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa24410bab0 con 0x7fa244100cb0 2024-09-19T09:19:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa24a517640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa24410bdb0 con 0x7fa244100390 2024-09-19T09:19:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa248ab4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2441a3c70 0x7fa2441a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa243fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa244100390 0x7fa244100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa248ab4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2441a3c70 0x7fa2441a6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34740/0 (socket says 172.21.15.16:34740) 2024-09-19T09:19:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa243fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa244100390 0x7fa244100770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:47330/0 (socket says 172.21.15.16:47330) 2024-09-19T09:19:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa248ab4640 1 -- 172.21.15.16:0/3531644276 learned_addr learned my addr 172.21.15.16:0/3531644276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:19:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa2437fe640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa244100cb0 0x7fa2441a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa2437fe640 1 -- 172.21.15.16:0/3531644276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa244100cb0 msgr2=0x7fa2441a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T09:19:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa2437fe640 1 -- 172.21.15.16:0/3531644276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa244100cb0 msgr2=0x7fa2441a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T09:19:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa2437fe640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa244100cb0 0x7fa2441a3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T09:19:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa2437fe640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa244100cb0 0x7fa2441a3730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T09:19:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa243fff640 1 -- 172.21.15.16:0/3531644276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa244100cb0 msgr2=0x7fa2441a3730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T09:19:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa243fff640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa244100cb0 0x7fa2441a3730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa243fff640 1 -- 172.21.15.16:0/3531644276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2441a3c70 msgr2=0x7fa2441a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa243fff640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2441a3c70 0x7fa2441a6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa243fff640 1 -- 172.21.15.16:0/3531644276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2441a65d0 con 0x7fa244100390 2024-09-19T09:19:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.431+0000 7fa243fff640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa244100390 0x7fa244100770 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa234008000 tx=0x7fa2340307b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.432+0000 7fa2417fa640 1 -- 172.21.15.16:0/3531644276 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234040070 con 0x7fa244100390 2024-09-19T09:19:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.432+0000 7fa2417fa640 1 -- 172.21.15.16:0/3531644276 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa234004440 con 0x7fa244100390 2024-09-19T09:19:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.432+0000 7fa24a517640 1 -- 172.21.15.16:0/3531644276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2441a6860 con 0x7fa244100390 2024-09-19T09:19:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.432+0000 7fa2417fa640 1 -- 172.21.15.16:0/3531644276 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234031400 con 0x7fa244100390 2024-09-19T09:19:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.432+0000 7fa24a517640 1 -- 172.21.15.16:0/3531644276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2441a6d10 con 0x7fa244100390 2024-09-19T09:19:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.433+0000 7fa2417fa640 1 -- 172.21.15.16:0/3531644276 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fa234043070 con 0x7fa244100390 2024-09-19T09:19:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.433+0000 7fa24a517640 1 -- 172.21.15.16:0/3531644276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa244069f90 con 0x7fa244100390 2024-09-19T09:19:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.433+0000 7fa2417fa640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa22c039730 0x7fa22c03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.433+0000 7fa2417fa640 1 -- 172.21.15.16:0/3531644276 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa23406d7b0 con 0x7fa244100390 2024-09-19T09:19:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.433+0000 7fa2437fe640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa22c039730 0x7fa22c03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.434+0000 7fa2437fe640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa22c039730 0x7fa22c03bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa238006fd0 tx=0x7fa238008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.438+0000 7fa2417fa640 1 -- 172.21.15.16:0/3531644276 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa234038e10 con 0x7fa244100390 2024-09-19T09:19:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.653+0000 7fa24a517640 1 -- 172.21.15.16:0/3531644276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa2441082f0 con 0x7fa244100390 2024-09-19T09:19:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.654+0000 7fa2417fa640 1 -- 172.21.15.16:0/3531644276 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fa23403c820 con 0x7fa244100390 2024-09-19T09:19:30.655 INFO:teuthology.orchestra.run.smithi016.stdout:# minimal ceph.conf for ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:19:30.655 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-19T09:19:30.655 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:19:30.655 INFO:teuthology.orchestra.run.smithi016.stdout: mon_host = [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-09-19T09:19:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.656+0000 7fa24a517640 1 -- 172.21.15.16:0/3531644276 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa22c039730 msgr2=0x7fa22c03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.656+0000 7fa24a517640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa22c039730 0x7fa22c03bbf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa238006fd0 tx=0x7fa238008040 comp rx=0 tx=0).stop 2024-09-19T09:19:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.656+0000 7fa24a517640 1 -- 172.21.15.16:0/3531644276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa244100390 msgr2=0x7fa244100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.656+0000 7fa24a517640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa244100390 0x7fa244100770 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa234008000 tx=0x7fa2340307b0 comp rx=0 tx=0).stop 2024-09-19T09:19:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.656+0000 7fa24a517640 1 -- 172.21.15.16:0/3531644276 shutdown_connections 2024-09-19T09:19:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.656+0000 7fa24a517640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2441a3c70 0x7fa2441a6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.656+0000 7fa24a517640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa244100cb0 0x7fa2441a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.656+0000 7fa24a517640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa22c039730 0x7fa22c03bbf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.656+0000 7fa24a517640 1 --2- 172.21.15.16:0/3531644276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa244100390 0x7fa244100770 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.656+0000 7fa24a517640 1 -- 172.21.15.16:0/3531644276 >> 172.21.15.16:0/3531644276 conn(0x7fa244077310 msgr2=0x7fa2441054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.657+0000 7fa24a517640 1 -- 172.21.15.16:0/3531644276 shutdown_connections 2024-09-19T09:19:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:30.657+0000 7fa24a517640 1 -- 172.21.15.16:0/3531644276 wait complete. 2024-09-19T09:19:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: mon.c calling monitor election 2024-09-19T09:19:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:19:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: mon.a calling monitor election 2024-09-19T09:19:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:19:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T09:19:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: monmap epoch 3 2024-09-19T09:19:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: fsid ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:19:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: last_changed 2024-09-19T09:19:23.745204+0000 2024-09-19T09:19:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: created 2024-09-19T09:17:41.219429+0000 2024-09-19T09:19:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: min_mon_release 19 (squid) 2024-09-19T09:19:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: election_strategy: 1 2024-09-19T09:19:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-19T09:19:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-19T09:19:31.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-09-19T09:19:31.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: fsmap 2024-09-19T09:19:31.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:19:31.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: mgrmap e13: x(active, since 47s) 2024-09-19T09:19:31.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: overall HEALTH_OK 2024-09-19T09:19:31.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:31.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='client.? 172.21.15.142:0/3351153224' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T09:19:31.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:31.385 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T09:19:31.386 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:19:31.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T09:19:31.417 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:19:31.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:19:31.483 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-19T09:19:31.483 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T09:19:31.512 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-19T09:19:31.512 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:19:31.578 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-19T09:19:31.578 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T09:19:31.606 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-19T09:19:31.606 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T09:19:31.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: mon.b calling monitor election 2024-09-19T09:19:31.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-19T09:19:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-19T09:19:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-19T09:19:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: Updating smithi097:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:19:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: Updating smithi142:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:19:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: Updating smithi016:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:19:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: mon.b calling monitor election 2024-09-19T09:19:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: mon.c calling monitor election 2024-09-19T09:19:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: mon.a calling monitor election 2024-09-19T09:19:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T09:19:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: monmap epoch 3 2024-09-19T09:19:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: fsid ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:19:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: last_changed 2024-09-19T09:19:23.745204+0000 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: created 2024-09-19T09:17:41.219429+0000 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: min_mon_release 19 (squid) 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: election_strategy: 1 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: fsmap 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: mgrmap e13: x(active, since 49s) 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: overall HEALTH_OK 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T09:19:31.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:19:31.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:19:31.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:31.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: Reconfiguring daemon mon.a on smithi016 2024-09-19T09:19:31.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.668 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: mon.b calling monitor election 2024-09-19T09:19:31.668 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-19T09:19:31.668 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-19T09:19:31.668 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-19T09:19:31.668 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: Updating smithi097:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:19:31.668 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: Updating smithi142:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:19:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: Updating smithi016:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:19:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: mon.b calling monitor election 2024-09-19T09:19:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: mon.c calling monitor election 2024-09-19T09:19:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: mon.a calling monitor election 2024-09-19T09:19:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T09:19:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: monmap epoch 3 2024-09-19T09:19:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: fsid ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:19:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: last_changed 2024-09-19T09:19:23.745204+0000 2024-09-19T09:19:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: created 2024-09-19T09:17:41.219429+0000 2024-09-19T09:19:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: min_mon_release 19 (squid) 2024-09-19T09:19:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: election_strategy: 1 2024-09-19T09:19:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-19T09:19:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-19T09:19:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-09-19T09:19:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: fsmap 2024-09-19T09:19:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:19:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: mgrmap e13: x(active, since 49s) 2024-09-19T09:19:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: overall HEALTH_OK 2024-09-19T09:19:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T09:19:31.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:19:31.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:19:31.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:31.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: Reconfiguring daemon mon.a on smithi016 2024-09-19T09:19:31.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.673 INFO:tasks.cephadm:Adding mgr.x on smithi016 2024-09-19T09:19:31.673 INFO:tasks.cephadm:Adding mgr.y on smithi097 2024-09-19T09:19:31.674 INFO:tasks.cephadm:Adding mgr.z on smithi142 2024-09-19T09:19:31.674 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi016=x;smithi097=y;smithi142=z' 2024-09-19T09:19:31.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: mon.b calling monitor election 2024-09-19T09:19:31.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-19T09:19:31.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-19T09:19:31.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: Updating smithi142:/etc/ceph/ceph.conf 2024-09-19T09:19:31.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: Updating smithi097:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:19:31.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: Updating smithi142:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:19:31.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: Updating smithi016:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T09:19:31.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: mon.b calling monitor election 2024-09-19T09:19:31.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: mon.c calling monitor election 2024-09-19T09:19:31.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: mon.a calling monitor election 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: monmap epoch 3 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: fsid ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: last_changed 2024-09-19T09:19:23.745204+0000 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: created 2024-09-19T09:17:41.219429+0000 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: min_mon_release 19 (squid) 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: election_strategy: 1 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: 1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.c 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: fsmap 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: mgrmap e13: x(active, since 49s) 2024-09-19T09:19:31.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: overall HEALTH_OK 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:19:31.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:19:31.889 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:31.889 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: Reconfiguring daemon mon.a on smithi016 2024-09-19T09:19:31.889 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:31.965 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:19:32.411 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.391+0000 7f49fcbc2640 1 -- 172.21.15.142:0/1571791937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f8101b70 msgr2=0x7f49f8101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:32.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.391+0000 7f49fcbc2640 1 --2- 172.21.15.142:0/1571791937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f8101b70 0x7f49f8101f70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009880 tx=0x7f49e802a5d0 comp rx=0 tx=0).stop 2024-09-19T09:19:32.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.392+0000 7f49fcbc2640 1 -- 172.21.15.142:0/1571791937 shutdown_connections 2024-09-19T09:19:32.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.392+0000 7f49fcbc2640 1 --2- 172.21.15.142:0/1571791937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f8101b70 0x7f49f8101f70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:32.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.392+0000 7f49fcbc2640 1 -- 172.21.15.142:0/1571791937 >> 172.21.15.142:0/1571791937 conn(0x7f49f80fb510 msgr2=0x7f49f80fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:32.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.392+0000 7f49fcbc2640 1 -- 172.21.15.142:0/1571791937 shutdown_connections 2024-09-19T09:19:32.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.392+0000 7f49fcbc2640 1 -- 172.21.15.142:0/1571791937 wait complete. 2024-09-19T09:19:32.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.392+0000 7f49fcbc2640 1 Processor -- start 2024-09-19T09:19:32.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.392+0000 7f49fcbc2640 1 -- start start 2024-09-19T09:19:32.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.392+0000 7f49fcbc2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49f8101b70 0x7f49f80fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:32.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.392+0000 7f49fcbc2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49f80fbab0 0x7f49f80fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:32.412 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.392+0000 7f49fcbc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f80fc470 0x7f49f81a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:32.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49fcbc2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81066a0 con 0x7f49f80fc470 2024-09-19T09:19:32.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49fcbc2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8106520 con 0x7f49f8101b70 2024-09-19T09:19:32.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49fcbc2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8106820 con 0x7f49f80fbab0 2024-09-19T09:19:32.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49f5d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49f80fbab0 0x7f49f80fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:32.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49f5d9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49f80fbab0 0x7f49f80fbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:47584/0 (socket says 172.21.15.142:47584) 2024-09-19T09:19:32.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49f6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f80fc470 0x7f49f81a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:32.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49f5d9b640 1 -- 172.21.15.142:0/863215114 learned_addr learned my addr 172.21.15.142:0/863215114 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-19T09:19:32.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49f659c640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49f8101b70 0x7f49f80fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:32.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49f6d9d640 1 -- 172.21.15.142:0/863215114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49f80fbab0 msgr2=0x7f49f80fbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:32.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49f6d9d640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49f80fbab0 0x7f49f80fbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:32.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49f6d9d640 1 -- 172.21.15.142:0/863215114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49f8101b70 msgr2=0x7f49f80fd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:32.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49f6d9d640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49f8101b70 0x7f49f80fd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:32.413 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49f6d9d640 1 -- 172.21.15.142:0/863215114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f81a1610 con 0x7f49f80fc470 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49f659c640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49f8101b70 0x7f49f80fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49f6d9d640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f80fc470 0x7f49f81a1080 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f49f000bd50 tx=0x7f49f000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49ef7fe640 1 -- 172.21.15.142:0/863215114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0019070 con 0x7f49f80fc470 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49ef7fe640 1 -- 172.21.15.142:0/863215114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49f0014920 con 0x7f49f80fc470 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49fcbc2640 1 -- 172.21.15.142:0/863215114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f81a1900 con 0x7f49f80fc470 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49ef7fe640 1 -- 172.21.15.142:0/863215114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0012990 con 0x7f49f80fc470 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.393+0000 7f49fcbc2640 1 -- 172.21.15.142:0/863215114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f81a1ec0 con 0x7f49f80fc470 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.394+0000 7f49ef7fe640 1 -- 172.21.15.142:0/863215114 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f49f001c070 con 0x7f49f80fc470 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.395+0000 7f49fcbc2640 1 -- 172.21.15.142:0/863215114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f8190830 con 0x7f49f80fc470 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.395+0000 7f49ef7fe640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f49e4039730 0x7f49e403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.395+0000 7f49ef7fe640 1 -- 172.21.15.142:0/863215114 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f49f004c020 con 0x7f49f80fc470 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.395+0000 7f49f659c640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f49e4039730 0x7f49e403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.395+0000 7f49f659c640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f49e4039730 0x7f49e403bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f49e8000c00 tx=0x7f49e80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:32.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.400+0000 7f49ef7fe640 1 -- 172.21.15.142:0/863215114 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f49f0012bb0 con 0x7f49f80fc470 2024-09-19T09:19:32.583 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.581+0000 7f49fcbc2640 1 -- 172.21.15.142:0/863215114 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi097=y;smithi142=z", "target": ["mon-mgr", ""]}) -- 0x7f49f80626c0 con 0x7f49e4039730 2024-09-19T09:19:32.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.586+0000 7f49ef7fe640 1 -- 172.21.15.142:0/863215114 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f49f80626c0 con 0x7f49e4039730 2024-09-19T09:19:32.588 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mgr update... 2024-09-19T09:19:32.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.589+0000 7f49fcbc2640 1 -- 172.21.15.142:0/863215114 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f49e4039730 msgr2=0x7f49e403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:32.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.589+0000 7f49fcbc2640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f49e4039730 0x7f49e403bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f49e8000c00 tx=0x7f49e80315b0 comp rx=0 tx=0).stop 2024-09-19T09:19:32.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.589+0000 7f49fcbc2640 1 -- 172.21.15.142:0/863215114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f80fc470 msgr2=0x7f49f81a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:32.590 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.589+0000 7f49fcbc2640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f80fc470 0x7f49f81a1080 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f49f000bd50 tx=0x7f49f000be40 comp rx=0 tx=0).stop 2024-09-19T09:19:32.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.589+0000 7f49fcbc2640 1 -- 172.21.15.142:0/863215114 shutdown_connections 2024-09-19T09:19:32.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.589+0000 7f49fcbc2640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f49e4039730 0x7f49e403bbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:32.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.589+0000 7f49fcbc2640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49f80fbab0 0x7f49f80fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:32.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.589+0000 7f49fcbc2640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f80fc470 0x7f49f81a1080 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:32.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.589+0000 7f49fcbc2640 1 --2- 172.21.15.142:0/863215114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49f8101b70 0x7f49f80fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:32.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.589+0000 7f49fcbc2640 1 -- 172.21.15.142:0/863215114 >> 172.21.15.142:0/863215114 conn(0x7f49f80fb510 msgr2=0x7f49f80ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:32.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.589+0000 7f49fcbc2640 1 -- 172.21.15.142:0/863215114 shutdown_connections 2024-09-19T09:19:32.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:19:32.589+0000 7f49fcbc2640 1 -- 172.21.15.142:0/863215114 wait complete. 2024-09-19T09:19:32.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:32 smithi016 ceph-mon[30979]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:32.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:32 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:32.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:32 smithi142 ceph-mon[35032]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:32.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:32 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:32.885 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:32 smithi097 ceph-mon[34864]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:32.885 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:32 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:19:33.003 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:19:32 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:19:32.745+0000 7f44a8f5e640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-19T09:19:33.149 DEBUG:teuthology.orchestra.run.smithi097:mgr.y> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T09:19:33.153 DEBUG:teuthology.orchestra.run.smithi142:mgr.z> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T09:19:33.158 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T09:19:33.158 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:19:33.158 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T09:19:33.179 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T09:19:33.179 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-09-19T09:19:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T09:19:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:19:33.240 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-09-19T09:19:33.240 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:19:33.240 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:19:33.240 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-19 09:18:50.253983662 +0000 2024-09-19T09:19:33.240 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-19 09:14:50.065024035 +0000 2024-09-19T09:19:33.240 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-19 09:14:50.065024035 +0000 2024-09-19T09:19:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-19 09:14:50.065024035 +0000 2024-09-19T09:19:33.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T09:19:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-19T09:19:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-19T09:19:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000111858 s, 4.6 MB/s 2024-09-19T09:19:33.313 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T09:19:33.373 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-09-19T09:19:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T09:19:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:19:33.434 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 679 Links: 1 2024-09-19T09:19:33.434 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:19:33.434 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:19:33.434 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-19 09:18:50.282983189 +0000 2024-09-19T09:19:33.434 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-19 09:14:50.401018273 +0000 2024-09-19T09:19:33.434 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-19 09:14:50.401018273 +0000 2024-09-19T09:19:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-19 09:14:50.401018273 +0000 2024-09-19T09:19:33.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T09:19:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-19T09:19:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-19T09:19:33.502 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000107849 s, 4.7 MB/s 2024-09-19T09:19:33.503 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T09:19:33.562 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-09-19T09:19:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T09:19:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:19:33.620 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-09-19T09:19:33.620 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:19:33.620 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:19:33.620 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-19 09:18:50.309982748 +0000 2024-09-19T09:19:33.620 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-19 09:14:50.727012683 +0000 2024-09-19T09:19:33.620 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-19 09:14:50.727012683 +0000 2024-09-19T09:19:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-19 09:14:50.727012683 +0000 2024-09-19T09:19:33.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T09:19:33.682 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:33 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:33 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:33 smithi016 ceph-mon[30979]: Reconfiguring mon.b (monmap changed)... 2024-09-19T09:19:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:33 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:19:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:33 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:19:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:33 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:33 smithi016 ceph-mon[30979]: Reconfiguring daemon mon.b on smithi097 2024-09-19T09:19:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:33 smithi016 ceph-mon[30979]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi097=y;smithi142=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:19:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:33 smithi016 ceph-mon[30979]: Saving service mgr spec with placement smithi016=x;smithi097=y;smithi142=z;count:3 2024-09-19T09:19:33.683 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:33 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:33.684 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-19T09:19:33.684 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-19T09:19:33.685 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000112478 s, 4.6 MB/s 2024-09-19T09:19:33.686 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T09:19:33.744 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-09-19T09:19:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T09:19:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:19:33.802 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 716 Links: 1 2024-09-19T09:19:33.802 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:19:33.802 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:19:33.802 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-19 09:18:50.337982290 +0000 2024-09-19T09:19:33.802 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-19 09:14:51.066006869 +0000 2024-09-19T09:19:33.802 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-19 09:14:51.066006869 +0000 2024-09-19T09:19:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-19 09:14:51.066006869 +0000 2024-09-19T09:19:33.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T09:19:33.807 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:33 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:33.807 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:33 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:33.807 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:33 smithi097 ceph-mon[34864]: Reconfiguring mon.b (monmap changed)... 2024-09-19T09:19:33.807 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:33 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:19:33.807 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:33 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:19:33.807 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:33 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:33.807 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:33 smithi097 ceph-mon[34864]: Reconfiguring daemon mon.b on smithi097 2024-09-19T09:19:33.808 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:33 smithi097 ceph-mon[34864]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi097=y;smithi142=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:19:33.808 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:33 smithi097 ceph-mon[34864]: Saving service mgr spec with placement smithi016=x;smithi097=y;smithi142=z;count:3 2024-09-19T09:19:33.808 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:33 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-19T09:19:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-19T09:19:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000110299 s, 4.6 MB/s 2024-09-19T09:19:33.870 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T09:19:33.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:33 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:33.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:33 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:33.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:33 smithi142 ceph-mon[35032]: Reconfiguring mon.b (monmap changed)... 2024-09-19T09:19:33.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:33 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:19:33.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:33 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:19:33.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:33 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:33.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:33 smithi142 ceph-mon[35032]: Reconfiguring daemon mon.b on smithi097 2024-09-19T09:19:33.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:33 smithi142 ceph-mon[35032]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi097=y;smithi142=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:19:33.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:33 smithi142 ceph-mon[35032]: Saving service mgr spec with placement smithi016=x;smithi097=y;smithi142=z;count:3 2024-09-19T09:19:33.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:33 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:33.928 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-19T09:19:33.928 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T09:19:33.944 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T09:19:33.944 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-09-19T09:19:34.001 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T09:19:34.001 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:19:34.001 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-19T09:19:34.001 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:19:34.001 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:19:34.001 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-19 09:19:23.214774584 +0000 2024-09-19T09:19:34.001 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-19 09:14:57.538245608 +0000 2024-09-19T09:19:34.001 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-19 09:14:57.538245608 +0000 2024-09-19T09:19:34.001 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-19 09:14:57.538245608 +0000 2024-09-19T09:19:34.002 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T09:19:34.065 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-19T09:19:34.065 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-19T09:19:34.065 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000272198 s, 1.9 MB/s 2024-09-19T09:19:34.066 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T09:19:34.124 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-09-19T09:19:34.181 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T09:19:34.181 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:19:34.181 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-19T09:19:34.181 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:19:34.181 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:19:34.181 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-19 09:19:23.215774567 +0000 2024-09-19T09:19:34.181 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-19 09:14:57.855240526 +0000 2024-09-19T09:19:34.181 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-19 09:14:57.855240526 +0000 2024-09-19T09:19:34.181 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-19 09:14:57.855240526 +0000 2024-09-19T09:19:34.182 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T09:19:34.246 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-19T09:19:34.246 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-19T09:19:34.246 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.00021731 s, 2.4 MB/s 2024-09-19T09:19:34.247 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T09:19:34.304 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-09-19T09:19:34.360 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T09:19:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:19:34.361 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-19T09:19:34.361 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:19:34.361 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:19:34.361 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-19 09:19:23.215774567 +0000 2024-09-19T09:19:34.361 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-19 09:14:58.185235235 +0000 2024-09-19T09:19:34.361 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-19 09:14:58.185235235 +0000 2024-09-19T09:19:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-19 09:14:58.185235235 +0000 2024-09-19T09:19:34.362 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T09:19:34.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-19T09:19:34.424 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-19T09:19:34.424 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.0002058 s, 2.5 MB/s 2024-09-19T09:19:34.425 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T09:19:34.482 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-09-19T09:19:34.538 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T09:19:34.538 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:19:34.538 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-19T09:19:34.538 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:19:34.538 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:19:34.538 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-19 09:19:23.215774567 +0000 2024-09-19T09:19:34.538 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-19 09:14:58.515229945 +0000 2024-09-19T09:19:34.538 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-19 09:14:58.515229945 +0000 2024-09-19T09:19:34.539 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-19 09:14:58.515229945 +0000 2024-09-19T09:19:34.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T09:19:34.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-19T09:19:34.602 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-19T09:19:34.602 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000223331 s, 2.3 MB/s 2024-09-19T09:19:34.603 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T09:19:34.660 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-19T09:19:34.660 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T09:19:34.676 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T09:19:34.677 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-09-19T09:19:34.732 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T09:19:34.732 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:19:34.732 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T09:19:34.732 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:19:34.732 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:19:34.732 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-19 09:19:14.553472703 +0000 2024-09-19T09:19:34.732 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-19 09:14:59.148621956 +0000 2024-09-19T09:19:34.732 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-19 09:14:59.148621956 +0000 2024-09-19T09:19:34.733 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-19 09:14:59.148621956 +0000 2024-09-19T09:19:34.733 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T09:19:34.797 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-19T09:19:34.797 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-19T09:19:34.797 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000273202 s, 1.9 MB/s 2024-09-19T09:19:34.798 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T09:19:34.854 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-09-19T09:19:34.914 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T09:19:34.914 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:19:34.914 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-19T09:19:34.914 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:19:34.914 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:19:34.914 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-19 09:19:14.553472703 +0000 2024-09-19T09:19:34.914 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-19 09:14:59.489621574 +0000 2024-09-19T09:19:34.914 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-19 09:14:59.489621574 +0000 2024-09-19T09:19:34.915 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-19 09:14:59.489621574 +0000 2024-09-19T09:19:34.915 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T09:19:34.978 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-19T09:19:34.978 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-19T09:19:34.978 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.00021446 s, 2.4 MB/s 2024-09-19T09:19:34.979 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T09:19:35.036 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-09-19T09:19:35.091 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T09:19:35.091 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:19:35.091 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-19T09:19:35.091 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:19:35.092 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:19:35.092 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-19 09:19:14.554472701 +0000 2024-09-19T09:19:35.092 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-19 09:14:59.825621199 +0000 2024-09-19T09:19:35.092 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-19 09:14:59.825621199 +0000 2024-09-19T09:19:35.092 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-19 09:14:59.825621199 +0000 2024-09-19T09:19:35.092 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T09:19:35.120 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:34 smithi142 ceph-mon[35032]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:35.120 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:34 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:35.120 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:34 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:35.121 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:34 smithi142 ceph-mon[35032]: Reconfiguring mon.c (monmap changed)... 2024-09-19T09:19:35.121 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:34 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:19:35.121 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:34 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:19:35.121 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:34 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:35.121 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:34 smithi142 ceph-mon[35032]: Reconfiguring daemon mon.c on smithi142 2024-09-19T09:19:35.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:34 smithi097 ceph-mon[34864]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:35.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:34 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:35.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:34 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:35.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:34 smithi097 ceph-mon[34864]: Reconfiguring mon.c (monmap changed)... 2024-09-19T09:19:35.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:34 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:19:35.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:34 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:19:35.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:34 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:35.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:34 smithi097 ceph-mon[34864]: Reconfiguring daemon mon.c on smithi142 2024-09-19T09:19:35.145 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-19T09:19:35.145 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-19T09:19:35.145 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000222224 s, 2.3 MB/s 2024-09-19T09:19:35.146 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T09:19:35.203 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-09-19T09:19:35.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:34 smithi016 ceph-mon[30979]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:35.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:34 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:35.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:34 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:35.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:34 smithi016 ceph-mon[30979]: Reconfiguring mon.c (monmap changed)... 2024-09-19T09:19:35.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:34 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T09:19:35.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:34 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T09:19:35.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:34 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:35.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:34 smithi016 ceph-mon[30979]: Reconfiguring daemon mon.c on smithi142 2024-09-19T09:19:35.259 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T09:19:35.259 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T09:19:35.259 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-19T09:19:35.259 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:19:35.260 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T09:19:35.260 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-19 09:19:14.554472701 +0000 2024-09-19T09:19:35.260 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-19 09:15:00.164620820 +0000 2024-09-19T09:19:35.260 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-19 09:15:00.164620820 +0000 2024-09-19T09:19:35.260 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-09-19 09:15:00.164620820 +0000 2024-09-19T09:19:35.260 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T09:19:35.323 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-19T09:19:35.324 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-19T09:19:35.324 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000230614 s, 2.2 MB/s 2024-09-19T09:19:35.325 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T09:19:35.381 INFO:tasks.cephadm:Deploying osd.0 on smithi016 with /dev/vg_nvme/lv_4... 2024-09-19T09:19:35.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T09:19:35.644 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:36.426 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:36.427 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:36.427 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:36.427 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:36.427 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:36.427 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:36.427 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:19:36.427 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T09:19:36.427 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:19:36.427 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:36.427 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:36 smithi142 ceph-mon[35032]: Deploying daemon mgr.z on smithi142 2024-09-19T09:19:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:19:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T09:19:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:19:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:36 smithi016 ceph-mon[30979]: Deploying daemon mgr.z on smithi142 2024-09-19T09:19:36.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:36.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:36.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:36.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:36.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:36.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:36.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:19:36.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T09:19:36.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:19:36.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:36.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:36 smithi097 ceph-mon[34864]: Deploying daemon mgr.z on smithi142 2024-09-19T09:19:37.128 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:36 smithi142 systemd[1]: Starting Ceph mgr.z for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:19:37.242 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:37 smithi016 ceph-mon[30979]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:37.447 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:37 smithi142 ceph-mon[35032]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:37.447 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:37 smithi142 podman[36830]: 2024-09-19 09:19:37.059446261 +0000 UTC m=+0.015485896 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:19:37.448 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:37 smithi142 podman[36830]: 2024-09-19 09:19:37.178463906 +0000 UTC m=+0.134503527 container create 65283b2503fb1673002141afd2c695bb896badf8d4574630ce0f8ab74c61571e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T09:19:37.448 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:37 smithi142 podman[36830]: 2024-09-19 09:19:37.372555972 +0000 UTC m=+0.328595592 container init 65283b2503fb1673002141afd2c695bb896badf8d4574630ce0f8ab74c61571e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T09:19:37.448 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:37 smithi142 podman[36830]: 2024-09-19 09:19:37.375207058 +0000 UTC m=+0.331246682 container start 65283b2503fb1673002141afd2c695bb896badf8d4574630ce0f8ab74c61571e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:19:37.448 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:37 smithi142 ceph-mgr[36849]: -- 172.21.15.142:0/957753124 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2af4bf0e0 con 0x55b2af49d000 2024-09-19T09:19:37.448 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:37 smithi142 bash[36830]: 65283b2503fb1673002141afd2c695bb896badf8d4574630ce0f8ab74c61571e 2024-09-19T09:19:37.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:37 smithi097 ceph-mon[34864]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:37.707 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:37 smithi142 systemd[1]: Started Ceph mgr.z for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:19:37.707 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:37 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:37.497+0000 7f171fd75040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T09:19:37.708 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:37 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:37.560+0000 7f171fd75040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T09:19:37.708 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:37 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:37.706+0000 7f171fd75040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T09:19:38.060 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:37 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:37.772+0000 7f171fd75040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T09:19:38.061 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:37 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:37.918+0000 7f171fd75040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:19:38.061 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:38 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:38.057+0000 7f171fd75040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T09:19:38.091 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:19:38.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-09-19T09:19:38.372 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:19:38.628 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:38 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:38.372+0000 7f171fd75040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T09:19:38.628 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:38 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:38.432+0000 7f171fd75040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T09:19:38.628 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:38 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:38.618+0000 7f171fd75040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T09:19:38.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:38 smithi142 ceph-mon[35032]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:19:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T09:19:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:19:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:38 smithi142 ceph-mon[35032]: Deploying daemon mgr.y on smithi097 2024-09-19T09:19:38.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:38 smithi016 ceph-mon[30979]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:38.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:38.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:38.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:38.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:19:38.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T09:19:38.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:19:38.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:38.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:38 smithi016 ceph-mon[30979]: Deploying daemon mgr.y on smithi097 2024-09-19T09:19:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.818+0000 7f7001e9b640 1 -- 172.21.15.16:0/1088242110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc104fd0 msgr2=0x7f6ffc1073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.818+0000 7f7001e9b640 1 --2- 172.21.15.16:0/1088242110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc104fd0 0x7f6ffc1073c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009900 tx=0x7f6ff002a410 comp rx=0 tx=0).stop 2024-09-19T09:19:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.818+0000 7f7001e9b640 1 -- 172.21.15.16:0/1088242110 shutdown_connections 2024-09-19T09:19:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.818+0000 7f7001e9b640 1 --2- 172.21.15.16:0/1088242110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ffc107900 0x7f6ffc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.818+0000 7f7001e9b640 1 --2- 172.21.15.16:0/1088242110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc104fd0 0x7f6ffc1073c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.818+0000 7f7001e9b640 1 --2- 172.21.15.16:0/1088242110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ffc1026a0 0x7f6ffc104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.818+0000 7f7001e9b640 1 -- 172.21.15.16:0/1088242110 >> 172.21.15.16:0/1088242110 conn(0x7f6ffc0fc2a0 msgr2=0x7f6ffc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:19:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.818+0000 7f7001e9b640 1 -- 172.21.15.16:0/1088242110 shutdown_connections 2024-09-19T09:19:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.818+0000 7f7001e9b640 1 -- 172.21.15.16:0/1088242110 wait complete. 2024-09-19T09:19:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.818+0000 7f7001e9b640 1 Processor -- start 2024-09-19T09:19:38.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f7001e9b640 1 -- start start 2024-09-19T09:19:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f7001e9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ffc1026a0 0x7f6ffc197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f7001e9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc104fd0 0x7f6ffc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f7001e9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ffc107900 0x7f6ffc19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f7001e9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc10d8d0 con 0x7f6ffc104fd0 2024-09-19T09:19:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f7001e9b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc10d750 con 0x7f6ffc1026a0 2024-09-19T09:19:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f7001e9b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc10da50 con 0x7f6ffc107900 2024-09-19T09:19:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f6ffaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc104fd0 0x7f6ffc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f6ffaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc104fd0 0x7f6ffc198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50370/0 (socket says 172.21.15.16:50370) 2024-09-19T09:19:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f6ffaffd640 1 -- 172.21.15.16:0/3000351351 learned_addr learned my addr 172.21.15.16:0/3000351351 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:19:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f6ffbfff640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ffc107900 0x7f6ffc19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f6ffb7fe640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ffc1026a0 0x7f6ffc197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f6ffaffd640 1 -- 172.21.15.16:0/3000351351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ffc107900 msgr2=0x7f6ffc19f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f6ffaffd640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ffc107900 0x7f6ffc19f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f6ffaffd640 1 -- 172.21.15.16:0/3000351351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ffc1026a0 msgr2=0x7f6ffc197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:19:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f6ffaffd640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ffc1026a0 0x7f6ffc197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:19:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.819+0000 7f6ffaffd640 1 -- 172.21.15.16:0/3000351351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc19fb60 con 0x7f6ffc104fd0 2024-09-19T09:19:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.820+0000 7f6ffaffd640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc104fd0 0x7f6ffc198470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0031440 tx=0x7f6ff0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.820+0000 7f6ff8ff9640 1 -- 172.21.15.16:0/3000351351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff003f030 con 0x7f6ffc104fd0 2024-09-19T09:19:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.820+0000 7f6ff8ff9640 1 -- 172.21.15.16:0/3000351351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff0003ce0 con 0x7f6ffc104fd0 2024-09-19T09:19:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.820+0000 7f7001e9b640 1 -- 172.21.15.16:0/3000351351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ffc19fdf0 con 0x7f6ffc104fd0 2024-09-19T09:19:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.820+0000 7f6ff8ff9640 1 -- 172.21.15.16:0/3000351351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0035070 con 0x7f6ffc104fd0 2024-09-19T09:19:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.820+0000 7f7001e9b640 1 -- 172.21.15.16:0/3000351351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ffc1a0270 con 0x7f6ffc104fd0 2024-09-19T09:19:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.821+0000 7f6ff8ff9640 1 -- 172.21.15.16:0/3000351351 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f6ff0042030 con 0x7f6ffc104fd0 2024-09-19T09:19:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.821+0000 7f7001e9b640 1 -- 172.21.15.16:0/3000351351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ffc0fdd90 con 0x7f6ffc104fd0 2024-09-19T09:19:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.821+0000 7f6ff8ff9640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6fe4039500 0x7f6fe403b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:19:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.821+0000 7f6ff8ff9640 1 -- 172.21.15.16:0/3000351351 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6ff006cb70 con 0x7f6ffc104fd0 2024-09-19T09:19:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.821+0000 7f6ffb7fe640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6fe4039500 0x7f6fe403b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:19:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.821+0000 7f6ffb7fe640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6fe4039500 0x7f6fe403b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0095f0 tx=0x7f6fec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:19:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:38.826+0000 7f6ff8ff9640 1 -- 172.21.15.16:0/3000351351 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6ff00439a0 con 0x7f6ffc104fd0 2024-09-19T09:19:38.855 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:38 smithi097 ceph-mon[34864]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:38.855 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:38.855 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:38.855 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:38.855 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:19:38.855 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T09:19:38.855 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:19:38.855 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:38.855 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:38 smithi097 ceph-mon[34864]: Deploying daemon mgr.y on smithi097 2024-09-19T09:19:39.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:39.010+0000 7f7001e9b640 1 -- 172.21.15.16:0/3000351351 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6ffc062630 con 0x7f6fe4039500 2024-09-19T09:19:39.386 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:39 smithi097 systemd[1]: Starting Ceph mgr.y for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:19:39.727 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:39 smithi016 ceph-mon[30979]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:19:39.727 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:19:39.727 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:19:39.727 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:39.786 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:39 smithi097 ceph-mon[34864]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:19:39.786 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:19:39.786 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:19:39.786 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:39.787 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:39 smithi097 podman[36214]: 2024-09-19 09:19:39.515054571 +0000 UTC m=+0.112168779 container create ec045325526f007246863185e988d6f009c6d770a6132c11d03b3f0dacc08769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:19:39.787 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:39 smithi097 podman[36214]: 2024-09-19 09:19:39.41824216 +0000 UTC m=+0.015356369 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:19:39.787 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:39 smithi097 podman[36214]: 2024-09-19 09:19:39.75913552 +0000 UTC m=+0.356249729 container init ec045325526f007246863185e988d6f009c6d770a6132c11d03b3f0dacc08769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T09:19:39.787 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:39 smithi097 podman[36214]: 2024-09-19 09:19:39.761874379 +0000 UTC m=+0.358988588 container start ec045325526f007246863185e988d6f009c6d770a6132c11d03b3f0dacc08769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:19:39.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:39 smithi142 ceph-mon[35032]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:19:39.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:19:39.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:19:39.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:39.879 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:39 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:39.494+0000 7f171fd75040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T09:19:39.879 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:39 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:39.564+0000 7f171fd75040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T09:19:40.098 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:39 smithi097 ceph-mgr[36233]: -- 172.21.15.97:0/2965000697 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5642874f70e0 con 0x5642874d5000 2024-09-19T09:19:40.099 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:39 smithi097 bash[36214]: ec045325526f007246863185e988d6f009c6d770a6132c11d03b3f0dacc08769 2024-09-19T09:19:40.099 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:39 smithi097 systemd[1]: Started Ceph mgr.y for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:19:40.099 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:39 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:39.883+0000 7fadb12bc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T09:19:40.099 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:39 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:39.942+0000 7fadb12bc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T09:19:40.099 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:40 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:40.097+0000 7fadb12bc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T09:19:40.385 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:40 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:40.166+0000 7fadb12bc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T09:19:40.385 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:40 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:40.331+0000 7fadb12bc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:19:40.570 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:40.318+0000 7f171fd75040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T09:19:40.570 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:40.444+0000 7f171fd75040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T09:19:40.803 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:40 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:40.479+0000 7fadb12bc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T09:19:40.804 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:40 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:40.802+0000 7fadb12bc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T09:19:40.878 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:40.589+0000 7f171fd75040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:19:40.878 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:40.656+0000 7f171fd75040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T09:19:40.878 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:40.810+0000 7f171fd75040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T09:19:40.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:40 smithi016 ceph-mon[30979]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:40.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:40 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:40.903 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:40 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:40.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:40 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:41.060 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:40 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:40.863+0000 7fadb12bc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T09:19:41.060 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:40 smithi097 ceph-mon[34864]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:41.060 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:40 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:41.060 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:40 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:41.060 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:40 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:41.060 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:40 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:41.061 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:40 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:41.184 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:40.879+0000 7f171fd75040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T09:19:41.184 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:40.942+0000 7f171fd75040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T09:19:41.184 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:41.113+0000 7f171fd75040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T09:19:41.184 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:41.182+0000 7f171fd75040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T09:19:41.184 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:40 smithi142 ceph-mon[35032]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:41.184 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:40 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:41.184 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:40 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:41.184 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:40 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:41.184 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:40 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:41.185 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:40 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:41.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:40 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:41.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:40 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:41.385 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:41.059+0000 7fadb12bc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T09:19:41.450 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:41.249+0000 7f171fd75040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T09:19:41.450 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:41.332+0000 7f171fd75040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T09:19:41.450 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:41.390+0000 7f171fd75040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T09:19:41.450 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:41.449+0000 7f171fd75040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T09:19:42.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:42.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:42.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T09:19:42.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:19:42.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:19:42.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:42.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: Reconfiguring daemon mgr.x on smithi016 2024-09-19T09:19:42.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3027090496' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbf3b28b-2282-4057-9e65-19ceb78ff19f"}]: dispatch 2024-09-19T09:19:42.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3027090496' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbf3b28b-2282-4057-9e65-19ceb78ff19f"}]': finished 2024-09-19T09:19:42.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T09:19:42.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:19:42.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:41 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2101493989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:19:42.378 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:19:42 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z[36845]: 2024-09-19T09:19:42.232+0000 7f171fd75040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T09:19:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:42.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T09:19:42.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:19:42.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:19:42.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:42.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: Reconfiguring daemon mgr.x on smithi016 2024-09-19T09:19:42.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3027090496' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbf3b28b-2282-4057-9e65-19ceb78ff19f"}]: dispatch 2024-09-19T09:19:42.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3027090496' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbf3b28b-2282-4057-9e65-19ceb78ff19f"}]': finished 2024-09-19T09:19:42.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T09:19:42.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:19:42.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:41 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2101493989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:19:42.385 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:41.939+0000 7fadb12bc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T09:19:42.386 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:42 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:42.007+0000 7fadb12bc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T09:19:42.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:42.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:42.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T09:19:42.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T09:19:42.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T09:19:42.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:42.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: Reconfiguring daemon mgr.x on smithi016 2024-09-19T09:19:42.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:42.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3027090496' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbf3b28b-2282-4057-9e65-19ceb78ff19f"}]: dispatch 2024-09-19T09:19:42.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3027090496' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbf3b28b-2282-4057-9e65-19ceb78ff19f"}]': finished 2024-09-19T09:19:42.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T09:19:42.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:19:42.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:41 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2101493989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:19:42.978 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:42 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:42.718+0000 7fadb12bc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T09:19:42.978 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:42 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:42.834+0000 7fadb12bc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T09:19:42.979 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:42 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:42.976+0000 7fadb12bc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T09:19:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:43.074+0000 7f6ff8ff9640 1 -- 172.21.15.16:0/3000351351 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7f6ff003b460 con 0x7f6ffc104fd0 2024-09-19T09:19:43.255 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:43.037+0000 7fadb12bc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T09:19:43.255 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:43.186+0000 7fadb12bc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T09:19:43.255 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:43.254+0000 7fadb12bc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T09:19:43.255 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-mon[34864]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:43.256 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-mon[34864]: Standby manager daemon z started 2024-09-19T09:19:43.256 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-mon[34864]: from='mgr.? 172.21.15.142:0/568750531' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T09:19:43.256 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-mon[34864]: from='mgr.? 172.21.15.142:0/568750531' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T09:19:43.256 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-mon[34864]: from='mgr.? 172.21.15.142:0/568750531' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T09:19:43.256 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-mon[34864]: from='mgr.? 172.21.15.142:0/568750531' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T09:19:43.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:43 smithi142 ceph-mon[35032]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:43.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:43 smithi142 ceph-mon[35032]: Standby manager daemon z started 2024-09-19T09:19:43.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:43 smithi142 ceph-mon[35032]: from='mgr.? 172.21.15.142:0/568750531' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T09:19:43.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:43 smithi142 ceph-mon[35032]: from='mgr.? 172.21.15.142:0/568750531' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T09:19:43.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:43 smithi142 ceph-mon[35032]: from='mgr.? 172.21.15.142:0/568750531' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T09:19:43.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:43 smithi142 ceph-mon[35032]: from='mgr.? 172.21.15.142:0/568750531' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T09:19:43.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:43 smithi016 ceph-mon[30979]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:43.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:43 smithi016 ceph-mon[30979]: Standby manager daemon z started 2024-09-19T09:19:43.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:43 smithi016 ceph-mon[30979]: from='mgr.? 172.21.15.142:0/568750531' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T09:19:43.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:43 smithi016 ceph-mon[30979]: from='mgr.? 172.21.15.142:0/568750531' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T09:19:43.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:43 smithi016 ceph-mon[30979]: from='mgr.? 172.21.15.142:0/568750531' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T09:19:43.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:43 smithi016 ceph-mon[30979]: from='mgr.? 172.21.15.142:0/568750531' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T09:19:43.535 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:43.314+0000 7fadb12bc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T09:19:43.535 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:43.469+0000 7fadb12bc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T09:19:43.535 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:43.534+0000 7fadb12bc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T09:19:43.801 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:43.601+0000 7fadb12bc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T09:19:43.801 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:43.681+0000 7fadb12bc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T09:19:43.801 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:43.739+0000 7fadb12bc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T09:19:43.801 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:43.797+0000 7fadb12bc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T09:19:44.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:44 smithi142 ceph-mon[35032]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T09:19:44.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:44 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T09:19:44.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:44 smithi097 ceph-mon[34864]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T09:19:44.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:44 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T09:19:44.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:44 smithi016 ceph-mon[30979]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T09:19:44.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:44 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T09:19:44.885 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:19:44 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y[36229]: 2024-09-19T09:19:44.580+0000 7fadb12bc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T09:19:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:19:45.087+0000 7f6ff8ff9640 1 -- 172.21.15.16:0/3000351351 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f6ff0030c30 con 0x7f6ffc104fd0 2024-09-19T09:19:45.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:45 smithi142 ceph-mon[35032]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:45.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:45 smithi142 ceph-mon[35032]: Standby manager daemon y started 2024-09-19T09:19:45.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:45 smithi142 ceph-mon[35032]: from='mgr.? 172.21.15.97:0/3647215624' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T09:19:45.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:45 smithi142 ceph-mon[35032]: from='mgr.? 172.21.15.97:0/3647215624' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T09:19:45.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:45 smithi142 ceph-mon[35032]: from='mgr.? 172.21.15.97:0/3647215624' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T09:19:45.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:45 smithi142 ceph-mon[35032]: from='mgr.? 172.21.15.97:0/3647215624' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T09:19:45.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:45 smithi097 ceph-mon[34864]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:45.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:45 smithi097 ceph-mon[34864]: Standby manager daemon y started 2024-09-19T09:19:45.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:45 smithi097 ceph-mon[34864]: from='mgr.? 172.21.15.97:0/3647215624' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T09:19:45.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:45 smithi097 ceph-mon[34864]: from='mgr.? 172.21.15.97:0/3647215624' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T09:19:45.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:45 smithi097 ceph-mon[34864]: from='mgr.? 172.21.15.97:0/3647215624' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T09:19:45.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:45 smithi097 ceph-mon[34864]: from='mgr.? 172.21.15.97:0/3647215624' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T09:19:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:45 smithi016 ceph-mon[30979]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:45 smithi016 ceph-mon[30979]: Standby manager daemon y started 2024-09-19T09:19:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:45 smithi016 ceph-mon[30979]: from='mgr.? 172.21.15.97:0/3647215624' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T09:19:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:45 smithi016 ceph-mon[30979]: from='mgr.? 172.21.15.97:0/3647215624' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T09:19:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:45 smithi016 ceph-mon[30979]: from='mgr.? 172.21.15.97:0/3647215624' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T09:19:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:45 smithi016 ceph-mon[30979]: from='mgr.? 172.21.15.97:0/3647215624' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T09:19:46.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:46 smithi142 ceph-mon[35032]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-19T09:19:46.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T09:19:46.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:46 smithi097 ceph-mon[34864]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-19T09:19:46.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T09:19:46.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:46 smithi016 ceph-mon[30979]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-19T09:19:46.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T09:19:47.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:47 smithi016 ceph-mon[30979]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:47.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:47 smithi142 ceph-mon[35032]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:47.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:47 smithi097 ceph-mon[34864]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:49.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:49 smithi142 ceph-mon[35032]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:49.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:49 smithi097 ceph-mon[34864]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:49.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:49 smithi016 ceph-mon[30979]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:50.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:50.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:50.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:50.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:50.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:50.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:50.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:50.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:50.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:50.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:50.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:50.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:50.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:50.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:50.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:50.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:50.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:19:50.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:51.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:51 smithi016 ceph-mon[30979]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:51.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:51 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T09:19:51.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:51 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:51.506 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:51 smithi097 ceph-mon[34864]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:51.506 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:51 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T09:19:51.506 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:51 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:51.548 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:51 smithi142 ceph-mon[35032]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:51.549 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:51 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T09:19:51.549 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:51 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:19:52.440 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:52 smithi016 ceph-mon[30979]: Deploying daemon osd.0 on smithi016 2024-09-19T09:19:52.495 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:52 smithi142 ceph-mon[35032]: Deploying daemon osd.0 on smithi016 2024-09-19T09:19:52.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:52 smithi097 ceph-mon[34864]: Deploying daemon osd.0 on smithi016 2024-09-19T09:19:53.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:53 smithi016 ceph-mon[30979]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:53.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:53 smithi142 ceph-mon[35032]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:53.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:53 smithi097 ceph-mon[34864]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:55.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:55 smithi016 ceph-mon[30979]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:55.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:55 smithi142 ceph-mon[35032]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:55.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:55 smithi097 ceph-mon[34864]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:57.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:57 smithi016 ceph-mon[30979]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:57.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:57 smithi142 ceph-mon[35032]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:57.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:57 smithi097 ceph-mon[34864]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:58.444 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:58 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:58.444 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:58 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:58.445 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:58 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:58.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:58 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:58.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:58 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:58.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:58 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:58.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:58 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:19:58.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:58 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:58.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:58 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:19:59.482 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:19:59 smithi016 ceph-mon[30979]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:59.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:19:59 smithi142 ceph-mon[35032]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:19:59.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:19:59 smithi097 ceph-mon[34864]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:20:00.450 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:00 smithi016 ceph-mon[30979]: overall HEALTH_OK 2024-09-19T09:20:00.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:00 smithi142 ceph-mon[35032]: overall HEALTH_OK 2024-09-19T09:20:00.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:00 smithi097 ceph-mon[34864]: overall HEALTH_OK 2024-09-19T09:20:01.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:01 smithi097 ceph-mon[34864]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:20:01.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:01.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:01.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:01.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:01.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:01.474 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:01 smithi016 ceph-mon[30979]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:20:01.474 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:01.474 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:01.475 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:01.475 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:01.475 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:01.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:01 smithi142 ceph-mon[35032]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:20:01.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:01.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:01.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:01.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:01.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:01.882 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 0 on host 'smithi016' 2024-09-19T09:20:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:01.880+0000 7f6ff8ff9640 1 -- 172.21.15.16:0/3000351351 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6ffc062630 con 0x7f6fe4039500 2024-09-19T09:20:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:01.883+0000 7f7001e9b640 1 -- 172.21.15.16:0/3000351351 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6fe4039500 msgr2=0x7f6fe403b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:01.883+0000 7f7001e9b640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6fe4039500 0x7f6fe403b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0095f0 tx=0x7f6fec005f90 comp rx=0 tx=0).stop 2024-09-19T09:20:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:01.883+0000 7f7001e9b640 1 -- 172.21.15.16:0/3000351351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc104fd0 msgr2=0x7f6ffc198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:01.883+0000 7f7001e9b640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc104fd0 0x7f6ffc198470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0031440 tx=0x7f6ff0032ce0 comp rx=0 tx=0).stop 2024-09-19T09:20:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:01.883+0000 7f7001e9b640 1 -- 172.21.15.16:0/3000351351 shutdown_connections 2024-09-19T09:20:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:01.883+0000 7f7001e9b640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6fe4039500 0x7f6fe403b9c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:01.883+0000 7f7001e9b640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ffc107900 0x7f6ffc19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:01.883+0000 7f7001e9b640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc104fd0 0x7f6ffc198470 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:01.883+0000 7f7001e9b640 1 --2- 172.21.15.16:0/3000351351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ffc1026a0 0x7f6ffc197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:01.883+0000 7f7001e9b640 1 -- 172.21.15.16:0/3000351351 >> 172.21.15.16:0/3000351351 conn(0x7f6ffc0fc2a0 msgr2=0x7f6ffc1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:01.883+0000 7f7001e9b640 1 -- 172.21.15.16:0/3000351351 shutdown_connections 2024-09-19T09:20:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:01.883+0000 7f7001e9b640 1 -- 172.21.15.16:0/3000351351 wait complete. 2024-09-19T09:20:02.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:02.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:02.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:02.453 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:02.453 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:02.453 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:02.484 DEBUG:teuthology.orchestra.run.smithi016:osd.0> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T09:20:02.487 INFO:tasks.cephadm:Deploying osd.1 on smithi097 with /dev/vg_nvme/lv_4... 2024-09-19T09:20:02.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T09:20:02.511 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:02.511 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:02.511 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:02.741 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:20:03.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:03 smithi016 ceph-mon[30979]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:20:03.518 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:03 smithi097 ceph-mon[34864]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:20:03.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:03 smithi142 ceph-mon[35032]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:20:04.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:04 smithi016 ceph-mon[30979]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T09:20:04.253 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:20:04 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0[42219]: 2024-09-19T09:20:04.072+0000 7f05e471c640 -1 osd.0 0 log_to_monitors true 2024-09-19T09:20:04.470 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:04 smithi097 ceph-mon[34864]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T09:20:04.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:04 smithi142 ceph-mon[35032]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T09:20:04.977 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:20:05.003 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2024-09-19T09:20:05.294 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:20:05.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:05 smithi016 ceph-mon[30979]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:20:05.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:05 smithi016 ceph-mon[30979]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T09:20:05.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:05 smithi016 ceph-mon[30979]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T09:20:05.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:05 smithi016 ceph-mon[30979]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-19T09:20:05.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:05 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:05.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:05 smithi097 ceph-mon[34864]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:20:05.487 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:05 smithi097 ceph-mon[34864]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T09:20:05.487 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:05 smithi097 ceph-mon[34864]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T09:20:05.487 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:05 smithi097 ceph-mon[34864]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-19T09:20:05.487 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:05 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:05.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:05 smithi142 ceph-mon[35032]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:20:05.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:05 smithi142 ceph-mon[35032]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T09:20:05.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:05 smithi142 ceph-mon[35032]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T09:20:05.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:05 smithi142 ceph-mon[35032]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-19T09:20:05.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:05 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.754+0000 7fdfdc30b640 1 -- 172.21.15.97:0/2129716047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4101cd0 msgr2=0x7fdfd4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.754+0000 7fdfdc30b640 1 --2- 172.21.15.97:0/2129716047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4101cd0 0x7fdfd4102150 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009900 tx=0x7fdfc402a410 comp rx=0 tx=0).stop 2024-09-19T09:20:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 -- 172.21.15.97:0/2129716047 shutdown_connections 2024-09-19T09:20:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 --2- 172.21.15.97:0/2129716047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfd4102690 0x7fdfd4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 --2- 172.21.15.97:0/2129716047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4101cd0 0x7fdfd4102150 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 --2- 172.21.15.97:0/2129716047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4100ad0 0x7fdfd4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 -- 172.21.15.97:0/2129716047 >> 172.21.15.97:0/2129716047 conn(0x7fdfd40fc280 msgr2=0x7fdfd40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 -- 172.21.15.97:0/2129716047 shutdown_connections 2024-09-19T09:20:05.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 -- 172.21.15.97:0/2129716047 wait complete. 2024-09-19T09:20:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 Processor -- start 2024-09-19T09:20:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 -- start start 2024-09-19T09:20:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4100ad0 0x7fdfd406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfd4101cd0 0x7fdfd419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4102690 0x7fdfd419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd410c750 con 0x7fdfd4102690 2024-09-19T09:20:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd410c5d0 con 0x7fdfd4100ad0 2024-09-19T09:20:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfdc30b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd410c8d0 con 0x7fdfd4101cd0 2024-09-19T09:20:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.755+0000 7fdfda0a7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4100ad0 0x7fdfd406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:05.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfda0a7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4100ad0 0x7fdfd406aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36840/0 (socket says 172.21.15.97:36840) 2024-09-19T09:20:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfda0a7640 1 -- 172.21.15.97:0/690017230 learned_addr learned my addr 172.21.15.97:0/690017230 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-19T09:20:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfda8a8640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4102690 0x7fdfd419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfd98a6640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfd4101cd0 0x7fdfd419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfda0a7640 1 -- 172.21.15.97:0/690017230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfd4101cd0 msgr2=0x7fdfd419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfda0a7640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfd4101cd0 0x7fdfd419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfda0a7640 1 -- 172.21.15.97:0/690017230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4102690 msgr2=0x7fdfd419f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfda0a7640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4102690 0x7fdfd419f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfda0a7640 1 -- 172.21.15.97:0/690017230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd419fb70 con 0x7fdfd4100ad0 2024-09-19T09:20:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfda8a8640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4102690 0x7fdfd419f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:05.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfda0a7640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4100ad0 0x7fdfd406aad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000d950 tx=0x7fdfd000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfcb7fe640 1 -- 172.21.15.97:0/690017230 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0017020 con 0x7fdfd4100ad0 2024-09-19T09:20:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfcb7fe640 1 -- 172.21.15.97:0/690017230 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd0009c30 con 0x7fdfd4100ad0 2024-09-19T09:20:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfdc30b640 1 -- 172.21.15.97:0/690017230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd419fe00 con 0x7fdfd4100ad0 2024-09-19T09:20:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfcb7fe640 1 -- 172.21.15.97:0/690017230 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd000f720 con 0x7fdfd4100ad0 2024-09-19T09:20:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.756+0000 7fdfdc30b640 1 -- 172.21.15.97:0/690017230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd41a0250 con 0x7fdfd4100ad0 2024-09-19T09:20:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.757+0000 7fdfdc30b640 1 -- 172.21.15.97:0/690017230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd4107f20 con 0x7fdfd4100ad0 2024-09-19T09:20:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.758+0000 7fdfcb7fe640 1 -- 172.21.15.97:0/690017230 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7fdfd001a020 con 0x7fdfd4100ad0 2024-09-19T09:20:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.759+0000 7fdfcb7fe640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdfc00a31b0 0x7fdfc00a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.759+0000 7fdfcb7fe640 1 -- 172.21.15.97:0/690017230 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fdfd00cdff0 con 0x7fdfd4100ad0 2024-09-19T09:20:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.759+0000 7fdfd98a6640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdfc00a31b0 0x7fdfc00a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:05.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.760+0000 7fdfd98a6640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdfc00a31b0 0x7fdfc00a5670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40314e0 tx=0x7fdfc4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:05.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.763+0000 7fdfcb7fe640 1 -- 172.21.15.97:0/690017230 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdfd0021020 con 0x7fdfd4100ad0 2024-09-19T09:20:05.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:05.941+0000 7fdfdc30b640 1 -- 172.21.15.97:0/690017230 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fdfd4062630 con 0x7fdfc00a31b0 2024-09-19T09:20:06.188 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:20:05 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0[42219]: 2024-09-19T09:20:05.905+0000 7f05dfe7c640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T09:20:06.188 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:20:05 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0[42219]: 2024-09-19T09:20:05.908+0000 7f05d84ae640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:20:06.458 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-19T09:20:06.458 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T09:20:06.458 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:06.458 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:06.459 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:06.459 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:06.459 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:06.459 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:06.459 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:06.459 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:06.459 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:06.459 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' 2024-09-19T09:20:06.459 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:20:06.459 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:20:06.459 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:06.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-19T09:20:06.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T09:20:06.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:06.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:06.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:06.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:06.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:06.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:06.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:06.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:06.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:06.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' 2024-09-19T09:20:06.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:20:06.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:20:06.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:06.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-19T09:20:06.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T09:20:06.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:06.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:06.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:06.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:06.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:06.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:06.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:06.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:06.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:06.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623]' entity='osd.0' 2024-09-19T09:20:06.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:20:06.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:20:06.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:07.450 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:07 smithi097 ceph-mon[34864]: purged_snaps scrub starts 2024-09-19T09:20:07.450 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:07 smithi097 ceph-mon[34864]: purged_snaps scrub ok 2024-09-19T09:20:07.450 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:07 smithi097 ceph-mon[34864]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:20:07.451 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:07 smithi097 ceph-mon[34864]: Detected new or changed devices on smithi016 2024-09-19T09:20:07.451 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:07 smithi097 ceph-mon[34864]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-19T09:20:07.451 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:07 smithi097 ceph-mon[34864]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:20:07.451 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:07.451 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:07 smithi097 ceph-mon[34864]: osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623] boot 2024-09-19T09:20:07.451 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:07 smithi097 ceph-mon[34864]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T09:20:07.451 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:07.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:07 smithi016 ceph-mon[30979]: purged_snaps scrub starts 2024-09-19T09:20:07.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:07 smithi016 ceph-mon[30979]: purged_snaps scrub ok 2024-09-19T09:20:07.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:07 smithi016 ceph-mon[30979]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:20:07.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:07 smithi016 ceph-mon[30979]: Detected new or changed devices on smithi016 2024-09-19T09:20:07.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:07 smithi016 ceph-mon[30979]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-19T09:20:07.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:07 smithi016 ceph-mon[30979]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:20:07.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:07.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:07 smithi016 ceph-mon[30979]: osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623] boot 2024-09-19T09:20:07.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:07 smithi016 ceph-mon[30979]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T09:20:07.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:07.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:07 smithi142 ceph-mon[35032]: purged_snaps scrub starts 2024-09-19T09:20:07.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:07 smithi142 ceph-mon[35032]: purged_snaps scrub ok 2024-09-19T09:20:07.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:07 smithi142 ceph-mon[35032]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T09:20:07.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:07 smithi142 ceph-mon[35032]: Detected new or changed devices on smithi016 2024-09-19T09:20:07.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:07 smithi142 ceph-mon[35032]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-19T09:20:07.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:07 smithi142 ceph-mon[35032]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:20:07.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:07.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:07 smithi142 ceph-mon[35032]: osd.0 [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623] boot 2024-09-19T09:20:07.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:07 smithi142 ceph-mon[35032]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T09:20:07.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T09:20:09.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:08 smithi097 ceph-mon[34864]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:09.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:08 smithi097 ceph-mon[34864]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T09:20:09.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:08 smithi097 ceph-mon[34864]: from='client.? 172.21.15.97:0/2374364694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2517aeb-1d79-41ad-a180-369eecb9732d"}]: dispatch 2024-09-19T09:20:09.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:08 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2517aeb-1d79-41ad-a180-369eecb9732d"}]: dispatch 2024-09-19T09:20:09.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:08 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2517aeb-1d79-41ad-a180-369eecb9732d"}]': finished 2024-09-19T09:20:09.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:08 smithi097 ceph-mon[34864]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T09:20:09.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:08 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:09.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:08 smithi097 ceph-mon[34864]: from='client.? 172.21.15.97:0/1213893713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:20:09.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:08 smithi016 ceph-mon[30979]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:09.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:08 smithi016 ceph-mon[30979]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T09:20:09.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:08 smithi016 ceph-mon[30979]: from='client.? 172.21.15.97:0/2374364694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2517aeb-1d79-41ad-a180-369eecb9732d"}]: dispatch 2024-09-19T09:20:09.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:08 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2517aeb-1d79-41ad-a180-369eecb9732d"}]: dispatch 2024-09-19T09:20:09.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:08 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2517aeb-1d79-41ad-a180-369eecb9732d"}]': finished 2024-09-19T09:20:09.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:08 smithi016 ceph-mon[30979]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T09:20:09.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:08 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:09.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:08 smithi016 ceph-mon[30979]: from='client.? 172.21.15.97:0/1213893713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:20:09.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:08 smithi142 ceph-mon[35032]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:09.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:08 smithi142 ceph-mon[35032]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T09:20:09.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:08 smithi142 ceph-mon[35032]: from='client.? 172.21.15.97:0/2374364694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2517aeb-1d79-41ad-a180-369eecb9732d"}]: dispatch 2024-09-19T09:20:09.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:08 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2517aeb-1d79-41ad-a180-369eecb9732d"}]: dispatch 2024-09-19T09:20:09.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:08 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2517aeb-1d79-41ad-a180-369eecb9732d"}]': finished 2024-09-19T09:20:09.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:08 smithi142 ceph-mon[35032]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T09:20:09.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:08 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:09.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:08 smithi142 ceph-mon[35032]: from='client.? 172.21.15.97:0/1213893713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:20:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:10 smithi016 ceph-mon[30979]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:11.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:10 smithi142 ceph-mon[35032]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:11.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:10 smithi097 ceph-mon[34864]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:13.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:12 smithi016 ceph-mon[30979]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:13.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:12 smithi097 ceph-mon[34864]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:13.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:12 smithi142 ceph-mon[35032]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:15.165 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:14 smithi097 ceph-mon[34864]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:15.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:14 smithi016 ceph-mon[30979]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:15.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:14 smithi142 ceph-mon[35032]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:17.118 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:16 smithi097 ceph-mon[34864]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:17.119 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:16 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T09:20:17.119 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:16 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:17.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:16 smithi016 ceph-mon[30979]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:17.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:16 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T09:20:17.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:16 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:17.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:16 smithi142 ceph-mon[35032]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:17.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:16 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T09:20:17.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:16 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:18.025 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:17 smithi097 ceph-mon[34864]: Deploying daemon osd.1 on smithi097 2024-09-19T09:20:18.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:17 smithi016 ceph-mon[30979]: Deploying daemon osd.1 on smithi097 2024-09-19T09:20:18.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:17 smithi142 ceph-mon[35032]: Deploying daemon osd.1 on smithi097 2024-09-19T09:20:19.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:18 smithi097 ceph-mon[34864]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:19.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:18 smithi016 ceph-mon[30979]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:19.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:18 smithi142 ceph-mon[35032]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:21.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:20 smithi016 ceph-mon[30979]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:21.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:20 smithi142 ceph-mon[35032]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:21.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:20 smithi097 ceph-mon[34864]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:23.242 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:22 smithi097 ceph-mon[34864]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:23.242 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:22 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:23.242 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:22 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:23.242 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:22 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:23.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:22 smithi016 ceph-mon[30979]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:23.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:22 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:23.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:22 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:23.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:22 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:22 smithi142 ceph-mon[35032]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:22 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:22 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:22 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:25.119 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:24 smithi097 ceph-mon[34864]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:25.248 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:25.248 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:25.248 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:25.248 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:25.248 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:25.256 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:24 smithi016 ceph-mon[30979]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:25.256 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:25.256 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:25.256 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:25.256 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:25.256 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:25.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:24 smithi142 ceph-mon[35032]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:25.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:25.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:26.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:26.804+0000 7fdfcb7fe640 1 -- 172.21.15.97:0/690017230 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdfd4062630 con 0x7fdfc00a31b0 2024-09-19T09:20:26.806 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 1 on host 'smithi097' 2024-09-19T09:20:26.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:26.806+0000 7fdfdc30b640 1 -- 172.21.15.97:0/690017230 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdfc00a31b0 msgr2=0x7fdfc00a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:26.806+0000 7fdfdc30b640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdfc00a31b0 0x7fdfc00a5670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40314e0 tx=0x7fdfc4000bc0 comp rx=0 tx=0).stop 2024-09-19T09:20:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:26.806+0000 7fdfdc30b640 1 -- 172.21.15.97:0/690017230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4100ad0 msgr2=0x7fdfd406aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:26.806+0000 7fdfdc30b640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4100ad0 0x7fdfd406aad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000d950 tx=0x7fdfd000de10 comp rx=0 tx=0).stop 2024-09-19T09:20:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:26.807+0000 7fdfdc30b640 1 -- 172.21.15.97:0/690017230 shutdown_connections 2024-09-19T09:20:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:26.807+0000 7fdfdc30b640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdfc00a31b0 0x7fdfc00a5670 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:26.807+0000 7fdfdc30b640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfd4101cd0 0x7fdfd419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:26.807+0000 7fdfdc30b640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfd4102690 0x7fdfd419f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:26.807+0000 7fdfdc30b640 1 --2- 172.21.15.97:0/690017230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfd4100ad0 0x7fdfd406aad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:26.807+0000 7fdfdc30b640 1 -- 172.21.15.97:0/690017230 >> 172.21.15.97:0/690017230 conn(0x7fdfd40fc280 msgr2=0x7fdfd41050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:26.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:26.807+0000 7fdfdc30b640 1 -- 172.21.15.97:0/690017230 shutdown_connections 2024-09-19T09:20:26.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:20:26.807+0000 7fdfdc30b640 1 -- 172.21.15.97:0/690017230 wait complete. 2024-09-19T09:20:27.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:26 smithi097 ceph-mon[34864]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:27.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:27.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:27.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:27.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:26 smithi016 ceph-mon[30979]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:27.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:27.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:27.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:27.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:26 smithi142 ceph-mon[35032]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:27.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:27.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:27.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:27.446 DEBUG:teuthology.orchestra.run.smithi097:osd.1> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T09:20:27.449 INFO:tasks.cephadm:Deploying osd.2 on smithi142 with /dev/vg_nvme/lv_4... 2024-09-19T09:20:27.449 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T09:20:27.704 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:20:29.059 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:28 smithi142 ceph-mon[35032]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:29.059 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:28 smithi142 ceph-mon[35032]: from='osd.1 [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T09:20:29.059 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:28 smithi142 ceph-mon[35032]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T09:20:29.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:28 smithi097 ceph-mon[34864]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:29.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:28 smithi097 ceph-mon[34864]: from='osd.1 [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T09:20:29.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:28 smithi097 ceph-mon[34864]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T09:20:29.137 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:20:28 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1[39353]: 2024-09-19T09:20:28.912+0000 7f749c0bd640 -1 osd.1 0 log_to_monitors true 2024-09-19T09:20:29.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:28 smithi016 ceph-mon[30979]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:29.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:28 smithi016 ceph-mon[30979]: from='osd.1 [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T09:20:29.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:28 smithi016 ceph-mon[30979]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T09:20:30.054 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:20:30.071 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:29 smithi097 ceph-mon[34864]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T09:20:30.071 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:29 smithi097 ceph-mon[34864]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T09:20:30.071 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:29 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:30.071 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:29 smithi097 ceph-mon[34864]: from='osd.1 [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:20:30.071 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:29 smithi097 ceph-mon[34864]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:20:30.079 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2024-09-19T09:20:30.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:29 smithi142 ceph-mon[35032]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T09:20:30.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:29 smithi142 ceph-mon[35032]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T09:20:30.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:29 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:30.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:29 smithi142 ceph-mon[35032]: from='osd.1 [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:20:30.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:29 smithi142 ceph-mon[35032]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:20:30.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:29 smithi016 ceph-mon[30979]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T09:20:30.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:29 smithi016 ceph-mon[30979]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T09:20:30.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:29 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:30.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:29 smithi016 ceph-mon[30979]: from='osd.1 [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:20:30.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:29 smithi016 ceph-mon[30979]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:20:30.364 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:20:30.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.802+0000 7ff847dbb640 1 -- 172.21.15.142:0/4007629150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff840100ad0 msgr2=0x7ff840100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:30.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.802+0000 7ff847dbb640 1 --2- 172.21.15.142:0/4007629150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff840100ad0 0x7ff840100ed0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff838009900 tx=0x7ff83802a410 comp rx=0 tx=0).stop 2024-09-19T09:20:30.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.802+0000 7ff847dbb640 1 -- 172.21.15.142:0/4007629150 shutdown_connections 2024-09-19T09:20:30.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.802+0000 7ff847dbb640 1 --2- 172.21.15.142:0/4007629150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840102690 0x7ff840108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.802+0000 7ff847dbb640 1 --2- 172.21.15.142:0/4007629150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff840101cd0 0x7ff840102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.802+0000 7ff847dbb640 1 --2- 172.21.15.142:0/4007629150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff840100ad0 0x7ff840100ed0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.802+0000 7ff847dbb640 1 -- 172.21.15.142:0/4007629150 >> 172.21.15.142:0/4007629150 conn(0x7ff8400fc280 msgr2=0x7ff8400fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:30.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.802+0000 7ff847dbb640 1 -- 172.21.15.142:0/4007629150 shutdown_connections 2024-09-19T09:20:30.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.802+0000 7ff847dbb640 1 -- 172.21.15.142:0/4007629150 wait complete. 2024-09-19T09:20:30.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.802+0000 7ff847dbb640 1 Processor -- start 2024-09-19T09:20:30.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff847dbb640 1 -- start start 2024-09-19T09:20:30.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff847dbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840100ad0 0x7ff84010b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff847dbb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff840101cd0 0x7ff840109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff847dbb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff840102690 0x7ff840109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff847dbb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff84010d880 con 0x7ff840100ad0 2024-09-19T09:20:30.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff847dbb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff84010d700 con 0x7ff840102690 2024-09-19T09:20:30.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff847dbb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff84010da00 con 0x7ff840101cd0 2024-09-19T09:20:30.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff845356640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff840101cd0 0x7ff840109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff845356640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff840101cd0 0x7ff840109aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:39086/0 (socket says 172.21.15.142:39086) 2024-09-19T09:20:30.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff845356640 1 -- 172.21.15.142:0/912193266 learned_addr learned my addr 172.21.15.142:0/912193266 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-19T09:20:30.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff845b57640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840100ad0 0x7ff84010b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff846358640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff840102690 0x7ff840109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff845356640 1 -- 172.21.15.142:0/912193266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff840102690 msgr2=0x7ff840109fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:30.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff845356640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff840102690 0x7ff840109fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff845356640 1 -- 172.21.15.142:0/912193266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840100ad0 msgr2=0x7ff84010b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:30.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff845356640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840100ad0 0x7ff84010b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:30.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff845356640 1 -- 172.21.15.142:0/912193266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84010a8a0 con 0x7ff840101cd0 2024-09-19T09:20:30.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff845b57640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840100ad0 0x7ff84010b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:30.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff845356640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff840101cd0 0x7ff840109aa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff83400e9f0 tx=0x7ff83400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:30.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.803+0000 7ff83effd640 1 -- 172.21.15.142:0/912193266 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83400cd80 con 0x7ff840101cd0 2024-09-19T09:20:30.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.804+0000 7ff83effd640 1 -- 172.21.15.142:0/912193266 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff834005500 con 0x7ff840101cd0 2024-09-19T09:20:30.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.804+0000 7ff847dbb640 1 -- 172.21.15.142:0/912193266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8401a2110 con 0x7ff840101cd0 2024-09-19T09:20:30.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.804+0000 7ff83effd640 1 -- 172.21.15.142:0/912193266 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834010690 con 0x7ff840101cd0 2024-09-19T09:20:30.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.804+0000 7ff847dbb640 1 -- 172.21.15.142:0/912193266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8401a25d0 con 0x7ff840101cd0 2024-09-19T09:20:30.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.805+0000 7ff847dbb640 1 -- 172.21.15.142:0/912193266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff840107f70 con 0x7ff840101cd0 2024-09-19T09:20:30.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.806+0000 7ff83effd640 1 -- 172.21.15.142:0/912193266 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7ff8340046c0 con 0x7ff840101cd0 2024-09-19T09:20:30.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.806+0000 7ff83effd640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8280a31b0 0x7ff8280a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:30.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.806+0000 7ff83effd640 1 -- 172.21.15.142:0/912193266 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7ff8340ce490 con 0x7ff840101cd0 2024-09-19T09:20:30.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.807+0000 7ff845b57640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8280a31b0 0x7ff8280a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:30.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.807+0000 7ff845b57640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8280a31b0 0x7ff8280a5670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff838006010 tx=0x7ff838032cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:30.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.810+0000 7ff83effd640 1 -- 172.21.15.142:0/912193266 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff83409cb20 con 0x7ff840101cd0 2024-09-19T09:20:30.987 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:30.986+0000 7ff847dbb640 1 -- 172.21.15.142:0/912193266 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff840062630 con 0x7ff8280a31b0 2024-09-19T09:20:31.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:31.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-19T09:20:31.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T09:20:31.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:31.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:31.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: Detected new or changed devices on smithi097 2024-09-19T09:20:31.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:31.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:31.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:31.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: Adjusting osd_memory_target on smithi097 to 17088M 2024-09-19T09:20:31.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:31.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:31.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:31.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:31.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:30 smithi142 ceph-mon[35032]: from='osd.1 ' entity='osd.1' 2024-09-19T09:20:31.135 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1[39353]: 2024-09-19T09:20:30.733+0000 7f749781d640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T09:20:31.135 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1[39353]: 2024-09-19T09:20:30.737+0000 7f748fe4f640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:20:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-19T09:20:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T09:20:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: Detected new or changed devices on smithi097 2024-09-19T09:20:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: Adjusting osd_memory_target on smithi097 to 17088M 2024-09-19T09:20:31.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:31.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:30 smithi097 ceph-mon[34864]: from='osd.1 ' entity='osd.1' 2024-09-19T09:20:31.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:31.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-19T09:20:31.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T09:20:31.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:31.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:31.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: Detected new or changed devices on smithi097 2024-09-19T09:20:31.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:31.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:31.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:31.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: Adjusting osd_memory_target on smithi097 to 17088M 2024-09-19T09:20:31.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:31.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:31.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:31.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:31.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:30 smithi016 ceph-mon[30979]: from='osd.1 ' entity='osd.1' 2024-09-19T09:20:32.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:31 smithi097 ceph-mon[34864]: purged_snaps scrub starts 2024-09-19T09:20:32.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:31 smithi097 ceph-mon[34864]: purged_snaps scrub ok 2024-09-19T09:20:32.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:32.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:31 smithi097 ceph-mon[34864]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:20:32.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:20:32.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:20:32.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:32.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:31 smithi097 ceph-mon[34864]: osd.1 [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349] boot 2024-09-19T09:20:32.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:31 smithi097 ceph-mon[34864]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T09:20:32.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:32.234 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:31 smithi142 ceph-mon[35032]: purged_snaps scrub starts 2024-09-19T09:20:32.235 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:31 smithi142 ceph-mon[35032]: purged_snaps scrub ok 2024-09-19T09:20:32.235 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:32.235 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:31 smithi142 ceph-mon[35032]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:20:32.235 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:20:32.235 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:20:32.235 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:32.235 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:31 smithi142 ceph-mon[35032]: osd.1 [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349] boot 2024-09-19T09:20:32.235 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:31 smithi142 ceph-mon[35032]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T09:20:32.235 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:32.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:31 smithi016 ceph-mon[30979]: purged_snaps scrub starts 2024-09-19T09:20:32.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:31 smithi016 ceph-mon[30979]: purged_snaps scrub ok 2024-09-19T09:20:32.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:32.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:31 smithi016 ceph-mon[30979]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:20:32.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:20:32.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:20:32.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:32.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:31 smithi016 ceph-mon[30979]: osd.1 [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349] boot 2024-09-19T09:20:32.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:31 smithi016 ceph-mon[30979]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T09:20:32.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T09:20:33.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:32 smithi016 ceph-mon[30979]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:33.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:32 smithi016 ceph-mon[30979]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T09:20:33.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:32 smithi142 ceph-mon[35032]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:33.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:32 smithi142 ceph-mon[35032]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T09:20:33.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:32 smithi097 ceph-mon[34864]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-19T09:20:33.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:32 smithi097 ceph-mon[34864]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T09:20:34.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:33 smithi142 ceph-mon[35032]: from='client.? 172.21.15.142:0/1168437938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c367518-5085-471a-98cb-ec5bf5facdf4"}]: dispatch 2024-09-19T09:20:34.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:33 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c367518-5085-471a-98cb-ec5bf5facdf4"}]: dispatch 2024-09-19T09:20:34.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:33 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c367518-5085-471a-98cb-ec5bf5facdf4"}]': finished 2024-09-19T09:20:34.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:33 smithi142 ceph-mon[35032]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T09:20:34.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:33 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:34.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:33 smithi142 ceph-mon[35032]: from='client.? 172.21.15.142:0/3310158576' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:20:34.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:33 smithi016 ceph-mon[30979]: from='client.? 172.21.15.142:0/1168437938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c367518-5085-471a-98cb-ec5bf5facdf4"}]: dispatch 2024-09-19T09:20:34.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:33 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c367518-5085-471a-98cb-ec5bf5facdf4"}]: dispatch 2024-09-19T09:20:34.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:33 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c367518-5085-471a-98cb-ec5bf5facdf4"}]': finished 2024-09-19T09:20:34.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:33 smithi016 ceph-mon[30979]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T09:20:34.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:33 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:34.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:33 smithi016 ceph-mon[30979]: from='client.? 172.21.15.142:0/3310158576' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:20:34.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:33 smithi097 ceph-mon[34864]: from='client.? 172.21.15.142:0/1168437938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c367518-5085-471a-98cb-ec5bf5facdf4"}]: dispatch 2024-09-19T09:20:34.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:33 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c367518-5085-471a-98cb-ec5bf5facdf4"}]: dispatch 2024-09-19T09:20:34.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:33 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c367518-5085-471a-98cb-ec5bf5facdf4"}]': finished 2024-09-19T09:20:34.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:33 smithi097 ceph-mon[34864]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T09:20:34.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:33 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:34.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:33 smithi097 ceph-mon[34864]: from='client.? 172.21.15.142:0/3310158576' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:20:35.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:35 smithi016 ceph-mon[30979]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:35.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:35 smithi142 ceph-mon[35032]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:35.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:35 smithi097 ceph-mon[34864]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:37.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:37 smithi016 ceph-mon[30979]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:37.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:37 smithi142 ceph-mon[35032]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:37.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:37 smithi097 ceph-mon[34864]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:39 smithi016 ceph-mon[30979]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:39.617 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:39 smithi142 ceph-mon[35032]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:39.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:39 smithi097 ceph-mon[34864]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:41.338 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:41 smithi142 ceph-mon[35032]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:41.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:41 smithi016 ceph-mon[30979]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:41.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:41 smithi097 ceph-mon[34864]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:42.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T09:20:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:42.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T09:20:42.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:42.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T09:20:42.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:43.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:43 smithi142 ceph-mon[35032]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:43.377 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:43 smithi142 ceph-mon[35032]: Deploying daemon osd.2 on smithi142 2024-09-19T09:20:43.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:43 smithi016 ceph-mon[30979]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:43.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:43 smithi016 ceph-mon[30979]: Deploying daemon osd.2 on smithi142 2024-09-19T09:20:43.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:43 smithi097 ceph-mon[34864]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:43.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:43 smithi097 ceph-mon[34864]: Deploying daemon osd.2 on smithi142 2024-09-19T09:20:45.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:45 smithi142 ceph-mon[35032]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:45 smithi016 ceph-mon[30979]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:45.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:45 smithi097 ceph-mon[34864]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:47.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:47 smithi142 ceph-mon[35032]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:47 smithi016 ceph-mon[30979]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:47.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:47 smithi097 ceph-mon[34864]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:48.534 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:48 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:48.534 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:48 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:48.534 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:48 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:48.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:48 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:48.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:48 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:48.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:48 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:48.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:48 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:48.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:48 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:48.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:48 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:49.366 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:49 smithi142 ceph-mon[35032]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:49.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:49 smithi097 ceph-mon[34864]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:49.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:49 smithi016 ceph-mon[30979]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:50.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:50 smithi142 ceph-mon[35032]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:50.537 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:50 smithi097 ceph-mon[34864]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:50.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:50 smithi016 ceph-mon[30979]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:51.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:51.499+0000 7ff83effd640 1 -- 172.21.15.142:0/912193266 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff840062630 con 0x7ff8280a31b0 2024-09-19T09:20:51.501 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 2 on host 'smithi142' 2024-09-19T09:20:51.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:51.501+0000 7ff847dbb640 1 -- 172.21.15.142:0/912193266 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8280a31b0 msgr2=0x7ff8280a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:51.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:51.501+0000 7ff847dbb640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8280a31b0 0x7ff8280a5670 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff838006010 tx=0x7ff838032cf0 comp rx=0 tx=0).stop 2024-09-19T09:20:51.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:51.502+0000 7ff847dbb640 1 -- 172.21.15.142:0/912193266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff840101cd0 msgr2=0x7ff840109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:51.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:51.502+0000 7ff847dbb640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff840101cd0 0x7ff840109aa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff83400e9f0 tx=0x7ff83400edf0 comp rx=0 tx=0).stop 2024-09-19T09:20:51.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:51.502+0000 7ff847dbb640 1 -- 172.21.15.142:0/912193266 shutdown_connections 2024-09-19T09:20:51.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:51.502+0000 7ff847dbb640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8280a31b0 0x7ff8280a5670 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:51.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:51.502+0000 7ff847dbb640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff840101cd0 0x7ff840109aa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:51.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:51.502+0000 7ff847dbb640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff840102690 0x7ff840109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:51.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:51.502+0000 7ff847dbb640 1 --2- 172.21.15.142:0/912193266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840100ad0 0x7ff84010b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:51.503 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:51.502+0000 7ff847dbb640 1 -- 172.21.15.142:0/912193266 >> 172.21.15.142:0/912193266 conn(0x7ff8400fc280 msgr2=0x7ff840105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:51.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:51.502+0000 7ff847dbb640 1 -- 172.21.15.142:0/912193266 shutdown_connections 2024-09-19T09:20:51.504 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:20:51.502+0000 7ff847dbb640 1 -- 172.21.15.142:0/912193266 wait complete. 2024-09-19T09:20:51.804 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:51 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:51.956 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:51 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:51.956 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:51 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:51.956 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:51 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:51.956 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:51 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:51.956 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:51 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:51.956 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:51 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:51.956 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:51 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:51.957 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:51 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:51.957 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:51 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:51.957 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:51 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:51.957 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:51 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:51.957 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:51 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:51.957 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:51 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:51.957 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:51 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:51.957 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:51 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:52.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:51 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:52.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:51 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:52.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:51 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:52.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:51 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:52.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:51 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:52.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:51 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:20:52.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:51 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:52.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:51 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:52.097 DEBUG:teuthology.orchestra.run.smithi142:osd.2> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T09:20:52.141 INFO:tasks.cephadm:Deploying osd.3 on smithi016 with /dev/vg_nvme/lv_3... 2024-09-19T09:20:52.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T09:20:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:20:52.820 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:52 smithi016 ceph-mon[30979]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:52.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:52 smithi142 ceph-mon[35032]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:52.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:52 smithi097 ceph-mon[34864]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:54.605 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:20:54 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2[40014]: 2024-09-19T09:20:54.283+0000 7fead2170640 -1 osd.2 0 log_to_monitors true 2024-09-19T09:20:54.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:54 smithi016 ceph-mon[30979]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:54.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:54 smithi016 ceph-mon[30979]: from='osd.2 [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T09:20:54.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:54 smithi016 ceph-mon[30979]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T09:20:54.860 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:20:54.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:54 smithi142 ceph-mon[35032]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:54.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:54 smithi142 ceph-mon[35032]: from='osd.2 [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T09:20:54.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:54 smithi142 ceph-mon[35032]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T09:20:54.885 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:54 smithi097 ceph-mon[34864]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:54.885 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:54 smithi097 ceph-mon[34864]: from='osd.2 [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T09:20:54.885 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:54 smithi097 ceph-mon[34864]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T09:20:54.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-09-19T09:20:55.146 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:20:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 -- 172.21.15.16:0/2272151144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101770 msgr2=0x7f0878101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 --2- 172.21.15.16:0/2272151144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101770 0x7f0878101c20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0870009ca0 tx=0x7f087002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 -- 172.21.15.16:0/2272151144 shutdown_connections 2024-09-19T09:20:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 --2- 172.21.15.16:0/2272151144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0878100dd0 0x7f0878101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 --2- 172.21.15.16:0/2272151144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101770 0x7f0878101c20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 --2- 172.21.15.16:0/2272151144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0878100490 0x7f0878100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 -- 172.21.15.16:0/2272151144 >> 172.21.15.16:0/2272151144 conn(0x7f08780fc2a0 msgr2=0x7f08780fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:20:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 -- 172.21.15.16:0/2272151144 shutdown_connections 2024-09-19T09:20:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 -- 172.21.15.16:0/2272151144 wait complete. 2024-09-19T09:20:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 Processor -- start 2024-09-19T09:20:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 -- start start 2024-09-19T09:20:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0878100490 0x7f087819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0878100dd0 0x7f087819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101770 0x7f08781a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f087810fa60 con 0x7f0878101770 2024-09-19T09:20:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f087810f8e0 con 0x7f0878100490 2024-09-19T09:20:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.646+0000 7f087fb6d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f087810fbe0 con 0x7f0878100dd0 2024-09-19T09:20:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087e10a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101770 0x7f08781a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087e10a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101770 0x7f08781a15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58076/0 (socket says 172.21.15.16:58076) 2024-09-19T09:20:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087e10a640 1 -- 172.21.15.16:0/2085458690 learned_addr learned my addr 172.21.15.16:0/2085458690 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:20:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087d909640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0878100490 0x7f087819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087d108640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0878100dd0 0x7f087819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087e10a640 1 -- 172.21.15.16:0/2085458690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0878100dd0 msgr2=0x7f087819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087e10a640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0878100dd0 0x7f087819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087e10a640 1 -- 172.21.15.16:0/2085458690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0878100490 msgr2=0x7f087819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:20:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087e10a640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0878100490 0x7f087819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:20:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087e10a640 1 -- 172.21.15.16:0/2085458690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08781a1c60 con 0x7f0878101770 2024-09-19T09:20:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087d909640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0878100490 0x7f087819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:20:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087e10a640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101770 0x7f08781a15f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0870005e30 tx=0x7f0870031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f086effd640 1 -- 172.21.15.16:0/2085458690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870003c30 con 0x7f0878101770 2024-09-19T09:20:55.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f086effd640 1 -- 172.21.15.16:0/2085458690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870003dd0 con 0x7f0878101770 2024-09-19T09:20:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f086effd640 1 -- 172.21.15.16:0/2085458690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087002fde0 con 0x7f0878101770 2024-09-19T09:20:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087fb6d640 1 -- 172.21.15.16:0/2085458690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08781a1ef0 con 0x7f0878101770 2024-09-19T09:20:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.647+0000 7f087fb6d640 1 -- 172.21.15.16:0/2085458690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08781a2350 con 0x7f0878101770 2024-09-19T09:20:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.648+0000 7f087fb6d640 1 -- 172.21.15.16:0/2085458690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f087810b1e0 con 0x7f0878101770 2024-09-19T09:20:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.649+0000 7f086effd640 1 -- 172.21.15.16:0/2085458690 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f0870004050 con 0x7f0878101770 2024-09-19T09:20:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.650+0000 7f086effd640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f08640a3260 0x7f08640a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:20:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.651+0000 7f087d909640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f08640a3260 0x7f08640a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:20:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.651+0000 7f086effd640 1 -- 172.21.15.16:0/2085458690 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f08700ef3d0 con 0x7f0878101770 2024-09-19T09:20:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.651+0000 7f087d909640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f08640a3260 0x7f08640a5720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f087400abe0 tx=0x7f087400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:20:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.654+0000 7f086effd640 1 -- 172.21.15.16:0/2085458690 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f08700bd840 con 0x7f0878101770 2024-09-19T09:20:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:20:55.838+0000 7f087fb6d640 1 -- 172.21.15.16:0/2085458690 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0878062630 con 0x7f08640a3260 2024-09-19T09:20:55.841 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T09:20:55.841 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T09:20:55.841 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:55.841 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: from='osd.2 [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:20:55.841 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:20:55.841 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: Detected new or changed devices on smithi142 2024-09-19T09:20:55.841 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:55.841 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:55.841 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:55.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: Adjusting osd_memory_target on smithi142 to 17088M 2024-09-19T09:20:55.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:55.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:55.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:55.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:55 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:55.878 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2[40014]: 2024-09-19T09:20:55.619+0000 7feacd8d0640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T09:20:55.878 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2[40014]: 2024-09-19T09:20:55.626+0000 7feac9f0a640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:20:55.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T09:20:55.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T09:20:55.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:55.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: from='osd.2 [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:20:55.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:20:55.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: Detected new or changed devices on smithi142 2024-09-19T09:20:55.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:55.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:55.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:55.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: Adjusting osd_memory_target on smithi142 to 17088M 2024-09-19T09:20:55.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:55.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:55.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:55.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:55 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:55.885 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T09:20:55.885 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T09:20:55.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:55.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: from='osd.2 [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:20:55.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:20:55.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: Detected new or changed devices on smithi142 2024-09-19T09:20:55.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:55.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:55.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:20:55.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: Adjusting osd_memory_target on smithi142 to 17088M 2024-09-19T09:20:55.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:55.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:55.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:20:55.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:55 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:20:56.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:56 smithi142 ceph-mon[35032]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:56.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:56 smithi142 ceph-mon[35032]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-19T09:20:56.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:56 smithi142 ceph-mon[35032]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T09:20:56.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:56.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:56.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:56 smithi142 ceph-mon[35032]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:20:56.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:20:56.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:20:56.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:56.885 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:56 smithi097 ceph-mon[34864]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:56.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:56 smithi097 ceph-mon[34864]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-19T09:20:56.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:56 smithi097 ceph-mon[34864]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T09:20:56.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:56.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:56.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:56 smithi097 ceph-mon[34864]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:20:56.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:20:56.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:20:56.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:56.949 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:56 smithi016 ceph-mon[30979]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:56.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:56 smithi016 ceph-mon[30979]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-19T09:20:56.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:56 smithi016 ceph-mon[30979]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T09:20:56.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:56.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:56.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:56 smithi016 ceph-mon[30979]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:20:56.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:20:56.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:20:56.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:20:57.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:57 smithi016 ceph-mon[30979]: purged_snaps scrub starts 2024-09-19T09:20:57.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:57 smithi016 ceph-mon[30979]: purged_snaps scrub ok 2024-09-19T09:20:57.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:57 smithi016 ceph-mon[30979]: osd.2 [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951] boot 2024-09-19T09:20:57.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:57 smithi016 ceph-mon[30979]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T09:20:57.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:57 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:58.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:57 smithi142 ceph-mon[35032]: purged_snaps scrub starts 2024-09-19T09:20:58.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:57 smithi142 ceph-mon[35032]: purged_snaps scrub ok 2024-09-19T09:20:58.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:57 smithi142 ceph-mon[35032]: osd.2 [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951] boot 2024-09-19T09:20:58.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:57 smithi142 ceph-mon[35032]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T09:20:58.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:57 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:58.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:57 smithi097 ceph-mon[34864]: purged_snaps scrub starts 2024-09-19T09:20:58.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:57 smithi097 ceph-mon[34864]: purged_snaps scrub ok 2024-09-19T09:20:58.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:57 smithi097 ceph-mon[34864]: osd.2 [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951] boot 2024-09-19T09:20:58.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:57 smithi097 ceph-mon[34864]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T09:20:58.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:57 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T09:20:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:58 smithi142 ceph-mon[35032]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:58 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:20:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:58 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T09:20:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:58 smithi142 ceph-mon[35032]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T09:20:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:58 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:20:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:58 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2050805453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7118925-f0fa-46b3-8552-3e3bb76a6489"}]: dispatch 2024-09-19T09:20:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:58 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:20:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:58 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2050805453' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7118925-f0fa-46b3-8552-3e3bb76a6489"}]': finished 2024-09-19T09:20:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:58 smithi142 ceph-mon[35032]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T09:20:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:58 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:20:59.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:58 smithi097 ceph-mon[34864]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:59.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:58 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:20:59.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:58 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T09:20:59.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:58 smithi097 ceph-mon[34864]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T09:20:59.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:58 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:20:59.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:58 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2050805453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7118925-f0fa-46b3-8552-3e3bb76a6489"}]: dispatch 2024-09-19T09:20:59.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:58 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:20:59.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:58 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2050805453' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7118925-f0fa-46b3-8552-3e3bb76a6489"}]': finished 2024-09-19T09:20:59.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:58 smithi097 ceph-mon[34864]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T09:20:59.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:58 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:20:59.233 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:20:58 smithi016 sudo[47000]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T09:20:59.233 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:20:58 smithi016 sudo[47000]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:20:59.233 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:20:58 smithi016 sudo[47000]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:20:59.233 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:20:58 smithi016 sudo[47000]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:20:59.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:58 smithi016 ceph-mon[30979]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-19T09:20:59.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:58 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:20:59.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:58 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T09:20:59.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:58 smithi016 ceph-mon[30979]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T09:20:59.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:58 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:20:59.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:58 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2050805453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7118925-f0fa-46b3-8552-3e3bb76a6489"}]: dispatch 2024-09-19T09:20:59.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:58 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:20:59.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:58 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2050805453' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7118925-f0fa-46b3-8552-3e3bb76a6489"}]': finished 2024-09-19T09:20:59.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:58 smithi016 ceph-mon[30979]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T09:20:59.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:58 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:20:59.503 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:20:59 smithi016 sudo[47052]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T09:20:59.503 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:20:59 smithi016 sudo[47052]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:20:59.503 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:20:59 smithi016 sudo[47052]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:20:59.503 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:20:59 smithi016 sudo[47052]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:20:59.758 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:20:59 smithi097 sudo[43075]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T09:20:59.758 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:20:59 smithi097 sudo[43075]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:20:59.758 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:20:59 smithi097 sudo[43075]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:20:59.759 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:20:59 smithi097 sudo[43075]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:21:00.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:59 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2640465540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:21:00.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:59 smithi142 ceph-mon[35032]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T09:21:00.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:59 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:00.028 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:20:59 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:59 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2640465540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:21:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:59 smithi097 ceph-mon[34864]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T09:21:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:59 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:20:59 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:00.136 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:20:59 smithi097 sudo[43078]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T09:21:00.136 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:20:59 smithi097 sudo[43078]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:21:00.137 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:20:59 smithi097 sudo[43078]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:21:00.137 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:20:59 smithi097 sudo[43078]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:21:00.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:59 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2640465540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:21:00.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:59 smithi016 ceph-mon[30979]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T09:21:00.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:59 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:00.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:20:59 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:00.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:00.275+0000 7f086effd640 1 -- 172.21.15.16:0/2085458690 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f08700b97b0 con 0x7f0878101770 2024-09-19T09:21:00.284 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:21:00 smithi142 sudo[43839]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T09:21:00.284 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:21:00 smithi142 sudo[43839]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:21:00.284 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:21:00 smithi142 sudo[43839]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:21:00.284 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:21:00 smithi142 sudo[43839]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:21:00.628 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:21:00 smithi142 sudo[43842]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T09:21:00.628 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:21:00 smithi142 sudo[43842]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:21:00.628 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:21:00 smithi142 sudo[43842]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:21:00.629 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:21:00 smithi142 sudo[43842]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:21:01.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:00 smithi016 sudo[47434]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T09:21:01.208 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:00 smithi016 sudo[47434]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:21:01.208 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:00 smithi016 sudo[47434]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:21:01.208 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:00 smithi016 sudo[47434]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:21:01.471 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:01 smithi142 ceph-mon[35032]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:01.471 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:01.471 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:01 smithi142 ceph-mon[35032]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T09:21:01.471 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:01.471 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:01 smithi142 ceph-mon[35032]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:21:01.471 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:01 smithi142 sudo[43845]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T09:21:01.471 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:01 smithi142 sudo[43845]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:21:01.472 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:01 smithi142 sudo[43845]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:21:01.472 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:01 smithi142 sudo[43845]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:21:01.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:01 smithi016 sudo[47440]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T09:21:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:01 smithi016 sudo[47440]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:21:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:01 smithi016 sudo[47440]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:21:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:01 smithi016 sudo[47440]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:21:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:01 smithi016 ceph-mon[30979]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:01 smithi016 ceph-mon[30979]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T09:21:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:01 smithi016 ceph-mon[30979]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:21:01.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:01 smithi097 ceph-mon[34864]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:01.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:01.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:01 smithi097 ceph-mon[34864]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T09:21:01.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:01.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:01 smithi097 ceph-mon[34864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:21:01.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:01 smithi142 sudo[43848]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T09:21:01.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:01 smithi142 sudo[43848]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:21:01.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:01 smithi142 sudo[43848]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:21:01.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:01 smithi142 sudo[43848]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:21:02.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:01 smithi097 sudo[43081]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T09:21:02.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:01 smithi097 sudo[43081]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:21:02.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:01 smithi097 sudo[43081]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:21:02.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:01 smithi097 sudo[43081]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:21:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:02 smithi016 ceph-mon[30979]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:21:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:21:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:21:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:21:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:02 smithi016 ceph-mon[30979]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:21:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:21:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:21:02.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:21:02.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:02 smithi016 ceph-mon[30979]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:21:02.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:21:02.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:21:02.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:21:02.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:02 smithi142 ceph-mon[35032]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:21:02.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:21:02.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:21:02.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:21:02.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:02 smithi142 ceph-mon[35032]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:21:02.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:21:02.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:21:02.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:21:02.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:02 smithi142 ceph-mon[35032]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:21:02.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:21:02.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:21:02.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:21:02.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 sudo[43084]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T09:21:02.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 sudo[43084]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T09:21:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 sudo[43084]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T09:21:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 sudo[43084]: pam_unix(sudo:session): session closed for user root 2024-09-19T09:21:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 ceph-mon[34864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:21:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:21:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:21:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:21:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 ceph-mon[34864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:21:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:21:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:21:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:21:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 ceph-mon[34864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:21:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T09:21:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T09:21:02.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T09:21:03.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:03 smithi016 ceph-mon[30979]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:03.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:03 smithi016 ceph-mon[30979]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:21:03.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:03 smithi016 ceph-mon[30979]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:21:03.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:03 smithi142 ceph-mon[35032]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:03.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:03 smithi142 ceph-mon[35032]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:21:03.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:03 smithi142 ceph-mon[35032]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:21:03.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:03 smithi097 ceph-mon[34864]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:03.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:03 smithi097 ceph-mon[34864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T09:21:03.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:03 smithi097 ceph-mon[34864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T09:21:05.439 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:05 smithi016 ceph-mon[30979]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:05.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:05 smithi142 ceph-mon[35032]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:05.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:05 smithi097 ceph-mon[34864]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:07.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:07 smithi016 ceph-mon[30979]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:07.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T09:21:07.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:07.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:07 smithi142 ceph-mon[35032]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:07.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T09:21:07.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:07.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:07 smithi097 ceph-mon[34864]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:07.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T09:21:07.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:08.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:08 smithi016 ceph-mon[30979]: Deploying daemon osd.3 on smithi016 2024-09-19T09:21:08.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:08 smithi142 ceph-mon[35032]: Deploying daemon osd.3 on smithi016 2024-09-19T09:21:08.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:08 smithi097 ceph-mon[34864]: Deploying daemon osd.3 on smithi016 2024-09-19T09:21:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:09 smithi016 ceph-mon[30979]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-19T09:21:09.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:09 smithi142 ceph-mon[35032]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-19T09:21:09.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:09 smithi097 ceph-mon[34864]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-19T09:21:11.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:11 smithi016 ceph-mon[30979]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-19T09:21:11.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:11 smithi142 ceph-mon[35032]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-19T09:21:11.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:11 smithi097 ceph-mon[34864]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-19T09:21:13.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:13 smithi016 ceph-mon[30979]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-19T09:21:13.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:13 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:13.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:13 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:13.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:13 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:13.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:13 smithi142 ceph-mon[35032]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-19T09:21:13.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:13 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:13.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:13 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:13.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:13 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:13.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:13 smithi097 ceph-mon[34864]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-19T09:21:13.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:13 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:13.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:13 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:13.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:13 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:15 smithi016 ceph-mon[30979]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T09:21:15.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:15 smithi142 ceph-mon[35032]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T09:21:15.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:15 smithi097 ceph-mon[34864]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T09:21:16.946 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:16 smithi016 ceph-mon[30979]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:21:16.946 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:16 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:16.946 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:16 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:16.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:16 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:16.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:16 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:16.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:16 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:16.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:16.983+0000 7f086effd640 1 -- 172.21.15.16:0/2085458690 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0878062630 con 0x7f08640a3260 2024-09-19T09:21:16.984 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 3 on host 'smithi016' 2024-09-19T09:21:16.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:16.985+0000 7f087fb6d640 1 -- 172.21.15.16:0/2085458690 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f08640a3260 msgr2=0x7f08640a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:16.985+0000 7f087fb6d640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f08640a3260 0x7f08640a5720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f087400abe0 tx=0x7f087400a5c0 comp rx=0 tx=0).stop 2024-09-19T09:21:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:16.985+0000 7f087fb6d640 1 -- 172.21.15.16:0/2085458690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101770 msgr2=0x7f08781a15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:16.985+0000 7f087fb6d640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101770 0x7f08781a15f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0870005e30 tx=0x7f0870031ee0 comp rx=0 tx=0).stop 2024-09-19T09:21:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:16.985+0000 7f087fb6d640 1 -- 172.21.15.16:0/2085458690 shutdown_connections 2024-09-19T09:21:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:16.985+0000 7f087fb6d640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f08640a3260 0x7f08640a5720 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:16.985+0000 7f087fb6d640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0878100dd0 0x7f087819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:16.985+0000 7f087fb6d640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878101770 0x7f08781a15f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:16.985+0000 7f087fb6d640 1 --2- 172.21.15.16:0/2085458690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0878100490 0x7f087819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:16.985+0000 7f087fb6d640 1 -- 172.21.15.16:0/2085458690 >> 172.21.15.16:0/2085458690 conn(0x7f08780fc2a0 msgr2=0x7f08780fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:16.986+0000 7f087fb6d640 1 -- 172.21.15.16:0/2085458690 shutdown_connections 2024-09-19T09:21:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:21:16.986+0000 7f087fb6d640 1 -- 172.21.15.16:0/2085458690 wait complete. 2024-09-19T09:21:17.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:16 smithi142 ceph-mon[35032]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:21:17.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:16 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:17.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:16 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:17.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:16 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:17.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:16 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:17.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:16 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:17.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:16 smithi097 ceph-mon[34864]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:21:17.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:16 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:17.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:16 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:17.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:16 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:17.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:16 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:17.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:16 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:17.660 DEBUG:teuthology.orchestra.run.smithi016:osd.3> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T09:21:17.663 INFO:tasks.cephadm:Deploying osd.4 on smithi097 with /dev/vg_nvme/lv_3... 2024-09-19T09:21:17.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T09:21:17.915 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:21:17.981 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:17.981 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:17.981 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:18.036 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:18.036 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:18.036 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:18.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:18.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:18.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:19.221 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:18 smithi097 ceph-mon[34864]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:21:19.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:18 smithi016 ceph-mon[30979]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:21:19.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:18 smithi142 ceph-mon[35032]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-19T09:21:19.714 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:21:19 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3[49037]: 2024-09-19T09:21:19.422+0000 7f48bfbe6640 -1 osd.3 0 log_to_monitors true 2024-09-19T09:21:20.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:19 smithi016 ceph-mon[30979]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T09:21:20.273 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:19 smithi097 ceph-mon[34864]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T09:21:20.276 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:21:20.303 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2024-09-19T09:21:20.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:19 smithi142 ceph-mon[35032]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T09:21:20.554 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:21:21.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.996+0000 7f592df7a640 1 -- 172.21.15.97:0/1372648886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928107940 msgr2=0x7f5928109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:21.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.996+0000 7f592df7a640 1 --2- 172.21.15.97:0/1372648886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928107940 0x7f5928109d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5920009ca0 tx=0x7f592002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.996+0000 7f592df7a640 1 -- 172.21.15.97:0/1372648886 shutdown_connections 2024-09-19T09:21:21.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.996+0000 7f592df7a640 1 --2- 172.21.15.97:0/1372648886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928107940 0x7f5928109d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.996+0000 7f592df7a640 1 --2- 172.21.15.97:0/1372648886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5928105010 0x7f5928107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.996+0000 7f592df7a640 1 --2- 172.21.15.97:0/1372648886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59281026e0 0x7f5928104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.996+0000 7f592df7a640 1 -- 172.21.15.97:0/1372648886 >> 172.21.15.97:0/1372648886 conn(0x7f59280fc280 msgr2=0x7f59280fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.996+0000 7f592df7a640 1 -- 172.21.15.97:0/1372648886 shutdown_connections 2024-09-19T09:21:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.996+0000 7f592df7a640 1 -- 172.21.15.97:0/1372648886 wait complete. 2024-09-19T09:21:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.996+0000 7f592df7a640 1 Processor -- start 2024-09-19T09:21:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.996+0000 7f592df7a640 1 -- start start 2024-09-19T09:21:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f592df7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59281026e0 0x7f5928102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f592df7a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928105010 0x7f5928100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f592df7a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5928107940 0x7f5928100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f592df7a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f592810d860 con 0x7f59281026e0 2024-09-19T09:21:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f592df7a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f592810d6e0 con 0x7f5928105010 2024-09-19T09:21:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f592df7a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f592810d9e0 con 0x7f5928107940 2024-09-19T09:21:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f5926ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928105010 0x7f5928100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f5926ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928105010 0x7f5928100690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57432/0 (socket says 172.21.15.97:57432) 2024-09-19T09:21:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f5926ffd640 1 -- 172.21.15.97:0/2964573897 learned_addr learned my addr 172.21.15.97:0/2964573897 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-19T09:21:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f59277fe640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59281026e0 0x7f5928102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f5927fff640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5928107940 0x7f5928100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f5926ffd640 1 -- 172.21.15.97:0/2964573897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5928107940 msgr2=0x7f5928100bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f5926ffd640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5928107940 0x7f5928100bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f5926ffd640 1 -- 172.21.15.97:0/2964573897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59281026e0 msgr2=0x7f5928102040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f5926ffd640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59281026e0 0x7f5928102040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f5926ffd640 1 -- 172.21.15.97:0/2964573897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5928101490 con 0x7f5928105010 2024-09-19T09:21:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.997+0000 7f5926ffd640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928105010 0x7f5928100690 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f591c005fd0 tx=0x7f591c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:20.999+0000 7f5924ff9640 1 -- 172.21.15.97:0/2964573897 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c017070 con 0x7f5928105010 2024-09-19T09:21:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:21.000+0000 7f5924ff9640 1 -- 172.21.15.97:0/2964573897 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591c009c30 con 0x7f5928105010 2024-09-19T09:21:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:21.000+0000 7f5924ff9640 1 -- 172.21.15.97:0/2964573897 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c010460 con 0x7f5928105010 2024-09-19T09:21:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:21.000+0000 7f592df7a640 1 -- 172.21.15.97:0/2964573897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59281a1fe0 con 0x7f5928105010 2024-09-19T09:21:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:21.000+0000 7f592df7a640 1 -- 172.21.15.97:0/2964573897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59281a2490 con 0x7f5928105010 2024-09-19T09:21:21.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:21.001+0000 7f592df7a640 1 -- 172.21.15.97:0/2964573897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59280fdd70 con 0x7f5928105010 2024-09-19T09:21:21.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:21.002+0000 7f5924ff9640 1 -- 172.21.15.97:0/2964573897 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f591c003e70 con 0x7f5928105010 2024-09-19T09:21:21.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:21.002+0000 7f5924ff9640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f59100a32c0 0x7f59100a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:21.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:21.003+0000 7f5924ff9640 1 -- 172.21.15.97:0/2964573897 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7f591c0ce100 con 0x7f5928105010 2024-09-19T09:21:21.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:21.003+0000 7f59277fe640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f59100a32c0 0x7f59100a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:21.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:21.003+0000 7f59277fe640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f59100a32c0 0x7f59100a5780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5918009bd0 tx=0x7f5918005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:21.007+0000 7f5924ff9640 1 -- 172.21.15.97:0/2964573897 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f591c020030 con 0x7f5928105010 2024-09-19T09:21:21.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:21.189+0000 7f592df7a640 1 -- 172.21.15.97:0/2964573897 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5928062630 con 0x7f59100a32c0 2024-09-19T09:21:21.193 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:20 smithi097 ceph-mon[34864]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:21.193 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:20 smithi097 ceph-mon[34864]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T09:21:21.193 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:20 smithi097 ceph-mon[34864]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T09:21:21.193 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:20 smithi097 ceph-mon[34864]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-19T09:21:21.193 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:21.193 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:21.194 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:21.194 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:21:21.194 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:21:21.194 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:21.194 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:21.194 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:21.194 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:21.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:20 smithi016 ceph-mon[30979]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:21.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:20 smithi016 ceph-mon[30979]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T09:21:21.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:20 smithi016 ceph-mon[30979]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T09:21:21.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:20 smithi016 ceph-mon[30979]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-19T09:21:21.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:21.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:21.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:21.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:21:21.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:21:21.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:21.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:21.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:21.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:21.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:20 smithi142 ceph-mon[35032]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:21.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:20 smithi142 ceph-mon[35032]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T09:21:21.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:20 smithi142 ceph-mon[35032]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T09:21:21.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:20 smithi142 ceph-mon[35032]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-19T09:21:21.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:21.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:21.281 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:21.281 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:21:21.281 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:21:21.281 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:21.281 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:21.281 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:21.281 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:22.003 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:21:21 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3[49037]: 2024-09-19T09:21:21.694+0000 7f48bb346640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T09:21:22.003 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:21:21 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3[49037]: 2024-09-19T09:21:21.699+0000 7f48b3978640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:21:22.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: Detected new or changed devices on smithi016 2024-09-19T09:21:22.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-19T09:21:22.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-19T09:21:22.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T09:21:22.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:22.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:22.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:21:22.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:21:22.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:21:22.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:22.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' 2024-09-19T09:21:22.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004] boot 2024-09-19T09:21:22.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T09:21:22.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:22 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:22.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: Detected new or changed devices on smithi016 2024-09-19T09:21:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-19T09:21:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-19T09:21:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T09:21:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:21:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:21:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:21:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' 2024-09-19T09:21:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004] boot 2024-09-19T09:21:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T09:21:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:22 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:22.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: Detected new or changed devices on smithi016 2024-09-19T09:21:22.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-19T09:21:22.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-19T09:21:22.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T09:21:22.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:22.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:22.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:21:22.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:21:22.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:21:22.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:22.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: from='osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004]' entity='osd.3' 2024-09-19T09:21:22.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: osd.3 [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004] boot 2024-09-19T09:21:22.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: osdmap e24: 4 total, 4 up, 4 in 2024-09-19T09:21:22.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:22 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T09:21:23.161 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:23 smithi097 ceph-mon[34864]: purged_snaps scrub starts 2024-09-19T09:21:23.162 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:23 smithi097 ceph-mon[34864]: purged_snaps scrub ok 2024-09-19T09:21:23.162 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:23 smithi097 ceph-mon[34864]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:23.162 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:23 smithi097 ceph-mon[34864]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T09:21:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:23 smithi142 ceph-mon[35032]: purged_snaps scrub starts 2024-09-19T09:21:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:23 smithi142 ceph-mon[35032]: purged_snaps scrub ok 2024-09-19T09:21:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:23 smithi142 ceph-mon[35032]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:23.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:23 smithi142 ceph-mon[35032]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T09:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:23 smithi016 ceph-mon[30979]: purged_snaps scrub starts 2024-09-19T09:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:23 smithi016 ceph-mon[30979]: purged_snaps scrub ok 2024-09-19T09:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:23 smithi016 ceph-mon[30979]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-19T09:21:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:23 smithi016 ceph-mon[30979]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T09:21:24.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:24 smithi016 ceph-mon[30979]: from='client.? 172.21.15.97:0/3699836022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc1964f7-af9c-4853-843b-9818f87c7768"}]: dispatch 2024-09-19T09:21:24.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:24 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc1964f7-af9c-4853-843b-9818f87c7768"}]: dispatch 2024-09-19T09:21:24.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:24 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc1964f7-af9c-4853-843b-9818f87c7768"}]': finished 2024-09-19T09:21:24.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:24 smithi016 ceph-mon[30979]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T09:21:24.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:24.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:24 smithi016 ceph-mon[30979]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T09:21:24.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:24.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:24 smithi142 ceph-mon[35032]: from='client.? 172.21.15.97:0/3699836022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc1964f7-af9c-4853-843b-9818f87c7768"}]: dispatch 2024-09-19T09:21:24.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:24 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc1964f7-af9c-4853-843b-9818f87c7768"}]: dispatch 2024-09-19T09:21:24.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:24 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc1964f7-af9c-4853-843b-9818f87c7768"}]': finished 2024-09-19T09:21:24.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:24 smithi142 ceph-mon[35032]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T09:21:24.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:24.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:24 smithi142 ceph-mon[35032]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T09:21:24.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:24.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:24 smithi097 ceph-mon[34864]: from='client.? 172.21.15.97:0/3699836022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc1964f7-af9c-4853-843b-9818f87c7768"}]: dispatch 2024-09-19T09:21:24.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:24 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc1964f7-af9c-4853-843b-9818f87c7768"}]: dispatch 2024-09-19T09:21:24.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:24 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc1964f7-af9c-4853-843b-9818f87c7768"}]': finished 2024-09-19T09:21:24.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:24 smithi097 ceph-mon[34864]: osdmap e26: 5 total, 4 up, 5 in 2024-09-19T09:21:24.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:24.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:24 smithi097 ceph-mon[34864]: osdmap e27: 5 total, 4 up, 5 in 2024-09-19T09:21:24.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:25.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:25 smithi142 ceph-mon[35032]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:25 smithi142 ceph-mon[35032]: from='client.? 172.21.15.97:0/755645680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:21:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:25 smithi142 ceph-mon[35032]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T09:21:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:25 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:25.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:25 smithi097 ceph-mon[34864]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:25.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:25 smithi097 ceph-mon[34864]: from='client.? 172.21.15.97:0/755645680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:21:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:25 smithi097 ceph-mon[34864]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T09:21:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:25 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:25.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:25 smithi016 ceph-mon[30979]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:25.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:25 smithi016 ceph-mon[30979]: from='client.? 172.21.15.97:0/755645680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:21:25.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:25 smithi016 ceph-mon[30979]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T09:21:25.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:25 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:27.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:27 smithi142 ceph-mon[35032]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:27.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:27 smithi097 ceph-mon[34864]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:27.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:27 smithi016 ceph-mon[30979]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 524 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:29.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:29 smithi097 ceph-mon[34864]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-19T09:21:29.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:29 smithi142 ceph-mon[35032]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-19T09:21:29.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:29 smithi016 ceph-mon[30979]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-19T09:21:31.247 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:31 smithi097 ceph-mon[34864]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:21:31.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:31 smithi142 ceph-mon[35032]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:21:31.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:31 smithi016 ceph-mon[30979]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:21:33.318 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:33 smithi097 ceph-mon[34864]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T09:21:33.318 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:33 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T09:21:33.318 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:33 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:33.318 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:33 smithi097 ceph-mon[34864]: Deploying daemon osd.4 on smithi097 2024-09-19T09:21:33.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:33 smithi142 ceph-mon[35032]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T09:21:33.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:33 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T09:21:33.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:33 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:33.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:33 smithi142 ceph-mon[35032]: Deploying daemon osd.4 on smithi097 2024-09-19T09:21:33.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:33 smithi016 ceph-mon[30979]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-19T09:21:33.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:33 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T09:21:33.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:33 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:33.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:33 smithi016 ceph-mon[30979]: Deploying daemon osd.4 on smithi097 2024-09-19T09:21:35.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:35 smithi142 ceph-mon[35032]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T09:21:35.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:35 smithi097 ceph-mon[34864]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T09:21:35.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:35 smithi016 ceph-mon[30979]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T09:21:37.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:37 smithi097 ceph-mon[34864]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T09:21:37.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:37 smithi142 ceph-mon[35032]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T09:21:37.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:37 smithi016 ceph-mon[30979]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T09:21:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:39 smithi142 ceph-mon[35032]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:21:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:39.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:39.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:39 smithi097 ceph-mon[34864]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:21:39.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:39.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:39.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:39 smithi016 ceph-mon[30979]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:21:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:41.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:41 smithi097 ceph-mon[34864]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:41.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:41.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:41.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:41.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:41.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:41.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:41 smithi142 ceph-mon[35032]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:41.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:41 smithi016 ceph-mon[30979]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:41.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:41.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:41.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:41.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:41.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:41.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:41.914+0000 7f5924ff9640 1 -- 172.21.15.97:0/2964573897 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5928062630 con 0x7f59100a32c0 2024-09-19T09:21:41.915 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 4 on host 'smithi097' 2024-09-19T09:21:41.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:41.916+0000 7f592df7a640 1 -- 172.21.15.97:0/2964573897 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f59100a32c0 msgr2=0x7f59100a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:41.916+0000 7f592df7a640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f59100a32c0 0x7f59100a5780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5918009bd0 tx=0x7f5918005f90 comp rx=0 tx=0).stop 2024-09-19T09:21:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:41.916+0000 7f592df7a640 1 -- 172.21.15.97:0/2964573897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928105010 msgr2=0x7f5928100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:41.916+0000 7f592df7a640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928105010 0x7f5928100690 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f591c005fd0 tx=0x7f591c00dde0 comp rx=0 tx=0).stop 2024-09-19T09:21:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:41.917+0000 7f592df7a640 1 -- 172.21.15.97:0/2964573897 shutdown_connections 2024-09-19T09:21:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:41.917+0000 7f592df7a640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f59100a32c0 0x7f59100a5780 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:41.917+0000 7f592df7a640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5928107940 0x7f5928100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:41.917+0000 7f592df7a640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5928105010 0x7f5928100690 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:41.917+0000 7f592df7a640 1 --2- 172.21.15.97:0/2964573897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59281026e0 0x7f5928102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:41.917+0000 7f592df7a640 1 -- 172.21.15.97:0/2964573897 >> 172.21.15.97:0/2964573897 conn(0x7f59280fc280 msgr2=0x7f5928102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:41.917+0000 7f592df7a640 1 -- 172.21.15.97:0/2964573897 shutdown_connections 2024-09-19T09:21:41.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:21:41.917+0000 7f592df7a640 1 -- 172.21.15.97:0/2964573897 wait complete. 2024-09-19T09:21:42.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:42.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:42.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:42.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:42.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:42.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:42.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:21:42.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:42.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:42.554 DEBUG:teuthology.orchestra.run.smithi097:osd.4> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T09:21:42.556 INFO:tasks.cephadm:Deploying osd.5 on smithi142 with /dev/vg_nvme/lv_3... 2024-09-19T09:21:42.556 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T09:21:42.814 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:21:43.039 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:43 smithi097 ceph-mon[34864]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:43.200 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:43 smithi142 ceph-mon[35032]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:43.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:43 smithi016 ceph-mon[30979]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:44.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:44 smithi097 ceph-mon[34864]: from='osd.4 [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T09:21:44.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:44 smithi097 ceph-mon[34864]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T09:21:44.136 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:21:44 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:21:44.020+0000 7fc7cdc03640 -1 osd.4 0 log_to_monitors true 2024-09-19T09:21:44.194 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:44 smithi142 ceph-mon[35032]: from='osd.4 [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T09:21:44.194 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:44 smithi142 ceph-mon[35032]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T09:21:44.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:44 smithi016 ceph-mon[30979]: from='osd.4 [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T09:21:44.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:44 smithi016 ceph-mon[30979]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T09:21:45.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:45 smithi097 ceph-mon[34864]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:45.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:45 smithi097 ceph-mon[34864]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T09:21:45.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:45 smithi097 ceph-mon[34864]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T09:21:45.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:45 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:45.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:45 smithi097 ceph-mon[34864]: from='osd.4 [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:21:45.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:45 smithi097 ceph-mon[34864]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:21:45.276 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:45 smithi142 ceph-mon[35032]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:45.276 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:45 smithi142 ceph-mon[35032]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T09:21:45.276 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:45 smithi142 ceph-mon[35032]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T09:21:45.276 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:45 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:45.276 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:45 smithi142 ceph-mon[35032]: from='osd.4 [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:21:45.276 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:45 smithi142 ceph-mon[35032]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:21:45.278 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:21:45.304 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2024-09-19T09:21:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:45 smithi016 ceph-mon[30979]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:45 smithi016 ceph-mon[30979]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T09:21:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:45 smithi016 ceph-mon[30979]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T09:21:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:45 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:45 smithi016 ceph-mon[30979]: from='osd.4 [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:21:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:45 smithi016 ceph-mon[30979]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:21:45.556 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:21:46.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.022+0000 7f98d5644640 1 -- 172.21.15.142:0/1801408790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d01026c0 msgr2=0x7f98d0104ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:46.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.022+0000 7f98d5644640 1 --2- 172.21.15.142:0/1801408790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d01026c0 0x7f98d0104ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009900 tx=0x7f98c002a410 comp rx=0 tx=0).stop 2024-09-19T09:21:46.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.022+0000 7f98d5644640 1 -- 172.21.15.142:0/1801408790 shutdown_connections 2024-09-19T09:21:46.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.022+0000 7f98d5644640 1 --2- 172.21.15.142:0/1801408790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98d0107920 0x7f98d0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:46.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.022+0000 7f98d5644640 1 --2- 172.21.15.142:0/1801408790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98d0104ff0 0x7f98d01073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:46.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.022+0000 7f98d5644640 1 --2- 172.21.15.142:0/1801408790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d01026c0 0x7f98d0104ab0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:46.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.022+0000 7f98d5644640 1 -- 172.21.15.142:0/1801408790 >> 172.21.15.142:0/1801408790 conn(0x7f98d00fc280 msgr2=0x7f98d00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:21:46.026 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.022+0000 7f98d5644640 1 -- 172.21.15.142:0/1801408790 shutdown_connections 2024-09-19T09:21:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.022+0000 7f98d5644640 1 -- 172.21.15.142:0/1801408790 wait complete. 2024-09-19T09:21:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.023+0000 7f98d5644640 1 Processor -- start 2024-09-19T09:21:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.023+0000 7f98d5644640 1 -- start start 2024-09-19T09:21:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.023+0000 7f98d5644640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98d01026c0 0x7f98d0102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.023+0000 7f98d5644640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d0104ff0 0x7f98d0100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.023+0000 7f98d5644640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98d0107920 0x7f98d0100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.023+0000 7f98d5644640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98d010d880 con 0x7f98d01026c0 2024-09-19T09:21:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.023+0000 7f98d5644640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f98d010d700 con 0x7f98d0107920 2024-09-19T09:21:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.023+0000 7f98d5644640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98d010da00 con 0x7f98d0104ff0 2024-09-19T09:21:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98ce7fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d0104ff0 0x7f98d0100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98ce7fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d0104ff0 0x7f98d0100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:60108/0 (socket says 172.21.15.142:60108) 2024-09-19T09:21:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98ce7fc640 1 -- 172.21.15.142:0/3424034771 learned_addr learned my addr 172.21.15.142:0/3424034771 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-19T09:21:46.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98cf7fe640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98d0107920 0x7f98d0100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98ceffd640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98d01026c0 0x7f98d0102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98ce7fc640 1 -- 172.21.15.142:0/3424034771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98d0107920 msgr2=0x7f98d0100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98ce7fc640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98d0107920 0x7f98d0100bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98ce7fc640 1 -- 172.21.15.142:0/3424034771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98d01026c0 msgr2=0x7f98d0102020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:21:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98ce7fc640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98d01026c0 0x7f98d0102020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:21:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98ce7fc640 1 -- 172.21.15.142:0/3424034771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d0101470 con 0x7f98d0104ff0 2024-09-19T09:21:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98cf7fe640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98d0107920 0x7f98d0100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:21:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98ce7fc640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d0104ff0 0x7f98d0100670 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98c800ea50 tx=0x7f98c800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98b7fff640 1 -- 172.21.15.142:0/3424034771 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c800cdc0 con 0x7f98d0104ff0 2024-09-19T09:21:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98d5644640 1 -- 172.21.15.142:0/3424034771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d01a20f0 con 0x7f98d0104ff0 2024-09-19T09:21:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98b7fff640 1 -- 172.21.15.142:0/3424034771 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c8005020 con 0x7f98d0104ff0 2024-09-19T09:21:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98b7fff640 1 -- 172.21.15.142:0/3424034771 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8011690 con 0x7f98d0104ff0 2024-09-19T09:21:46.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.024+0000 7f98d5644640 1 -- 172.21.15.142:0/3424034771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d01a2600 con 0x7f98d0104ff0 2024-09-19T09:21:46.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.025+0000 7f98d5644640 1 -- 172.21.15.142:0/3424034771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d01053d0 con 0x7f98d0104ff0 2024-09-19T09:21:46.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.026+0000 7f98b7fff640 1 -- 172.21.15.142:0/3424034771 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f98c8004b20 con 0x7f98d0104ff0 2024-09-19T09:21:46.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.027+0000 7f98b7fff640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f98b80a32c0 0x7f98b80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:21:46.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.027+0000 7f98b7fff640 1 -- 172.21.15.142:0/3424034771 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3821+0+0 (secure 0 0 0) 0x7f98c80ce920 con 0x7f98d0104ff0 2024-09-19T09:21:46.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.027+0000 7f98ceffd640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f98b80a32c0 0x7f98b80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:21:46.030 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.028+0000 7f98ceffd640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f98b80a32c0 0x7f98b80a5780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f98c00314e0 tx=0x7f98c0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:21:46.032 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.032+0000 7f98b7fff640 1 -- 172.21.15.142:0/3424034771 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f98c8016030 con 0x7f98d0104ff0 2024-09-19T09:21:46.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-19T09:21:46.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T09:21:46.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:46.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:46.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: Detected new or changed devices on smithi097 2024-09-19T09:21:46.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:46.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:46.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:21:46.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:21:46.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: Adjusting osd_memory_target on smithi097 to 8544M 2024-09-19T09:21:46.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:46.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:46.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:46.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:46.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:46 smithi142 ceph-mon[35032]: from='osd.4 ' entity='osd.4' 2024-09-19T09:21:46.214 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:21:46.213+0000 7f98d5644640 1 -- 172.21.15.142:0/3424034771 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f98d0062630 con 0x7f98b80a32c0 2024-09-19T09:21:46.385 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:21:45 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:21:45.906+0000 7fc7c9363640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T09:21:46.386 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:21:45 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:21:45.916+0000 7fc7c1995640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:21:46.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-19T09:21:46.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T09:21:46.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:46.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:46.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: Detected new or changed devices on smithi097 2024-09-19T09:21:46.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:46.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:46.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:21:46.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:21:46.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: Adjusting osd_memory_target on smithi097 to 8544M 2024-09-19T09:21:46.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:46.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:46.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:46.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:46.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:46 smithi097 ceph-mon[34864]: from='osd.4 ' entity='osd.4' 2024-09-19T09:21:46.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-19T09:21:46.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T09:21:46.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:46.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:46.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: Detected new or changed devices on smithi097 2024-09-19T09:21:46.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:46.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:46.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:21:46.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:21:46.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: Adjusting osd_memory_target on smithi097 to 8544M 2024-09-19T09:21:46.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:46.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:46.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:21:46.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:21:46.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:46 smithi016 ceph-mon[30979]: from='osd.4 ' entity='osd.4' 2024-09-19T09:21:47.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:47 smithi142 ceph-mon[35032]: purged_snaps scrub starts 2024-09-19T09:21:47.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:47 smithi142 ceph-mon[35032]: purged_snaps scrub ok 2024-09-19T09:21:47.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:47 smithi142 ceph-mon[35032]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:47 smithi142 ceph-mon[35032]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:21:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:21:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:21:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:47 smithi142 ceph-mon[35032]: osd.4 [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825] boot 2024-09-19T09:21:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:47 smithi142 ceph-mon[35032]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T09:21:47.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:47.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:47 smithi097 ceph-mon[34864]: purged_snaps scrub starts 2024-09-19T09:21:47.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:47 smithi097 ceph-mon[34864]: purged_snaps scrub ok 2024-09-19T09:21:47.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:47 smithi097 ceph-mon[34864]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:47.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:47.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:47 smithi097 ceph-mon[34864]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:21:47.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:21:47.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:21:47.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:47.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:47 smithi097 ceph-mon[34864]: osd.4 [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825] boot 2024-09-19T09:21:47.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:47 smithi097 ceph-mon[34864]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T09:21:47.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:47 smithi016 ceph-mon[30979]: purged_snaps scrub starts 2024-09-19T09:21:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:47 smithi016 ceph-mon[30979]: purged_snaps scrub ok 2024-09-19T09:21:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:47 smithi016 ceph-mon[30979]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-19T09:21:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:47 smithi016 ceph-mon[30979]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:21:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:21:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:21:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:47 smithi016 ceph-mon[30979]: osd.4 [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825] boot 2024-09-19T09:21:47.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:47 smithi016 ceph-mon[30979]: osdmap e31: 5 total, 5 up, 5 in 2024-09-19T09:21:47.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T09:21:49.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:49 smithi142 ceph-mon[35032]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 559 MiB used, 446 GiB / 447 GiB avail 2024-09-19T09:21:49.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:49 smithi142 ceph-mon[35032]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T09:21:49.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:49 smithi142 ceph-mon[35032]: from='client.? 172.21.15.142:0/2145381880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7fbafc-d9ba-4036-a89d-aa295b8a23a7"}]: dispatch 2024-09-19T09:21:49.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:49 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7fbafc-d9ba-4036-a89d-aa295b8a23a7"}]: dispatch 2024-09-19T09:21:49.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:49 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d7fbafc-d9ba-4036-a89d-aa295b8a23a7"}]': finished 2024-09-19T09:21:49.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:49 smithi142 ceph-mon[35032]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T09:21:49.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:49 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:21:49.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:49 smithi097 ceph-mon[34864]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 559 MiB used, 446 GiB / 447 GiB avail 2024-09-19T09:21:49.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:49 smithi097 ceph-mon[34864]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T09:21:49.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:49 smithi097 ceph-mon[34864]: from='client.? 172.21.15.142:0/2145381880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7fbafc-d9ba-4036-a89d-aa295b8a23a7"}]: dispatch 2024-09-19T09:21:49.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:49 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7fbafc-d9ba-4036-a89d-aa295b8a23a7"}]: dispatch 2024-09-19T09:21:49.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:49 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d7fbafc-d9ba-4036-a89d-aa295b8a23a7"}]': finished 2024-09-19T09:21:49.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:49 smithi097 ceph-mon[34864]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T09:21:49.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:49 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:21:49.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:49 smithi016 ceph-mon[30979]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 559 MiB used, 446 GiB / 447 GiB avail 2024-09-19T09:21:49.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:49 smithi016 ceph-mon[30979]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T09:21:49.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:49 smithi016 ceph-mon[30979]: from='client.? 172.21.15.142:0/2145381880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7fbafc-d9ba-4036-a89d-aa295b8a23a7"}]: dispatch 2024-09-19T09:21:49.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:49 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7fbafc-d9ba-4036-a89d-aa295b8a23a7"}]: dispatch 2024-09-19T09:21:49.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:49 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d7fbafc-d9ba-4036-a89d-aa295b8a23a7"}]': finished 2024-09-19T09:21:49.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:49 smithi016 ceph-mon[30979]: osdmap e33: 6 total, 5 up, 6 in 2024-09-19T09:21:49.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:49 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:21:50.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:50 smithi142 ceph-mon[35032]: from='client.? 172.21.15.142:0/2743185472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:21:50.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:50 smithi097 ceph-mon[34864]: from='client.? 172.21.15.142:0/2743185472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:21:50.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:50 smithi016 ceph-mon[30979]: from='client.? 172.21.15.142:0/2743185472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:21:51.339 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:51 smithi097 ceph-mon[34864]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 559 MiB used, 446 GiB / 447 GiB avail 2024-09-19T09:21:51.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:51 smithi142 ceph-mon[35032]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 559 MiB used, 446 GiB / 447 GiB avail 2024-09-19T09:21:51.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:51 smithi016 ceph-mon[30979]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 559 MiB used, 446 GiB / 447 GiB avail 2024-09-19T09:21:53.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:53 smithi142 ceph-mon[35032]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 559 MiB used, 446 GiB / 447 GiB avail 2024-09-19T09:21:53.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:53 smithi097 ceph-mon[34864]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 559 MiB used, 446 GiB / 447 GiB avail 2024-09-19T09:21:53.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:53 smithi016 ceph-mon[30979]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 559 MiB used, 446 GiB / 447 GiB avail 2024-09-19T09:21:55.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:55 smithi142 ceph-mon[35032]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:21:55.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:55 smithi097 ceph-mon[34864]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:21:55.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:55 smithi016 ceph-mon[30979]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:21:57.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:57 smithi142 ceph-mon[35032]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:21:57.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:57 smithi097 ceph-mon[34864]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:21:57.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:57 smithi016 ceph-mon[30979]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:21:58.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:58 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T09:21:58.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:58 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:58.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:58 smithi142 ceph-mon[35032]: Deploying daemon osd.5 on smithi142 2024-09-19T09:21:58.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:58 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T09:21:58.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:58 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:58.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:58 smithi097 ceph-mon[34864]: Deploying daemon osd.5 on smithi142 2024-09-19T09:21:58.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:58 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T09:21:58.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:58 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:21:58.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:58 smithi016 ceph-mon[30979]: Deploying daemon osd.5 on smithi142 2024-09-19T09:21:59.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:21:59 smithi142 ceph-mon[35032]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:21:59.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:21:59 smithi097 ceph-mon[34864]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:21:59.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:21:59 smithi016 ceph-mon[30979]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:01.363 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:01 smithi142 ceph-mon[35032]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:01.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:01 smithi097 ceph-mon[34864]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:01 smithi016 ceph-mon[30979]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:03.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:03 smithi142 ceph-mon[35032]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:03.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:03 smithi097 ceph-mon[34864]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:03.502 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:03 smithi016 ceph-mon[30979]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:04.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:04 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:04.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:04 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:04.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:04 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:04.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:04 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:04.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:04 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:04.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:04 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:04 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:04 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:04 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:05.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:05 smithi142 ceph-mon[35032]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:05.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:05 smithi016 ceph-mon[30979]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:05.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:05 smithi097 ceph-mon[34864]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:06.980 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:06 smithi142 ceph-mon[35032]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:06.980 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:06.980 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:06.981 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:06.981 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:06.981 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:06 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:07.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:06 smithi016 ceph-mon[30979]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:07.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:07.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:07.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:07.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:07.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:06 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:07.021 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:22:07.020+0000 7f98b7fff640 1 -- 172.21.15.142:0/3424034771 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f98d0062630 con 0x7f98b80a32c0 2024-09-19T09:22:07.021 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 5 on host 'smithi142' 2024-09-19T09:22:07.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:22:07.022+0000 7f98d5644640 1 -- 172.21.15.142:0/3424034771 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f98b80a32c0 msgr2=0x7f98b80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:07.023 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:22:07.022+0000 7f98d5644640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f98b80a32c0 0x7f98b80a5780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f98c00314e0 tx=0x7f98c0000bc0 comp rx=0 tx=0).stop 2024-09-19T09:22:07.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:22:07.022+0000 7f98d5644640 1 -- 172.21.15.142:0/3424034771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d0104ff0 msgr2=0x7f98d0100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:07.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:22:07.022+0000 7f98d5644640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d0104ff0 0x7f98d0100670 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98c800ea50 tx=0x7f98c800ef10 comp rx=0 tx=0).stop 2024-09-19T09:22:07.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:22:07.022+0000 7f98d5644640 1 -- 172.21.15.142:0/3424034771 shutdown_connections 2024-09-19T09:22:07.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:22:07.022+0000 7f98d5644640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f98b80a32c0 0x7f98b80a5780 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:07.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:22:07.022+0000 7f98d5644640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d0104ff0 0x7f98d0100670 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:07.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:22:07.022+0000 7f98d5644640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98d0107920 0x7f98d0100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:07.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:22:07.022+0000 7f98d5644640 1 --2- 172.21.15.142:0/3424034771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98d01026c0 0x7f98d0102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:07.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:22:07.022+0000 7f98d5644640 1 -- 172.21.15.142:0/3424034771 >> 172.21.15.142:0/3424034771 conn(0x7f98d00fc280 msgr2=0x7f98d0102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:07.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:22:07.023+0000 7f98d5644640 1 -- 172.21.15.142:0/3424034771 shutdown_connections 2024-09-19T09:22:07.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:22:07.023+0000 7f98d5644640 1 -- 172.21.15.142:0/3424034771 wait complete. 2024-09-19T09:22:07.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:06 smithi097 ceph-mon[34864]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:07.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:07.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:07.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:07.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:07.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:06 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:07.620 DEBUG:teuthology.orchestra.run.smithi142:osd.5> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T09:22:07.632 INFO:tasks.cephadm:Deploying osd.6 on smithi016 with /dev/vg_nvme/lv_2... 2024-09-19T09:22:07.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T09:22:07.855 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:07.855 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:07.856 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:08.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:08.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:08.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:08.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:08.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:08.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:09.085 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:09 smithi142 ceph-mon[35032]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:09.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:09 smithi016 ceph-mon[30979]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:09.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:09 smithi097 ceph-mon[34864]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:09.799 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:22:09 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:22:09.435+0000 7f5b46735640 -1 osd.5 0 log_to_monitors true 2024-09-19T09:22:10.112 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:10 smithi142 ceph-mon[35032]: from='osd.5 [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T09:22:10.112 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:10 smithi142 ceph-mon[35032]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T09:22:10.323 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:10 smithi016 ceph-mon[30979]: from='osd.5 [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T09:22:10.323 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:10 smithi016 ceph-mon[30979]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T09:22:10.326 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:22:10.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-09-19T09:22:10.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:10 smithi097 ceph-mon[34864]: from='osd.5 [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T09:22:10.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:10 smithi097 ceph-mon[34864]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T09:22:10.661 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:22:11.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.084+0000 7ff8d2267640 1 -- 172.21.15.16:0/3293303344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc102690 msgr2=0x7ff8cc108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:11.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.084+0000 7ff8d2267640 1 --2- 172.21.15.16:0/3293303344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc102690 0x7ff8cc108f20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009ca0 tx=0x7ff8c402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:22:11.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.084+0000 7ff8d2267640 1 -- 172.21.15.16:0/3293303344 shutdown_connections 2024-09-19T09:22:11.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.084+0000 7ff8d2267640 1 --2- 172.21.15.16:0/3293303344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8cc101cd0 0x7ff8cc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:11.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.084+0000 7ff8d2267640 1 --2- 172.21.15.16:0/3293303344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc102690 0x7ff8cc108f20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:11.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.084+0000 7ff8d2267640 1 --2- 172.21.15.16:0/3293303344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8cc100ad0 0x7ff8cc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:11.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.084+0000 7ff8d2267640 1 -- 172.21.15.16:0/3293303344 >> 172.21.15.16:0/3293303344 conn(0x7ff8cc0fc280 msgr2=0x7ff8cc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:11.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.084+0000 7ff8d2267640 1 -- 172.21.15.16:0/3293303344 shutdown_connections 2024-09-19T09:22:11.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.084+0000 7ff8d2267640 1 -- 172.21.15.16:0/3293303344 wait complete. 2024-09-19T09:22:11.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.084+0000 7ff8d2267640 1 Processor -- start 2024-09-19T09:22:11.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.084+0000 7ff8d2267640 1 -- start start 2024-09-19T09:22:11.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d2267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8cc100ad0 0x7ff8cc104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:11.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d2267640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8cc101cd0 0x7ff8cc1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:11.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d2267640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc102690 0x7ff8cc1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:11.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d2267640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc10c750 con 0x7ff8cc102690 2024-09-19T09:22:11.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d2267640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc10c5d0 con 0x7ff8cc100ad0 2024-09-19T09:22:11.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d2267640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc10c8d0 con 0x7ff8cc101cd0 2024-09-19T09:22:11.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d0804640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc102690 0x7ff8cc1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:11.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d0804640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc102690 0x7ff8cc1035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60024/0 (socket says 172.21.15.16:60024) 2024-09-19T09:22:11.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d0804640 1 -- 172.21.15.16:0/2806071917 learned_addr learned my addr 172.21.15.16:0/2806071917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:22:11.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8cb7fe640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8cc101cd0 0x7ff8cc1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:11.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8cbfff640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8cc100ad0 0x7ff8cc104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:11.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d0804640 1 -- 172.21.15.16:0/2806071917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8cc101cd0 msgr2=0x7ff8cc1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:11.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d0804640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8cc101cd0 0x7ff8cc1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:11.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d0804640 1 -- 172.21.15.16:0/2806071917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8cc100ad0 msgr2=0x7ff8cc104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:11.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d0804640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8cc100ad0 0x7ff8cc104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:11.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d0804640 1 -- 172.21.15.16:0/2806071917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc103eb0 con 0x7ff8cc102690 2024-09-19T09:22:11.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.085+0000 7ff8d0804640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc102690 0x7ff8cc1035f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4031ae0 tx=0x7ff8c4031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:11.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.086+0000 7ff8c97fa640 1 -- 172.21.15.16:0/2806071917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4003c30 con 0x7ff8cc102690 2024-09-19T09:22:11.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.086+0000 7ff8d2267640 1 -- 172.21.15.16:0/2806071917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc1a1fc0 con 0x7ff8cc102690 2024-09-19T09:22:11.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.086+0000 7ff8c97fa640 1 -- 172.21.15.16:0/2806071917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8c4003dd0 con 0x7ff8cc102690 2024-09-19T09:22:11.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.086+0000 7ff8c97fa640 1 -- 172.21.15.16:0/2806071917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c402fcb0 con 0x7ff8cc102690 2024-09-19T09:22:11.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.086+0000 7ff8d2267640 1 -- 172.21.15.16:0/2806071917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc1a2470 con 0x7ff8cc102690 2024-09-19T09:22:11.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.087+0000 7ff8d2267640 1 -- 172.21.15.16:0/2806071917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8cc1a26a0 con 0x7ff8cc102690 2024-09-19T09:22:11.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.088+0000 7ff8c97fa640 1 -- 172.21.15.16:0/2806071917 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff8c4004050 con 0x7ff8cc102690 2024-09-19T09:22:11.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.089+0000 7ff8c97fa640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8b40a3240 0x7ff8b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:11.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.089+0000 7ff8c97fa640 1 -- 172.21.15.16:0/2806071917 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4253+0+0 (secure 0 0 0) 0x7ff8c40efd90 con 0x7ff8cc102690 2024-09-19T09:22:11.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.089+0000 7ff8cbfff640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8b40a3240 0x7ff8b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:11.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.089+0000 7ff8cbfff640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8b40a3240 0x7ff8b40a5700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc101b30 tx=0x7ff8bc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:11.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.093+0000 7ff8c97fa640 1 -- 172.21.15.16:0/2806071917 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff8c40bda90 con 0x7ff8cc102690 2024-09-19T09:22:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T09:22:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T09:22:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: from='osd.5 [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:22:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:22:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:11.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:11.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:11.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:11.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:11.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:11.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:11.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:11.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:11.306+0000 7ff8d2267640 1 -- 172.21.15.16:0/2806071917 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff8cc0fe490 con 0x7ff8b40a3240 2024-09-19T09:22:11.378 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:22:11.032+0000 7f5b41e95640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T09:22:11.378 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:22:11.040+0000 7f5b3e4cf640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:22:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T09:22:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T09:22:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: from='osd.5 [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:22:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:22:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:11.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:11.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:11.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T09:22:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T09:22:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: from='osd.5 [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:22:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:22:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:12.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:12 smithi142 ceph-mon[35032]: Detected new or changed devices on smithi142 2024-09-19T09:22:12.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:12 smithi142 ceph-mon[35032]: Adjusting osd_memory_target on smithi142 to 8544M 2024-09-19T09:22:12.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:12 smithi142 ceph-mon[35032]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-19T09:22:12.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:12 smithi142 ceph-mon[35032]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T09:22:12.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:12 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:12.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:12 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:12.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:12 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:22:12.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:12 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:22:12.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:12 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:12.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:12 smithi097 ceph-mon[34864]: Detected new or changed devices on smithi142 2024-09-19T09:22:12.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:12 smithi097 ceph-mon[34864]: Adjusting osd_memory_target on smithi142 to 8544M 2024-09-19T09:22:12.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:12 smithi097 ceph-mon[34864]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-19T09:22:12.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:12 smithi097 ceph-mon[34864]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T09:22:12.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:12 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:12.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:12 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:12.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:12 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:22:12.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:12 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:22:12.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:12 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:12 smithi016 ceph-mon[30979]: Detected new or changed devices on smithi142 2024-09-19T09:22:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:12 smithi016 ceph-mon[30979]: Adjusting osd_memory_target on smithi142 to 8544M 2024-09-19T09:22:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:12 smithi016 ceph-mon[30979]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-19T09:22:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:12 smithi016 ceph-mon[30979]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T09:22:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:12 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:12 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:12 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:22:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:12 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:22:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:12 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:13.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:13 smithi016 ceph-mon[30979]: purged_snaps scrub starts 2024-09-19T09:22:13.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:13 smithi016 ceph-mon[30979]: purged_snaps scrub ok 2024-09-19T09:22:13.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:13 smithi016 ceph-mon[30979]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:13.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:13 smithi016 ceph-mon[30979]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:22:13.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:13 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:13.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:13 smithi016 ceph-mon[30979]: osd.5 [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986] boot 2024-09-19T09:22:13.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:13 smithi016 ceph-mon[30979]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T09:22:13.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:13 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:13.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:13 smithi142 ceph-mon[35032]: purged_snaps scrub starts 2024-09-19T09:22:13.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:13 smithi142 ceph-mon[35032]: purged_snaps scrub ok 2024-09-19T09:22:13.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:13 smithi142 ceph-mon[35032]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:13.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:13 smithi142 ceph-mon[35032]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:22:13.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:13 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:13.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:13 smithi142 ceph-mon[35032]: osd.5 [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986] boot 2024-09-19T09:22:13.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:13 smithi142 ceph-mon[35032]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T09:22:13.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:13 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:13.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:13 smithi097 ceph-mon[34864]: purged_snaps scrub starts 2024-09-19T09:22:13.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:13 smithi097 ceph-mon[34864]: purged_snaps scrub ok 2024-09-19T09:22:13.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:13 smithi097 ceph-mon[34864]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-19T09:22:13.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:13 smithi097 ceph-mon[34864]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:22:13.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:13 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:13.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:13 smithi097 ceph-mon[34864]: osd.5 [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986] boot 2024-09-19T09:22:13.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:13 smithi097 ceph-mon[34864]: osdmap e36: 6 total, 6 up, 6 in 2024-09-19T09:22:13.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:13 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T09:22:14.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:14 smithi016 ceph-mon[30979]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T09:22:14.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:14 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2510450404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3f346fc-c027-44f8-bd37-bccf6ec09e3b"}]: dispatch 2024-09-19T09:22:14.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:14 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2510450404' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3f346fc-c027-44f8-bd37-bccf6ec09e3b"}]': finished 2024-09-19T09:22:14.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:14 smithi016 ceph-mon[30979]: osdmap e38: 7 total, 6 up, 7 in 2024-09-19T09:22:14.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:14.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:14 smithi016 ceph-mon[30979]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T09:22:14.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:14.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:14 smithi142 ceph-mon[35032]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T09:22:14.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:14 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2510450404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3f346fc-c027-44f8-bd37-bccf6ec09e3b"}]: dispatch 2024-09-19T09:22:14.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:14 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2510450404' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3f346fc-c027-44f8-bd37-bccf6ec09e3b"}]': finished 2024-09-19T09:22:14.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:14 smithi142 ceph-mon[35032]: osdmap e38: 7 total, 6 up, 7 in 2024-09-19T09:22:14.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:14.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:14 smithi142 ceph-mon[35032]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T09:22:14.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:14.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:14 smithi097 ceph-mon[34864]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T09:22:14.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:14 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2510450404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3f346fc-c027-44f8-bd37-bccf6ec09e3b"}]: dispatch 2024-09-19T09:22:14.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:14 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2510450404' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3f346fc-c027-44f8-bd37-bccf6ec09e3b"}]': finished 2024-09-19T09:22:14.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:14 smithi097 ceph-mon[34864]: osdmap e38: 7 total, 6 up, 7 in 2024-09-19T09:22:14.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:14.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:14 smithi097 ceph-mon[34864]: osdmap e39: 7 total, 6 up, 7 in 2024-09-19T09:22:14.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:15.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:15 smithi142 ceph-mon[35032]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:15.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:15 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/931652800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:22:15.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:15 smithi142 ceph-mon[35032]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T09:22:15.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:15 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:15.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:15 smithi097 ceph-mon[34864]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:15.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:15 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/931652800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:22:15.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:15 smithi097 ceph-mon[34864]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T09:22:15.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:15 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:15 smithi016 ceph-mon[30979]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:15 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/931652800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:22:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:15 smithi016 ceph-mon[30979]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T09:22:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:15 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:17.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:17 smithi142 ceph-mon[35032]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:17.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:17 smithi097 ceph-mon[34864]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:17.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:17 smithi016 ceph-mon[30979]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:19.149 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:19 smithi016 ceph-mon[30979]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-19T09:22:19.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:19 smithi142 ceph-mon[35032]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-19T09:22:19.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:19 smithi097 ceph-mon[34864]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-19T09:22:21.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:21 smithi142 ceph-mon[35032]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:22:21.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:21 smithi097 ceph-mon[34864]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:22:21.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:21 smithi016 ceph-mon[30979]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:22:22.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:22 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T09:22:22.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:22 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:22.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:22 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T09:22:22.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:22 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:22.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:22 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T09:22:22.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:22 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:23 smithi142 ceph-mon[35032]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-19T09:22:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:23 smithi142 ceph-mon[35032]: Deploying daemon osd.6 on smithi016 2024-09-19T09:22:23.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:23 smithi097 ceph-mon[34864]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-19T09:22:23.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:23 smithi097 ceph-mon[34864]: Deploying daemon osd.6 on smithi016 2024-09-19T09:22:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:23 smithi016 ceph-mon[30979]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-19T09:22:23.491 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:23 smithi016 ceph-mon[30979]: Deploying daemon osd.6 on smithi016 2024-09-19T09:22:25.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:25 smithi016 ceph-mon[30979]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T09:22:25.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:25 smithi142 ceph-mon[35032]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T09:22:25.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:25 smithi097 ceph-mon[34864]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T09:22:27.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:27 smithi016 ceph-mon[30979]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T09:22:27.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:27 smithi142 ceph-mon[35032]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T09:22:27.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:27 smithi097 ceph-mon[34864]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T09:22:28.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:28.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:28.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:28.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:28 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:28.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:28 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:28.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:28 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:28.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:28.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:28.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:29.250 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:29 smithi016 ceph-mon[30979]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:22:29.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:29 smithi142 ceph-mon[35032]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:22:29.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:29 smithi097 ceph-mon[34864]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:22:31.193 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:31 smithi016 ceph-mon[30979]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:31.194 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:31.194 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:31.194 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:31.194 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:31.194 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:31 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:31.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:31 smithi142 ceph-mon[35032]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:31.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:31.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:31.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:31.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:31.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:31 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:31.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:31 smithi097 ceph-mon[34864]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:31.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:31.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:31.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:31.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:31.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:31 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:31.535 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 6 on host 'smithi016' 2024-09-19T09:22:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:31.534+0000 7ff8c97fa640 1 -- 172.21.15.16:0/2806071917 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff8cc0fe490 con 0x7ff8b40a3240 2024-09-19T09:22:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:31.536+0000 7ff8d2267640 1 -- 172.21.15.16:0/2806071917 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8b40a3240 msgr2=0x7ff8b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:31.536+0000 7ff8d2267640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8b40a3240 0x7ff8b40a5700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc101b30 tx=0x7ff8bc00b390 comp rx=0 tx=0).stop 2024-09-19T09:22:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:31.536+0000 7ff8d2267640 1 -- 172.21.15.16:0/2806071917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc102690 msgr2=0x7ff8cc1035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:31.536+0000 7ff8d2267640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc102690 0x7ff8cc1035f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4031ae0 tx=0x7ff8c4031ee0 comp rx=0 tx=0).stop 2024-09-19T09:22:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:31.537+0000 7ff8d2267640 1 -- 172.21.15.16:0/2806071917 shutdown_connections 2024-09-19T09:22:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:31.537+0000 7ff8d2267640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff8b40a3240 0x7ff8b40a5700 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:31.537+0000 7ff8d2267640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8cc101cd0 0x7ff8cc1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:31.537+0000 7ff8d2267640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8cc102690 0x7ff8cc1035f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:31.537+0000 7ff8d2267640 1 --2- 172.21.15.16:0/2806071917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8cc100ad0 0x7ff8cc104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:31.537+0000 7ff8d2267640 1 -- 172.21.15.16:0/2806071917 >> 172.21.15.16:0/2806071917 conn(0x7ff8cc0fc280 msgr2=0x7ff8cc0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:31.537+0000 7ff8d2267640 1 -- 172.21.15.16:0/2806071917 shutdown_connections 2024-09-19T09:22:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:22:31.537+0000 7ff8d2267640 1 -- 172.21.15.16:0/2806071917 wait complete. 2024-09-19T09:22:32.118 DEBUG:teuthology.orchestra.run.smithi016:osd.6> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T09:22:32.121 INFO:tasks.cephadm:Deploying osd.7 on smithi097 with /dev/vg_nvme/lv_2... 2024-09-19T09:22:32.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T09:22:32.377 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:22:32.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:32 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:32.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:32 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:32.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:32 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:32.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:32 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:32.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:32 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:32.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:32 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:32.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:32 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:32.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:32 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:32.398 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:32 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:33.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:33 smithi097 ceph-mon[34864]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:33.400 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:33 smithi016 ceph-mon[30979]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:33.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:33 smithi142 ceph-mon[35032]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:34.485 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:22:34 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:22:34.191+0000 7f7156044640 -1 osd.6 0 log_to_monitors true 2024-09-19T09:22:34.981 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:22:35.007 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2024-09-19T09:22:35.255 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:22:35.377 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:35 smithi097 ceph-mon[34864]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:35.377 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:35 smithi097 ceph-mon[34864]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T09:22:35.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:35 smithi016 ceph-mon[30979]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:35.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:35 smithi016 ceph-mon[30979]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T09:22:35.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:35 smithi142 ceph-mon[35032]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:35.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:35 smithi142 ceph-mon[35032]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T09:22:35.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.714+0000 7eff5e460640 1 -- 172.21.15.97:0/3479955790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58074c50 msgr2=0x7eff580750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.714+0000 7eff5e460640 1 --2- 172.21.15.97:0/3479955790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58074c50 0x7eff580750b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009900 tx=0x7eff4c02a410 comp rx=0 tx=0).stop 2024-09-19T09:22:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.714+0000 7eff5e460640 1 -- 172.21.15.97:0/3479955790 shutdown_connections 2024-09-19T09:22:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.714+0000 7eff5e460640 1 --2- 172.21.15.97:0/3479955790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff58109a90 0x7eff5810bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.714+0000 7eff5e460640 1 --2- 172.21.15.97:0/3479955790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58074c50 0x7eff580750b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.714+0000 7eff5e460640 1 --2- 172.21.15.97:0/3479955790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff58076220 0x7eff58074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.714+0000 7eff5e460640 1 -- 172.21.15.97:0/3479955790 >> 172.21.15.97:0/3479955790 conn(0x7eff580fd3c0 msgr2=0x7eff580ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.714+0000 7eff5e460640 1 -- 172.21.15.97:0/3479955790 shutdown_connections 2024-09-19T09:22:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.714+0000 7eff5e460640 1 -- 172.21.15.97:0/3479955790 wait complete. 2024-09-19T09:22:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.714+0000 7eff5e460640 1 Processor -- start 2024-09-19T09:22:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff5e460640 1 -- start start 2024-09-19T09:22:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff5e460640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58074c50 0x7eff5819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff5e460640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff58076220 0x7eff5819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff5e460640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff58109a90 0x7eff581a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff5e460640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff5810fab0 con 0x7eff58109a90 2024-09-19T09:22:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff5e460640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff5810f930 con 0x7eff58074c50 2024-09-19T09:22:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff5e460640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7eff5810fc30 con 0x7eff58076220 2024-09-19T09:22:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff57fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58074c50 0x7eff5819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff57fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58074c50 0x7eff5819a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50732/0 (socket says 172.21.15.97:50732) 2024-09-19T09:22:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff57fff640 1 -- 172.21.15.97:0/2944990726 learned_addr learned my addr 172.21.15.97:0/2944990726 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-19T09:22:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff5c9fd640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff58109a90 0x7eff581a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff577fe640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff58076220 0x7eff5819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff57fff640 1 -- 172.21.15.97:0/2944990726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff58076220 msgr2=0x7eff5819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff57fff640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff58076220 0x7eff5819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff57fff640 1 -- 172.21.15.97:0/2944990726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff58109a90 msgr2=0x7eff581a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff57fff640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff58109a90 0x7eff581a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff57fff640 1 -- 172.21.15.97:0/2944990726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff581a1d10 con 0x7eff58074c50 2024-09-19T09:22:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff5c9fd640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff58109a90 0x7eff581a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:22:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff577fe640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff58076220 0x7eff5819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:22:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.715+0000 7eff57fff640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58074c50 0x7eff5819a0e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7eff4800e9c0 tx=0x7eff4800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.716+0000 7eff557fa640 1 -- 172.21.15.97:0/2944990726 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4800cd80 con 0x7eff58074c50 2024-09-19T09:22:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.716+0000 7eff557fa640 1 -- 172.21.15.97:0/2944990726 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff48005500 con 0x7eff58074c50 2024-09-19T09:22:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.716+0000 7eff5e460640 1 -- 172.21.15.97:0/2944990726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff581a1fa0 con 0x7eff58074c50 2024-09-19T09:22:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.716+0000 7eff557fa640 1 -- 172.21.15.97:0/2944990726 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48010690 con 0x7eff58074c50 2024-09-19T09:22:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.716+0000 7eff5e460640 1 -- 172.21.15.97:0/2944990726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff581a2450 con 0x7eff58074c50 2024-09-19T09:22:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.717+0000 7eff5e460640 1 -- 172.21.15.97:0/2944990726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff58108810 con 0x7eff58074c50 2024-09-19T09:22:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.718+0000 7eff557fa640 1 -- 172.21.15.97:0/2944990726 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7eff480046c0 con 0x7eff58074c50 2024-09-19T09:22:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.718+0000 7eff557fa640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7eff400a3270 0x7eff400a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:22:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.718+0000 7eff557fa640 1 -- 172.21.15.97:0/2944990726 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4669+0+0 (secure 0 0 0) 0x7eff480cef50 con 0x7eff58074c50 2024-09-19T09:22:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.719+0000 7eff577fe640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7eff400a3270 0x7eff400a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:22:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.719+0000 7eff577fe640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7eff400a3270 0x7eff400a5730 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7eff4c000c00 tx=0x7eff4c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:22:35.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.722+0000 7eff557fa640 1 -- 172.21.15.97:0/2944990726 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7eff4809cab0 con 0x7eff58074c50 2024-09-19T09:22:35.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:35.899+0000 7eff5e460640 1 -- 172.21.15.97:0/2944990726 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7eff58062630 con 0x7eff400a3270 2024-09-19T09:22:36.308 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T09:22:36.308 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T09:22:36.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-19T09:22:36.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:36.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: Detected new or changed devices on smithi016 2024-09-19T09:22:36.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:36.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:36.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:36.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:36.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:36.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-19T09:22:36.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:36.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:36.309 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:36.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:36.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:22:36.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:22:36.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T09:22:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T09:22:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-19T09:22:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: Detected new or changed devices on smithi016 2024-09-19T09:22:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:36.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-19T09:22:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:22:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:22:36.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:36.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T09:22:36.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T09:22:36.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-19T09:22:36.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:36.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: Detected new or changed devices on smithi016 2024-09-19T09:22:36.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:36.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:36.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:36.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:36.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:22:36.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-19T09:22:36.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:36.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:36.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:36.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:36.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:22:36.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:22:36.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:37.253 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:22:36.947+0000 7f71517a4640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T09:22:37.253 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:22:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:22:36.958+0000 7f7149dd6640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:22:37.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:37 smithi016 ceph-mon[30979]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:37.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:37 smithi016 ceph-mon[30979]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:22:37.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:37 smithi016 ceph-mon[30979]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-19T09:22:37.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:37 smithi016 ceph-mon[30979]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T09:22:37.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:37 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:37.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:37 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:37.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:37 smithi016 ceph-mon[30979]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' 2024-09-19T09:22:37.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:37 smithi097 ceph-mon[34864]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:37.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:37 smithi097 ceph-mon[34864]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:22:37.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:37 smithi097 ceph-mon[34864]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-19T09:22:37.312 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:37 smithi097 ceph-mon[34864]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T09:22:37.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:37 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:37.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:37 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:37.313 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:37 smithi097 ceph-mon[34864]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' 2024-09-19T09:22:37.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:37 smithi142 ceph-mon[35032]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:37.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:37 smithi142 ceph-mon[35032]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:22:37.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:37 smithi142 ceph-mon[35032]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-19T09:22:37.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:37 smithi142 ceph-mon[35032]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T09:22:37.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:37 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:37.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:37 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:37.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:37 smithi142 ceph-mon[35032]: from='osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536]' entity='osd.6' 2024-09-19T09:22:38.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:38 smithi097 ceph-mon[34864]: purged_snaps scrub starts 2024-09-19T09:22:38.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:38 smithi097 ceph-mon[34864]: purged_snaps scrub ok 2024-09-19T09:22:38.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:38.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:38 smithi097 ceph-mon[34864]: osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536] boot 2024-09-19T09:22:38.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:38 smithi097 ceph-mon[34864]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T09:22:38.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:38.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:38 smithi016 ceph-mon[30979]: purged_snaps scrub starts 2024-09-19T09:22:38.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:38 smithi016 ceph-mon[30979]: purged_snaps scrub ok 2024-09-19T09:22:38.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:38.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:38 smithi016 ceph-mon[30979]: osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536] boot 2024-09-19T09:22:38.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:38 smithi016 ceph-mon[30979]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T09:22:38.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:38.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:38 smithi142 ceph-mon[35032]: purged_snaps scrub starts 2024-09-19T09:22:38.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:38 smithi142 ceph-mon[35032]: purged_snaps scrub ok 2024-09-19T09:22:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:38 smithi142 ceph-mon[35032]: osd.6 [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536] boot 2024-09-19T09:22:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:38 smithi142 ceph-mon[35032]: osdmap e43: 7 total, 7 up, 7 in 2024-09-19T09:22:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T09:22:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:39 smithi016 ceph-mon[30979]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:39 smithi016 ceph-mon[30979]: from='client.? 172.21.15.97:0/2586902437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d390b547-8949-4d4b-8481-c49d6ae7e424"}]: dispatch 2024-09-19T09:22:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:39 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d390b547-8949-4d4b-8481-c49d6ae7e424"}]: dispatch 2024-09-19T09:22:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:39 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d390b547-8949-4d4b-8481-c49d6ae7e424"}]': finished 2024-09-19T09:22:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:39 smithi016 ceph-mon[30979]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T09:22:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:22:39.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:39 smithi016 ceph-mon[30979]: from='client.? 172.21.15.97:0/2159442624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:22:39.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:39 smithi142 ceph-mon[35032]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:39.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:39 smithi142 ceph-mon[35032]: from='client.? 172.21.15.97:0/2586902437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d390b547-8949-4d4b-8481-c49d6ae7e424"}]: dispatch 2024-09-19T09:22:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:39 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d390b547-8949-4d4b-8481-c49d6ae7e424"}]: dispatch 2024-09-19T09:22:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:39 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d390b547-8949-4d4b-8481-c49d6ae7e424"}]': finished 2024-09-19T09:22:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:39 smithi142 ceph-mon[35032]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T09:22:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:22:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:39 smithi142 ceph-mon[35032]: from='client.? 172.21.15.97:0/2159442624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:22:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:39 smithi097 ceph-mon[34864]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-19T09:22:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:39 smithi097 ceph-mon[34864]: from='client.? 172.21.15.97:0/2586902437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d390b547-8949-4d4b-8481-c49d6ae7e424"}]: dispatch 2024-09-19T09:22:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:39 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d390b547-8949-4d4b-8481-c49d6ae7e424"}]: dispatch 2024-09-19T09:22:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:39 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d390b547-8949-4d4b-8481-c49d6ae7e424"}]': finished 2024-09-19T09:22:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:39 smithi097 ceph-mon[34864]: osdmap e44: 8 total, 7 up, 8 in 2024-09-19T09:22:39.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:22:39.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:39 smithi097 ceph-mon[34864]: from='client.? 172.21.15.97:0/2159442624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:22:41.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:41 smithi016 ceph-mon[30979]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:41.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:41 smithi142 ceph-mon[35032]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:41.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:41 smithi097 ceph-mon[34864]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:43 smithi097 ceph-mon[34864]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:43.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:43 smithi016 ceph-mon[30979]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:43.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:43 smithi142 ceph-mon[35032]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:45.426 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:45 smithi097 ceph-mon[34864]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:45 smithi016 ceph-mon[30979]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:45.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:45 smithi142 ceph-mon[35032]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:47.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:47 smithi097 ceph-mon[34864]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:47.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T09:22:47.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:47 smithi016 ceph-mon[30979]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T09:22:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:47.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:47 smithi142 ceph-mon[35032]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:47.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T09:22:47.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:48.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:48 smithi097 ceph-mon[34864]: Deploying daemon osd.7 on smithi097 2024-09-19T09:22:48.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:48 smithi016 ceph-mon[30979]: Deploying daemon osd.7 on smithi097 2024-09-19T09:22:48.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:48 smithi142 ceph-mon[35032]: Deploying daemon osd.7 on smithi097 2024-09-19T09:22:49.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:49 smithi097 ceph-mon[34864]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:49.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:49 smithi016 ceph-mon[30979]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:49.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:49 smithi142 ceph-mon[35032]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:51.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:51 smithi097 ceph-mon[34864]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:51.430 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:51 smithi142 ceph-mon[35032]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:51.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:51 smithi016 ceph-mon[30979]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:53.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:53 smithi097 ceph-mon[34864]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:53.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:53 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:53.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:53 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:53.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:53 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:53.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:53 smithi016 ceph-mon[30979]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:53.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:53.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:53.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:53.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:53 smithi142 ceph-mon[35032]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:53.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:53 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:53.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:53 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:53.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:53 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:55.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:55 smithi097 ceph-mon[34864]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:55.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:55 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:55.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:55 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:55.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:55 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:55.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:55 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:55.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:55 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:55.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:55 smithi016 ceph-mon[30979]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:55.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:55 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:55.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:55 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:55.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:55 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:55.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:55 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:55.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:55 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:55.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:55 smithi142 ceph-mon[35032]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:55.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:55 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:55.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:55 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:55.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:55 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:22:55.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:55 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:22:55.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:55 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:56.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:56.147+0000 7eff557fa640 1 -- 172.21.15.97:0/2944990726 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff58062630 con 0x7eff400a3270 2024-09-19T09:22:56.149 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 7 on host 'smithi097' 2024-09-19T09:22:56.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:56.150+0000 7eff5e460640 1 -- 172.21.15.97:0/2944990726 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7eff400a3270 msgr2=0x7eff400a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:56.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:56.150+0000 7eff5e460640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7eff400a3270 0x7eff400a5730 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7eff4c000c00 tx=0x7eff4c031400 comp rx=0 tx=0).stop 2024-09-19T09:22:56.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:56.150+0000 7eff5e460640 1 -- 172.21.15.97:0/2944990726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58074c50 msgr2=0x7eff5819a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:22:56.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:56.150+0000 7eff5e460640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58074c50 0x7eff5819a0e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7eff4800e9c0 tx=0x7eff4800ee80 comp rx=0 tx=0).stop 2024-09-19T09:22:56.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:56.150+0000 7eff5e460640 1 -- 172.21.15.97:0/2944990726 shutdown_connections 2024-09-19T09:22:56.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:56.150+0000 7eff5e460640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7eff400a3270 0x7eff400a5730 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:56.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:56.150+0000 7eff5e460640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff58076220 0x7eff5819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:56.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:56.150+0000 7eff5e460640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff58109a90 0x7eff581a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:56.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:56.150+0000 7eff5e460640 1 --2- 172.21.15.97:0/2944990726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff58074c50 0x7eff5819a0e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:22:56.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:56.150+0000 7eff5e460640 1 -- 172.21.15.97:0/2944990726 >> 172.21.15.97:0/2944990726 conn(0x7eff580fd3c0 msgr2=0x7eff581059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:22:56.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:56.150+0000 7eff5e460640 1 -- 172.21.15.97:0/2944990726 shutdown_connections 2024-09-19T09:22:56.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:22:56.150+0000 7eff5e460640 1 -- 172.21.15.97:0/2944990726 wait complete. 2024-09-19T09:22:56.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:56.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:56.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:56.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:56.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:56.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:56.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:22:56.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:56.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:22:56.772 DEBUG:teuthology.orchestra.run.smithi097:osd.7> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T09:22:56.774 INFO:tasks.cephadm:Deploying osd.8 on smithi142 with /dev/vg_nvme/lv_2... 2024-09-19T09:22:56.775 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T09:22:57.029 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:22:57.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:57 smithi097 ceph-mon[34864]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:57.444 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:57 smithi142 ceph-mon[35032]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:57.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:57 smithi016 ceph-mon[30979]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:59.136 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:22:58 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:22:58.880+0000 7fd5b3494640 -1 osd.7 0 log_to_monitors true 2024-09-19T09:22:59.465 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:59 smithi097 ceph-mon[34864]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:59.465 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:59 smithi097 ceph-mon[34864]: from='osd.7 [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T09:22:59.465 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:22:59 smithi097 ceph-mon[34864]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T09:22:59.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:59 smithi016 ceph-mon[30979]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:59.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:59 smithi016 ceph-mon[30979]: from='osd.7 [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T09:22:59.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:22:59 smithi016 ceph-mon[30979]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T09:22:59.541 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:59 smithi142 ceph-mon[35032]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:22:59.541 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:59 smithi142 ceph-mon[35032]: from='osd.7 [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T09:22:59.541 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:22:59 smithi142 ceph-mon[35032]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T09:22:59.657 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:22:59.683 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2024-09-19T09:22:59.935 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:23:00.327 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T09:23:00.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T09:23:00.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:00.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: from='osd.7 [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:23:00.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:23:00.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:00.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:00.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:00.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:00.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:00.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:00.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:00.329 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:00.329 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:00 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:00.416 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.382+0000 7f3373568640 1 -- 172.21.15.142:0/2166169958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c1037c0 msgr2=0x7f336c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.382+0000 7f3373568640 1 --2- 172.21.15.142:0/2166169958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c1037c0 0x7f336c10a050 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3364009ca0 tx=0x7f336402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.382+0000 7f3373568640 1 -- 172.21.15.142:0/2166169958 shutdown_connections 2024-09-19T09:23:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.382+0000 7f3373568640 1 --2- 172.21.15.142:0/2166169958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c1037c0 0x7f336c10a050 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.382+0000 7f3373568640 1 --2- 172.21.15.142:0/2166169958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c102e00 0x7f336c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.382+0000 7f3373568640 1 --2- 172.21.15.142:0/2166169958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f336c101c00 0x7f336c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.382+0000 7f3373568640 1 -- 172.21.15.142:0/2166169958 >> 172.21.15.142:0/2166169958 conn(0x7f336c0fd390 msgr2=0x7f336c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.382+0000 7f3373568640 1 -- 172.21.15.142:0/2166169958 shutdown_connections 2024-09-19T09:23:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.383+0000 7f3373568640 1 -- 172.21.15.142:0/2166169958 wait complete. 2024-09-19T09:23:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.383+0000 7f3373568640 1 Processor -- start 2024-09-19T09:23:00.417 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.383+0000 7f3373568640 1 -- start start 2024-09-19T09:23:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.383+0000 7f3373568640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c101c00 0x7f336c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.383+0000 7f3373568640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c102e00 0x7f336c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.383+0000 7f3373568640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f336c1037c0 0x7f336c19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.383+0000 7f3373568640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f336c10d860 con 0x7f336c102e00 2024-09-19T09:23:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.383+0000 7f3373568640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f336c10d6e0 con 0x7f336c1037c0 2024-09-19T09:23:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.383+0000 7f3373568640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f336c10d9e0 con 0x7f336c101c00 2024-09-19T09:23:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.383+0000 7f3371304640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c101c00 0x7f336c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.383+0000 7f3371304640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c101c00 0x7f336c197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:53874/0 (socket says 172.21.15.142:53874) 2024-09-19T09:23:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.383+0000 7f3371304640 1 -- 172.21.15.142:0/1095559543 learned_addr learned my addr 172.21.15.142:0/1095559543 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-19T09:23:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.384+0000 7f3371b05640 1 --2- 172.21.15.142:0/1095559543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f336c1037c0 0x7f336c19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.384+0000 7f3371304640 1 -- 172.21.15.142:0/1095559543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f336c1037c0 msgr2=0x7f336c19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.418 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.384+0000 7f3371304640 1 --2- 172.21.15.142:0/1095559543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f336c1037c0 0x7f336c19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.384+0000 7f3371304640 1 -- 172.21.15.142:0/1095559543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c102e00 msgr2=0x7f336c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.384+0000 7f3371304640 1 --2- 172.21.15.142:0/1095559543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c102e00 0x7f336c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.384+0000 7f3371304640 1 -- 172.21.15.142:0/1095559543 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c19fa80 con 0x7f336c101c00 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.384+0000 7f3371304640 1 --2- 172.21.15.142:0/1095559543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c101c00 0x7f336c197e50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f335c00e9c0 tx=0x7f335c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.384+0000 7f33627fc640 1 -- 172.21.15.142:0/1095559543 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c00cd80 con 0x7f336c101c00 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.384+0000 7f3373568640 1 -- 172.21.15.142:0/1095559543 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c19fd70 con 0x7f336c101c00 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.384+0000 7f33627fc640 1 -- 172.21.15.142:0/1095559543 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f335c0035f0 con 0x7f336c101c00 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.384+0000 7f33627fc640 1 -- 172.21.15.142:0/1095559543 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c010690 con 0x7f336c101c00 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.384+0000 7f3373568640 1 -- 172.21.15.142:0/1095559543 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c1a0270 con 0x7f336c101c00 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.385+0000 7f3373568640 1 -- 172.21.15.142:0/1095559543 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f336c108fb0 con 0x7f336c101c00 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.386+0000 7f33627fc640 1 -- 172.21.15.142:0/1095559543 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f335c0037a0 con 0x7f336c101c00 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.387+0000 7f33627fc640 1 --2- 172.21.15.142:0/1095559543 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f33580a3270 0x7f33580a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.387+0000 7f33627fc640 1 -- 172.21.15.142:0/1095559543 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5117+0+0 (secure 0 0 0) 0x7f335c0cf130 con 0x7f336c101c00 2024-09-19T09:23:00.419 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.387+0000 7f3370b03640 1 --2- 172.21.15.142:0/1095559543 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f33580a3270 0x7f33580a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:00.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.387+0000 7f3370b03640 1 --2- 172.21.15.142:0/1095559543 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f33580a3270 0x7f33580a5730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33680058c0 tx=0x7f3368005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:00.420 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.391+0000 7f33627fc640 1 -- 172.21.15.142:0/1095559543 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f335c09cad0 con 0x7f336c101c00 2024-09-19T09:23:00.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T09:23:00.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T09:23:00.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:00.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: from='osd.7 [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:23:00.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:23:00.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:00.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:00.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:00.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:00.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:00.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:00.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:00.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:00.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:00 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:00.568 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:00.566+0000 7f3373568640 1 -- 172.21.15.142:0/1095559543 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f336c062630 con 0x7f33580a3270 2024-09-19T09:23:00.635 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:23:00.204+0000 7fd5aebf4640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T09:23:00.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:23:00.213+0000 7fd5ab22e640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:23:00.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T09:23:00.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: osdmap e45: 8 total, 7 up, 8 in 2024-09-19T09:23:00.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:00.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: from='osd.7 [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:23:00.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:23:00.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:00.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:00.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:00.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:00.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:00.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:00.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:00.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:00.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:00 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:01.235 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:01 smithi142 ceph-mon[35032]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:23:01.235 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:01 smithi142 ceph-mon[35032]: Detected new or changed devices on smithi097 2024-09-19T09:23:01.236 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:01 smithi142 ceph-mon[35032]: Adjusting osd_memory_target on smithi097 to 5696M 2024-09-19T09:23:01.236 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:01 smithi142 ceph-mon[35032]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-19T09:23:01.236 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:01 smithi142 ceph-mon[35032]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T09:23:01.236 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:01.236 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:01.236 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:23:01.236 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:23:01.236 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:01 smithi016 ceph-mon[30979]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:23:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:01 smithi016 ceph-mon[30979]: Detected new or changed devices on smithi097 2024-09-19T09:23:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:01 smithi016 ceph-mon[30979]: Adjusting osd_memory_target on smithi097 to 5696M 2024-09-19T09:23:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:01 smithi016 ceph-mon[30979]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-19T09:23:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:01 smithi016 ceph-mon[30979]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T09:23:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:01.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:01.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:23:01.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:23:01.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:01.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:01 smithi097 ceph-mon[34864]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:23:01.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:01 smithi097 ceph-mon[34864]: Detected new or changed devices on smithi097 2024-09-19T09:23:01.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:01 smithi097 ceph-mon[34864]: Adjusting osd_memory_target on smithi097 to 5696M 2024-09-19T09:23:01.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:01 smithi097 ceph-mon[34864]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-19T09:23:01.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:01 smithi097 ceph-mon[34864]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T09:23:01.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:01.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:01.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:23:01.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:23:01.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:02.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:02 smithi097 ceph-mon[34864]: purged_snaps scrub starts 2024-09-19T09:23:02.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:02 smithi097 ceph-mon[34864]: purged_snaps scrub ok 2024-09-19T09:23:02.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:02 smithi097 ceph-mon[34864]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:02.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:02.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:02 smithi097 ceph-mon[34864]: osd.7 [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670] boot 2024-09-19T09:23:02.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:02 smithi097 ceph-mon[34864]: osdmap e47: 8 total, 8 up, 8 in 2024-09-19T09:23:02.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:02 smithi016 ceph-mon[30979]: purged_snaps scrub starts 2024-09-19T09:23:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:02 smithi016 ceph-mon[30979]: purged_snaps scrub ok 2024-09-19T09:23:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:02 smithi016 ceph-mon[30979]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:02 smithi016 ceph-mon[30979]: osd.7 [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670] boot 2024-09-19T09:23:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:02 smithi016 ceph-mon[30979]: osdmap e47: 8 total, 8 up, 8 in 2024-09-19T09:23:02.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:02.554 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:02 smithi142 ceph-mon[35032]: purged_snaps scrub starts 2024-09-19T09:23:02.554 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:02 smithi142 ceph-mon[35032]: purged_snaps scrub ok 2024-09-19T09:23:02.554 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:02 smithi142 ceph-mon[35032]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:02.554 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:02.554 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:02 smithi142 ceph-mon[35032]: osd.7 [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670] boot 2024-09-19T09:23:02.554 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:02 smithi142 ceph-mon[35032]: osdmap e47: 8 total, 8 up, 8 in 2024-09-19T09:23:02.554 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T09:23:03.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:03 smithi142 ceph-mon[35032]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:23:03.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:03 smithi142 ceph-mon[35032]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T09:23:03.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:03 smithi142 ceph-mon[35032]: from='client.? 172.21.15.142:0/3681228449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78a476c4-5deb-4f9e-a266-db4493850732"}]: dispatch 2024-09-19T09:23:03.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:03 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78a476c4-5deb-4f9e-a266-db4493850732"}]: dispatch 2024-09-19T09:23:03.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:03 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78a476c4-5deb-4f9e-a266-db4493850732"}]': finished 2024-09-19T09:23:03.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:03 smithi142 ceph-mon[35032]: osdmap e49: 9 total, 8 up, 9 in 2024-09-19T09:23:03.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:03 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:03.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:03 smithi016 ceph-mon[30979]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:23:03.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:03 smithi016 ceph-mon[30979]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T09:23:03.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:03 smithi016 ceph-mon[30979]: from='client.? 172.21.15.142:0/3681228449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78a476c4-5deb-4f9e-a266-db4493850732"}]: dispatch 2024-09-19T09:23:03.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:03 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78a476c4-5deb-4f9e-a266-db4493850732"}]: dispatch 2024-09-19T09:23:03.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:03 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78a476c4-5deb-4f9e-a266-db4493850732"}]': finished 2024-09-19T09:23:03.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:03 smithi016 ceph-mon[30979]: osdmap e49: 9 total, 8 up, 9 in 2024-09-19T09:23:03.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:03 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:03.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:03 smithi097 ceph-mon[34864]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-09-19T09:23:03.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:03 smithi097 ceph-mon[34864]: osdmap e48: 8 total, 8 up, 8 in 2024-09-19T09:23:03.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:03 smithi097 ceph-mon[34864]: from='client.? 172.21.15.142:0/3681228449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78a476c4-5deb-4f9e-a266-db4493850732"}]: dispatch 2024-09-19T09:23:03.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:03 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78a476c4-5deb-4f9e-a266-db4493850732"}]: dispatch 2024-09-19T09:23:03.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:03 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78a476c4-5deb-4f9e-a266-db4493850732"}]': finished 2024-09-19T09:23:03.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:03 smithi097 ceph-mon[34864]: osdmap e49: 9 total, 8 up, 9 in 2024-09-19T09:23:03.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:03 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:04 smithi016 ceph-mon[30979]: from='client.? 172.21.15.142:0/202848383' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:23:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:04 smithi016 ceph-mon[30979]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T09:23:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:04 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:04.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:04 smithi142 ceph-mon[35032]: from='client.? 172.21.15.142:0/202848383' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:23:04.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:04 smithi142 ceph-mon[35032]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T09:23:04.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:04 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:04.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:04 smithi097 ceph-mon[34864]: from='client.? 172.21.15.142:0/202848383' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:23:04.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:04 smithi097 ceph-mon[34864]: osdmap e50: 9 total, 8 up, 9 in 2024-09-19T09:23:04.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:04 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:05.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:05 smithi016 ceph-mon[30979]: pgmap v170: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:05.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:05 smithi016 ceph-mon[30979]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T09:23:05.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:05 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:05.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:05 smithi142 ceph-mon[35032]: pgmap v170: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:05.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:05 smithi142 ceph-mon[35032]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T09:23:05.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:05 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:05.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:05 smithi097 ceph-mon[34864]: pgmap v170: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:05.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:05 smithi097 ceph-mon[34864]: osdmap e51: 9 total, 8 up, 9 in 2024-09-19T09:23:05.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:05 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:07.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:07 smithi016 ceph-mon[30979]: pgmap v173: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:07.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:07 smithi142 ceph-mon[35032]: pgmap v173: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:07.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:07 smithi097 ceph-mon[34864]: pgmap v173: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:09.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:09 smithi142 ceph-mon[35032]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:09 smithi016 ceph-mon[30979]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:09.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:09 smithi097 ceph-mon[34864]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:11.325 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:11 smithi142 ceph-mon[35032]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:11.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:11 smithi016 ceph-mon[30979]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:11.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:11 smithi097 ceph-mon[34864]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:12.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:12 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T09:23:12.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:12 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:12 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T09:23:12.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:12 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:12.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:12 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T09:23:12.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:12 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:13.297 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:13 smithi142 ceph-mon[35032]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:13.297 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:13 smithi142 ceph-mon[35032]: Deploying daemon osd.8 on smithi142 2024-09-19T09:23:13.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:13 smithi016 ceph-mon[30979]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:13.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:13 smithi016 ceph-mon[30979]: Deploying daemon osd.8 on smithi142 2024-09-19T09:23:13.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:13 smithi097 ceph-mon[34864]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:13.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:13 smithi097 ceph-mon[34864]: Deploying daemon osd.8 on smithi142 2024-09-19T09:23:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:15 smithi016 ceph-mon[30979]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:15.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:15 smithi142 ceph-mon[35032]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:15.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:15 smithi097 ceph-mon[34864]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:17.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:17 smithi016 ceph-mon[30979]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:17.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:17.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:17.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:17.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:17 smithi142 ceph-mon[35032]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:17.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:17.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:17.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:17.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:17 smithi097 ceph-mon[34864]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:17.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:17.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:17.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:19.417 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:19 smithi142 ceph-mon[35032]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:19.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:19 smithi016 ceph-mon[30979]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:19.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:19 smithi097 ceph-mon[34864]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:20.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:20 smithi142 ceph-mon[35032]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:20.795 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:20.795 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:20.795 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:20.795 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:20.795 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:20.836 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 8 on host 'smithi142' 2024-09-19T09:23:20.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:20.834+0000 7f33627fc640 1 -- 172.21.15.142:0/1095559543 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f336c062630 con 0x7f33580a3270 2024-09-19T09:23:20.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:20.836+0000 7f3373568640 1 -- 172.21.15.142:0/1095559543 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f33580a3270 msgr2=0x7f33580a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:20.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:20.836+0000 7f3373568640 1 --2- 172.21.15.142:0/1095559543 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f33580a3270 0x7f33580a5730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33680058c0 tx=0x7f3368005850 comp rx=0 tx=0).stop 2024-09-19T09:23:20.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:20.836+0000 7f3373568640 1 -- 172.21.15.142:0/1095559543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c101c00 msgr2=0x7f336c197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:20.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:20.836+0000 7f3373568640 1 --2- 172.21.15.142:0/1095559543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c101c00 0x7f336c197e50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f335c00e9c0 tx=0x7f335c00ee80 comp rx=0 tx=0).stop 2024-09-19T09:23:20.840 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:20.836+0000 7f3373568640 1 -- 172.21.15.142:0/1095559543 shutdown_connections 2024-09-19T09:23:20.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:20.836+0000 7f3373568640 1 --2- 172.21.15.142:0/1095559543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f336c1037c0 0x7f336c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:20.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:20.836+0000 7f3373568640 1 --2- 172.21.15.142:0/1095559543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c102e00 0x7f336c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:20.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:20.836+0000 7f3373568640 1 --2- 172.21.15.142:0/1095559543 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f33580a3270 0x7f33580a5730 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:20.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:20.836+0000 7f3373568640 1 --2- 172.21.15.142:0/1095559543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f336c101c00 0x7f336c197e50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:20.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:20.836+0000 7f3373568640 1 -- 172.21.15.142:0/1095559543 >> 172.21.15.142:0/1095559543 conn(0x7f336c0fd390 msgr2=0x7f336c106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:20.841 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:20.837+0000 7f3373568640 1 -- 172.21.15.142:0/1095559543 shutdown_connections 2024-09-19T09:23:20.842 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:23:20.837+0000 7f3373568640 1 -- 172.21.15.142:0/1095559543 wait complete. 2024-09-19T09:23:21.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:20 smithi016 ceph-mon[30979]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:21.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:21.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:21.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:21.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:21.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:21.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:20 smithi097 ceph-mon[34864]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:21.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:21.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:21.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:21.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:21.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:21.542 DEBUG:teuthology.orchestra.run.smithi142:osd.8> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T09:23:21.546 INFO:tasks.cephadm:Deploying osd.9 on smithi016 with /dev/vg_nvme/lv_1... 2024-09-19T09:23:21.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T09:23:21.808 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:21.808 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:21.808 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:21.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:21.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:21.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:22.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:22.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:22.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:22.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:22 smithi142 ceph-mon[35032]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:23.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:22 smithi097 ceph-mon[34864]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:23.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:22 smithi016 ceph-mon[30979]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:23.836 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:23:23 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:23:23.515+0000 7fcae1f1b640 -1 osd.8 0 log_to_monitors true 2024-09-19T09:23:24.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:23 smithi142 ceph-mon[35032]: from='osd.8 [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T09:23:24.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:23 smithi142 ceph-mon[35032]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T09:23:24.133 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:23 smithi016 ceph-mon[30979]: from='osd.8 [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T09:23:24.133 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:23 smithi016 ceph-mon[30979]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T09:23:24.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:23 smithi097 ceph-mon[34864]: from='osd.8 [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T09:23:24.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:23 smithi097 ceph-mon[34864]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T09:23:24.546 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:23:24.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-09-19T09:23:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:23:25.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:25.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T09:23:25.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T09:23:25.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:25.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: from='osd.8 [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:23:25.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:23:25.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:25.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:25.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:25.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:25.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:25.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:25.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:25.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:25.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:25.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:25.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T09:23:25.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T09:23:25.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:25.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: from='osd.8 [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:23:25.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:23:25.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:25.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:25.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:25.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:25.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:25.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:25.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:25.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:25.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:25.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:25.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T09:23:25.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T09:23:25.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:25.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: from='osd.8 [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:23:25.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:23:25.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:25.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:25.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:25.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:25.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:25.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:25.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:25.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:25.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.290+0000 7f0478554640 1 -- 172.21.15.16:0/1285236174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04701078c0 msgr2=0x7f0470109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.290+0000 7f0478554640 1 --2- 172.21.15.16:0/1285236174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04701078c0 0x7f0470109d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0468009ca0 tx=0x7f046802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:23:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.290+0000 7f0478554640 1 -- 172.21.15.16:0/1285236174 shutdown_connections 2024-09-19T09:23:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.290+0000 7f0478554640 1 --2- 172.21.15.16:0/1285236174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0470104f90 0x7f0470107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.290+0000 7f0478554640 1 --2- 172.21.15.16:0/1285236174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04701078c0 0x7f0470109d10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.290+0000 7f0478554640 1 --2- 172.21.15.16:0/1285236174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470102660 0x7f0470104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.290+0000 7f0478554640 1 -- 172.21.15.16:0/1285236174 >> 172.21.15.16:0/1285236174 conn(0x7f04700fc280 msgr2=0x7f04700fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.291+0000 7f0478554640 1 -- 172.21.15.16:0/1285236174 shutdown_connections 2024-09-19T09:23:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.291+0000 7f0478554640 1 -- 172.21.15.16:0/1285236174 wait complete. 2024-09-19T09:23:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.291+0000 7f0478554640 1 Processor -- start 2024-09-19T09:23:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.291+0000 7f0478554640 1 -- start start 2024-09-19T09:23:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.291+0000 7f0478554640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0470102660 0x7f0470197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.291+0000 7f0478554640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470104f90 0x7f0470198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.291+0000 7f0478554640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04701078c0 0x7f047019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.291+0000 7f0478554640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f047010d880 con 0x7f0470102660 2024-09-19T09:23:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.291+0000 7f0478554640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f047010d700 con 0x7f0470104f90 2024-09-19T09:23:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.291+0000 7f0478554640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f047010da00 con 0x7f04701078c0 2024-09-19T09:23:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f0475aef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470104f90 0x7f0470198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f04762f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0470102660 0x7f0470197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f0475aef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470104f90 0x7f0470198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:37516/0 (socket says 172.21.15.16:37516) 2024-09-19T09:23:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f0475aef640 1 -- 172.21.15.16:0/3953025942 learned_addr learned my addr 172.21.15.16:0/3953025942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:23:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f0476af1640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04701078c0 0x7f047019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f04762f0640 1 -- 172.21.15.16:0/3953025942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04701078c0 msgr2=0x7f047019f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:25.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f04762f0640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04701078c0 0x7f047019f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f04762f0640 1 -- 172.21.15.16:0/3953025942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470104f90 msgr2=0x7f0470198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f04762f0640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470104f90 0x7f0470198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f04762f0640 1 -- 172.21.15.16:0/3953025942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047019fb10 con 0x7f0470102660 2024-09-19T09:23:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f0475aef640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470104f90 0x7f0470198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:23:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f04762f0640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0470102660 0x7f0470197ee0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f046c00d6d0 tx=0x7f046c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f04677fe640 1 -- 172.21.15.16:0/3953025942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c017070 con 0x7f0470102660 2024-09-19T09:23:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f0478554640 1 -- 172.21.15.16:0/3953025942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04701004b0 con 0x7f0470102660 2024-09-19T09:23:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f04677fe640 1 -- 172.21.15.16:0/3953025942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f046c009820 con 0x7f0470102660 2024-09-19T09:23:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f04677fe640 1 -- 172.21.15.16:0/3953025942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c004d00 con 0x7f0470102660 2024-09-19T09:23:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.292+0000 7f0478554640 1 -- 172.21.15.16:0/3953025942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0470100a40 con 0x7f0470102660 2024-09-19T09:23:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.294+0000 7f0478554640 1 -- 172.21.15.16:0/3953025942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04700fdd70 con 0x7f0470102660 2024-09-19T09:23:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.295+0000 7f04677fe640 1 -- 172.21.15.16:0/3953025942 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f046c0099d0 con 0x7f0470102660 2024-09-19T09:23:25.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.295+0000 7f04677fe640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f045c0a3270 0x7f045c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.296+0000 7f0475aef640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f045c0a3270 0x7f045c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.296+0000 7f04677fe640 1 -- 172.21.15.16:0/3953025942 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5549+0+0 (secure 0 0 0) 0x7f046c0ce930 con 0x7f0470102660 2024-09-19T09:23:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.296+0000 7f0475aef640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f045c0a3270 0x7f045c0a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0470199400 tx=0x7f0460005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:25.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.300+0000 7f04677fe640 1 -- 172.21.15.16:0/3953025942 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f046c09bad0 con 0x7f0470102660 2024-09-19T09:23:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:25.514+0000 7f0478554640 1 -- 172.21.15.16:0/3953025942 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0470062630 con 0x7f045c0a3270 2024-09-19T09:23:25.878 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:23:25 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:23:25.607+0000 7fcadd67b640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T09:23:25.878 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:23:25 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:23:25.625+0000 7fcad5cad640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:23:25.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:25 smithi142 ceph-mon[35032]: Detected new or changed devices on smithi142 2024-09-19T09:23:25.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:25 smithi142 ceph-mon[35032]: Adjusting osd_memory_target on smithi142 to 5696M 2024-09-19T09:23:25.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:25 smithi142 ceph-mon[35032]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-19T09:23:25.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:25 smithi142 ceph-mon[35032]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T09:23:25.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:25 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:25.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:25 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:25.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:25 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:23:25.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:25 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:23:25.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:25 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:25.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:25 smithi142 ceph-mon[35032]: from='osd.8 ' entity='osd.8' 2024-09-19T09:23:26.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:25 smithi097 ceph-mon[34864]: Detected new or changed devices on smithi142 2024-09-19T09:23:26.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:25 smithi097 ceph-mon[34864]: Adjusting osd_memory_target on smithi142 to 5696M 2024-09-19T09:23:26.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:25 smithi097 ceph-mon[34864]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-19T09:23:26.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:25 smithi097 ceph-mon[34864]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T09:23:26.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:25 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:26.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:25 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:26.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:25 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:23:26.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:25 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:23:26.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:25 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:26.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:25 smithi097 ceph-mon[34864]: from='osd.8 ' entity='osd.8' 2024-09-19T09:23:26.141 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:25 smithi016 ceph-mon[30979]: Detected new or changed devices on smithi142 2024-09-19T09:23:26.141 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:25 smithi016 ceph-mon[30979]: Adjusting osd_memory_target on smithi142 to 5696M 2024-09-19T09:23:26.141 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:25 smithi016 ceph-mon[30979]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-19T09:23:26.141 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:25 smithi016 ceph-mon[30979]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T09:23:26.141 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:25 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:26.141 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:25 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:26.141 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:25 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:23:26.141 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:25 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:23:26.141 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:25 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:26.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:25 smithi016 ceph-mon[30979]: from='osd.8 ' entity='osd.8' 2024-09-19T09:23:26.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:26 smithi016 ceph-mon[30979]: purged_snaps scrub starts 2024-09-19T09:23:26.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:26 smithi016 ceph-mon[30979]: purged_snaps scrub ok 2024-09-19T09:23:26.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:26 smithi016 ceph-mon[30979]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:26.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:26 smithi016 ceph-mon[30979]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:26.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:26.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:26 smithi016 ceph-mon[30979]: osd.8 [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004] boot 2024-09-19T09:23:26.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:26 smithi016 ceph-mon[30979]: osdmap e54: 9 total, 9 up, 9 in 2024-09-19T09:23:26.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:27.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:26 smithi142 ceph-mon[35032]: purged_snaps scrub starts 2024-09-19T09:23:27.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:26 smithi142 ceph-mon[35032]: purged_snaps scrub ok 2024-09-19T09:23:27.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:26 smithi142 ceph-mon[35032]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:27.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:26 smithi142 ceph-mon[35032]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:27.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:27.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:26 smithi142 ceph-mon[35032]: osd.8 [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004] boot 2024-09-19T09:23:27.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:26 smithi142 ceph-mon[35032]: osdmap e54: 9 total, 9 up, 9 in 2024-09-19T09:23:27.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:27.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:26 smithi097 ceph-mon[34864]: purged_snaps scrub starts 2024-09-19T09:23:27.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:26 smithi097 ceph-mon[34864]: purged_snaps scrub ok 2024-09-19T09:23:27.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:26 smithi097 ceph-mon[34864]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-19T09:23:27.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:26 smithi097 ceph-mon[34864]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:27.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:27.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:26 smithi097 ceph-mon[34864]: osd.8 [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004] boot 2024-09-19T09:23:27.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:26 smithi097 ceph-mon[34864]: osdmap e54: 9 total, 9 up, 9 in 2024-09-19T09:23:27.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T09:23:28.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:27 smithi142 ceph-mon[35032]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T09:23:28.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:27 smithi097 ceph-mon[34864]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T09:23:28.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:27 smithi016 ceph-mon[30979]: osdmap e55: 9 total, 9 up, 9 in 2024-09-19T09:23:29.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:28 smithi142 ceph-mon[35032]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:28 smithi142 ceph-mon[35032]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T09:23:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:28 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1869913306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0415d187-4a77-42f7-b8bb-2d0b52e9cb63"}]: dispatch 2024-09-19T09:23:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:28 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1869913306' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0415d187-4a77-42f7-b8bb-2d0b52e9cb63"}]': finished 2024-09-19T09:23:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:28 smithi142 ceph-mon[35032]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T09:23:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:28 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:29.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:28 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2823650468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:23:29.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:28 smithi097 ceph-mon[34864]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:29.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:28 smithi097 ceph-mon[34864]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T09:23:29.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:28 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1869913306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0415d187-4a77-42f7-b8bb-2d0b52e9cb63"}]: dispatch 2024-09-19T09:23:29.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:28 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1869913306' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0415d187-4a77-42f7-b8bb-2d0b52e9cb63"}]': finished 2024-09-19T09:23:29.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:28 smithi097 ceph-mon[34864]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T09:23:29.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:28 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:29.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:28 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2823650468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:23:29.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:28 smithi016 ceph-mon[30979]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:29.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:28 smithi016 ceph-mon[30979]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T09:23:29.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:28 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1869913306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0415d187-4a77-42f7-b8bb-2d0b52e9cb63"}]: dispatch 2024-09-19T09:23:29.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:28 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1869913306' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0415d187-4a77-42f7-b8bb-2d0b52e9cb63"}]': finished 2024-09-19T09:23:29.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:28 smithi016 ceph-mon[30979]: osdmap e57: 10 total, 9 up, 10 in 2024-09-19T09:23:29.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:28 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:29.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:28 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2823650468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:23:30.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:29 smithi016 ceph-mon[30979]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T09:23:30.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:29 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:30.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:29 smithi142 ceph-mon[35032]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T09:23:30.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:29 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:30.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:29 smithi097 ceph-mon[34864]: osdmap e58: 10 total, 9 up, 10 in 2024-09-19T09:23:30.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:29 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:31.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:30 smithi016 ceph-mon[30979]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:31.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:30 smithi142 ceph-mon[35032]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:31.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:30 smithi097 ceph-mon[34864]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:33.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:32 smithi016 ceph-mon[30979]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:33.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:32 smithi097 ceph-mon[34864]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:33.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:32 smithi142 ceph-mon[35032]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:35.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:34 smithi016 ceph-mon[30979]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:23:35.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:34 smithi142 ceph-mon[35032]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:23:35.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:34 smithi097 ceph-mon[34864]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T09:23:37.216 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:36 smithi016 ceph-mon[30979]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-19T09:23:37.216 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T09:23:37.217 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:37.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:36 smithi142 ceph-mon[35032]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-19T09:23:37.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T09:23:37.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:37.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:36 smithi097 ceph-mon[34864]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-19T09:23:37.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T09:23:37.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:38.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:37 smithi016 ceph-mon[30979]: Deploying daemon osd.9 on smithi016 2024-09-19T09:23:38.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:37 smithi142 ceph-mon[35032]: Deploying daemon osd.9 on smithi016 2024-09-19T09:23:38.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:37 smithi097 ceph-mon[34864]: Deploying daemon osd.9 on smithi016 2024-09-19T09:23:39.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:38 smithi016 ceph-mon[30979]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-19T09:23:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:38 smithi142 ceph-mon[35032]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-19T09:23:39.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:38 smithi097 ceph-mon[34864]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-19T09:23:41.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:40 smithi016 ceph-mon[30979]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T09:23:41.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:40 smithi142 ceph-mon[35032]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T09:23:41.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:40 smithi097 ceph-mon[34864]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-19T09:23:43.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:42 smithi016 ceph-mon[30979]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:23:43.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:43.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:43.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:43.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:42 smithi142 ceph-mon[35032]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:23:43.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:43.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:43.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:43.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:42 smithi097 ceph-mon[34864]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:23:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:45.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:44 smithi016 ceph-mon[30979]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:23:45.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:44 smithi142 ceph-mon[35032]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:23:45.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:44 smithi097 ceph-mon[34864]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:23:46.270 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 9 on host 'smithi016' 2024-09-19T09:23:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:46.268+0000 7f04677fe640 1 -- 172.21.15.16:0/3953025942 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0470062630 con 0x7f045c0a3270 2024-09-19T09:23:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:46.271+0000 7f0478554640 1 -- 172.21.15.16:0/3953025942 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f045c0a3270 msgr2=0x7f045c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:46.271+0000 7f0478554640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f045c0a3270 0x7f045c0a5730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0470199400 tx=0x7f0460005f90 comp rx=0 tx=0).stop 2024-09-19T09:23:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:46.271+0000 7f0478554640 1 -- 172.21.15.16:0/3953025942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0470102660 msgr2=0x7f0470197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:46.271+0000 7f0478554640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0470102660 0x7f0470197ee0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f046c00d6d0 tx=0x7f046c00db90 comp rx=0 tx=0).stop 2024-09-19T09:23:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:46.271+0000 7f0478554640 1 -- 172.21.15.16:0/3953025942 shutdown_connections 2024-09-19T09:23:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:46.271+0000 7f0478554640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f045c0a3270 0x7f045c0a5730 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:46.271+0000 7f0478554640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04701078c0 0x7f047019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:46.271+0000 7f0478554640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470104f90 0x7f0470198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:46.271+0000 7f0478554640 1 --2- 172.21.15.16:0/3953025942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0470102660 0x7f0470197ee0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:46.271+0000 7f0478554640 1 -- 172.21.15.16:0/3953025942 >> 172.21.15.16:0/3953025942 conn(0x7f04700fc280 msgr2=0x7f0470102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:46.271+0000 7f0478554640 1 -- 172.21.15.16:0/3953025942 shutdown_connections 2024-09-19T09:23:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:23:46.271+0000 7f0478554640 1 -- 172.21.15.16:0/3953025942 wait complete. 2024-09-19T09:23:46.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:46 smithi016 ceph-mon[30979]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:46.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:46.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:46.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:46.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:46 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:46 smithi142 ceph-mon[35032]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:46.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:46.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:46.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:46.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:46 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:46 smithi097 ceph-mon[34864]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:46.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:46.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:46.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:23:46.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:46 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:46.906 DEBUG:teuthology.orchestra.run.smithi016:osd.9> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T09:23:46.908 INFO:tasks.cephadm:Deploying osd.10 on smithi097 with /dev/vg_nvme/lv_1... 2024-09-19T09:23:46.909 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T09:23:47.163 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:23:48.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:48 smithi016 ceph-mon[30979]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:48.840 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:48 smithi097 ceph-mon[34864]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:48.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:48 smithi142 ceph-mon[35032]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:49.253 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:23:48 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:23:48.884+0000 7f4a090ad640 -1 osd.9 0 log_to_monitors true 2024-09-19T09:23:49.521 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-19T09:23:49.546 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2024-09-19T09:23:49.564 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:49 smithi097 ceph-mon[34864]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T09:23:49.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:49 smithi016 ceph-mon[30979]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T09:23:49.795 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:23:49.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:49 smithi142 ceph-mon[35032]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T09:23:50.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.241+0000 7fe7991e9640 1 -- 172.21.15.97:0/3609016040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe794100ad0 msgr2=0x7fe794100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.241+0000 7fe7991e9640 1 --2- 172.21.15.97:0/3609016040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe794100ad0 0x7fe794100ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe788009900 tx=0x7fe78802a410 comp rx=0 tx=0).stop 2024-09-19T09:23:50.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.241+0000 7fe7991e9640 1 -- 172.21.15.97:0/3609016040 shutdown_connections 2024-09-19T09:23:50.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.241+0000 7fe7991e9640 1 --2- 172.21.15.97:0/3609016040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe794101cd0 0x7fe794102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.241+0000 7fe7991e9640 1 --2- 172.21.15.97:0/3609016040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794102690 0x7fe794108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.241+0000 7fe7991e9640 1 --2- 172.21.15.97:0/3609016040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe794100ad0 0x7fe794100ed0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.241+0000 7fe7991e9640 1 -- 172.21.15.97:0/3609016040 >> 172.21.15.97:0/3609016040 conn(0x7fe7940fc280 msgr2=0x7fe7940fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:23:50.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.241+0000 7fe7991e9640 1 -- 172.21.15.97:0/3609016040 shutdown_connections 2024-09-19T09:23:50.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.241+0000 7fe7991e9640 1 -- 172.21.15.97:0/3609016040 wait complete. 2024-09-19T09:23:50.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.242+0000 7fe7991e9640 1 Processor -- start 2024-09-19T09:23:50.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.242+0000 7fe7991e9640 1 -- start start 2024-09-19T09:23:50.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.242+0000 7fe7991e9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe794100ad0 0x7fe794196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.242+0000 7fe7991e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794101cd0 0x7fe794197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.242+0000 7fe7991e9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe794102690 0x7fe79419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.242+0000 7fe7991e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe79410c730 con 0x7fe794101cd0 2024-09-19T09:23:50.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.242+0000 7fe7991e9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe79410c5b0 con 0x7fe794102690 2024-09-19T09:23:50.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.242+0000 7fe7991e9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe79410c8b0 con 0x7fe794100ad0 2024-09-19T09:23:50.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.242+0000 7fe79359e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe794102690 0x7fe79419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe79359e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe794102690 0x7fe79419e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58276/0 (socket says 172.21.15.97:58276) 2024-09-19T09:23:50.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe79359e640 1 -- 172.21.15.97:0/3530418147 learned_addr learned my addr 172.21.15.97:0/3530418147 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-19T09:23:50.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe79259c640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794101cd0 0x7fe794197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe792d9d640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe794100ad0 0x7fe794196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe79359e640 1 -- 172.21.15.97:0/3530418147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe794100ad0 msgr2=0x7fe794196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe79359e640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe794100ad0 0x7fe794196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe79359e640 1 -- 172.21.15.97:0/3530418147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794101cd0 msgr2=0x7fe794197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:23:50.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe79359e640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794101cd0 0x7fe794197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:23:50.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe79359e640 1 -- 172.21.15.97:0/3530418147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79419e940 con 0x7fe794102690 2024-09-19T09:23:50.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe792d9d640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe794100ad0 0x7fe794196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:23:50.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe79359e640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe794102690 0x7fe79419e2a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe78c007cb0 tx=0x7fe78c00e420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:50.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe77bfff640 1 -- 172.21.15.97:0/3530418147 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c007e00 con 0x7fe794102690 2024-09-19T09:23:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe7991e9640 1 -- 172.21.15.97:0/3530418147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe794069fd0 con 0x7fe794102690 2024-09-19T09:23:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe77bfff640 1 -- 172.21.15.97:0/3530418147 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78c00eb30 con 0x7fe794102690 2024-09-19T09:23:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe77bfff640 1 -- 172.21.15.97:0/3530418147 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0153f0 con 0x7fe794102690 2024-09-19T09:23:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.243+0000 7fe7991e9640 1 -- 172.21.15.97:0/3530418147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79406a510 con 0x7fe794102690 2024-09-19T09:23:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.244+0000 7fe7991e9640 1 -- 172.21.15.97:0/3530418147 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe794107870 con 0x7fe794102690 2024-09-19T09:23:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.245+0000 7fe77bfff640 1 -- 172.21.15.97:0/3530418147 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe78c004a40 con 0x7fe794102690 2024-09-19T09:23:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.246+0000 7fe77bfff640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe77c0a3240 0x7fe77c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:23:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.246+0000 7fe77bfff640 1 -- 172.21.15.97:0/3530418147 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5965+0+0 (secure 0 0 0) 0x7fe78c0d0eb0 con 0x7fe794102690 2024-09-19T09:23:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.246+0000 7fe792d9d640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe77c0a3240 0x7fe77c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:23:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.247+0000 7fe792d9d640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe77c0a3240 0x7fe77c0a5700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe7880314e0 tx=0x7fe788000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:23:50.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.249+0000 7fe77bfff640 1 -- 172.21.15.97:0/3530418147 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe78c100210 con 0x7fe794102690 2024-09-19T09:23:50.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:23:50.429+0000 7fe7991e9640 1 -- 172.21.15.97:0/3530418147 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe79406a830 con 0x7fe77c0a3240 2024-09-19T09:23:50.816 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:50.816 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T09:23:50.816 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T09:23:50.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-19T09:23:50.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:50.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: Detected new or changed devices on smithi016 2024-09-19T09:23:50.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:50.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:50.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:50.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:50.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:50.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:50.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-19T09:23:50.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:50.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:50.817 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:50.818 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:50.818 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:23:50.818 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:23:50.818 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:50.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T09:23:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T09:23:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-19T09:23:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: Detected new or changed devices on smithi016 2024-09-19T09:23:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-19T09:23:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:23:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:23:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:51.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:51.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T09:23:51.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: osdmap e59: 10 total, 9 up, 10 in 2024-09-19T09:23:51.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-19T09:23:51.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: Detected new or changed devices on smithi016 2024-09-19T09:23:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:23:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-19T09:23:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:23:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:23:51.005 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:23:51.005 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:23:51.005 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:23:51.503 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:23:51 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:23:51.220+0000 7f4a05844640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T09:23:51.503 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:23:51 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:23:51.233+0000 7f49fce3f640 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:23:51.780 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:51 smithi142 ceph-mon[35032]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:51.780 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:51 smithi142 ceph-mon[35032]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-19T09:23:51.780 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:51 smithi142 ceph-mon[35032]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T09:23:51.780 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:51 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:51.780 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:51 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:51.780 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:51 smithi142 ceph-mon[35032]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' 2024-09-19T09:23:51.885 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:51 smithi097 ceph-mon[34864]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:51.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:51 smithi097 ceph-mon[34864]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-19T09:23:51.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:51 smithi097 ceph-mon[34864]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T09:23:51.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:51 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:51.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:51 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:51.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:51 smithi097 ceph-mon[34864]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' 2024-09-19T09:23:52.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:51 smithi016 ceph-mon[30979]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:23:52.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:51 smithi016 ceph-mon[30979]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-19T09:23:52.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:51 smithi016 ceph-mon[30979]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T09:23:52.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:51 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:52.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:51 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:52.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:51 smithi016 ceph-mon[30979]: from='osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360]' entity='osd.9' 2024-09-19T09:23:52.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:52 smithi142 ceph-mon[35032]: purged_snaps scrub starts 2024-09-19T09:23:52.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:52 smithi142 ceph-mon[35032]: purged_snaps scrub ok 2024-09-19T09:23:52.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:52 smithi142 ceph-mon[35032]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:52.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:52 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:52.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:52 smithi142 ceph-mon[35032]: osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360] boot 2024-09-19T09:23:52.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:52 smithi142 ceph-mon[35032]: osdmap e61: 10 total, 10 up, 10 in 2024-09-19T09:23:52.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:52 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:52.885 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:52 smithi097 ceph-mon[34864]: purged_snaps scrub starts 2024-09-19T09:23:52.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:52 smithi097 ceph-mon[34864]: purged_snaps scrub ok 2024-09-19T09:23:52.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:52 smithi097 ceph-mon[34864]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:52.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:52 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:52.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:52 smithi097 ceph-mon[34864]: osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360] boot 2024-09-19T09:23:52.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:52 smithi097 ceph-mon[34864]: osdmap e61: 10 total, 10 up, 10 in 2024-09-19T09:23:52.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:52 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:53.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:52 smithi016 ceph-mon[30979]: purged_snaps scrub starts 2024-09-19T09:23:53.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:52 smithi016 ceph-mon[30979]: purged_snaps scrub ok 2024-09-19T09:23:53.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:52 smithi016 ceph-mon[30979]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-09-19T09:23:53.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:52 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:53.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:52 smithi016 ceph-mon[30979]: osd.9 [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360] boot 2024-09-19T09:23:53.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:52 smithi016 ceph-mon[30979]: osdmap e61: 10 total, 10 up, 10 in 2024-09-19T09:23:53.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:52 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T09:23:53.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:53 smithi097 ceph-mon[34864]: from='client.? 172.21.15.97:0/570811636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fa78f96-cdfe-42a0-8d27-0705a0c7df8e"}]: dispatch 2024-09-19T09:23:53.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:53 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fa78f96-cdfe-42a0-8d27-0705a0c7df8e"}]: dispatch 2024-09-19T09:23:53.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:53 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fa78f96-cdfe-42a0-8d27-0705a0c7df8e"}]': finished 2024-09-19T09:23:53.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:53 smithi097 ceph-mon[34864]: osdmap e62: 11 total, 10 up, 11 in 2024-09-19T09:23:53.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:53 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:23:53.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:53 smithi097 ceph-mon[34864]: from='client.? 172.21.15.97:0/1025022503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:23:53.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:53 smithi142 ceph-mon[35032]: from='client.? 172.21.15.97:0/570811636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fa78f96-cdfe-42a0-8d27-0705a0c7df8e"}]: dispatch 2024-09-19T09:23:53.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:53 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fa78f96-cdfe-42a0-8d27-0705a0c7df8e"}]: dispatch 2024-09-19T09:23:53.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:53 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fa78f96-cdfe-42a0-8d27-0705a0c7df8e"}]': finished 2024-09-19T09:23:53.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:53 smithi142 ceph-mon[35032]: osdmap e62: 11 total, 10 up, 11 in 2024-09-19T09:23:53.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:53 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:23:53.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:53 smithi142 ceph-mon[35032]: from='client.? 172.21.15.97:0/1025022503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:23:54.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:53 smithi016 ceph-mon[30979]: from='client.? 172.21.15.97:0/570811636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fa78f96-cdfe-42a0-8d27-0705a0c7df8e"}]: dispatch 2024-09-19T09:23:54.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:53 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fa78f96-cdfe-42a0-8d27-0705a0c7df8e"}]: dispatch 2024-09-19T09:23:54.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:53 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fa78f96-cdfe-42a0-8d27-0705a0c7df8e"}]': finished 2024-09-19T09:23:54.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:53 smithi016 ceph-mon[30979]: osdmap e62: 11 total, 10 up, 11 in 2024-09-19T09:23:54.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:23:54.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:53 smithi016 ceph-mon[30979]: from='client.? 172.21.15.97:0/1025022503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:23:55.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:54 smithi016 ceph-mon[30979]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:23:55.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:54 smithi016 ceph-mon[30979]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T09:23:55.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:54 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:23:55.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:54 smithi142 ceph-mon[35032]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:23:55.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:54 smithi142 ceph-mon[35032]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T09:23:55.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:54 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:23:55.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:54 smithi097 ceph-mon[34864]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:23:55.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:54 smithi097 ceph-mon[34864]: osdmap e63: 11 total, 10 up, 11 in 2024-09-19T09:23:55.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:54 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:23:57.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:56 smithi142 ceph-mon[35032]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:23:57.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:56 smithi097 ceph-mon[34864]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:23:57.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:56 smithi016 ceph-mon[30979]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:23:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:23:58 smithi142 ceph-mon[35032]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:23:59.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:23:58 smithi097 ceph-mon[34864]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:23:59.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:23:58 smithi016 ceph-mon[30979]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:00.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:00 smithi097 ceph-mon[34864]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:01.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:00 smithi142 ceph-mon[35032]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:01.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:00 smithi016 ceph-mon[30979]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:02.078 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T09:24:02.078 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:01 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:02.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T09:24:02.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:01 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:02.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T09:24:02.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:01 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:03.078 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:02 smithi097 ceph-mon[34864]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:03.078 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:02 smithi097 ceph-mon[34864]: Deploying daemon osd.10 on smithi097 2024-09-19T09:24:03.093 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:02 smithi142 ceph-mon[35032]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:03.094 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:02 smithi142 ceph-mon[35032]: Deploying daemon osd.10 on smithi097 2024-09-19T09:24:03.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:02 smithi016 ceph-mon[30979]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:03.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:02 smithi016 ceph-mon[30979]: Deploying daemon osd.10 on smithi097 2024-09-19T09:24:04.883 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:04 smithi097 ceph-mon[34864]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:05.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:04 smithi142 ceph-mon[35032]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:05.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:04 smithi016 ceph-mon[30979]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:07.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:06 smithi142 ceph-mon[35032]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:07.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:06 smithi097 ceph-mon[34864]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:07.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:06 smithi016 ceph-mon[30979]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:08.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:08.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:08.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:08.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:08.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:08.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:08.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:08.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:08.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:09.066 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:08 smithi097 ceph-mon[34864]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:09.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:08 smithi016 ceph-mon[30979]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:09.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:08 smithi142 ceph-mon[35032]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:24:10.394+0000 7fe77bfff640 1 -- 172.21.15.97:0/3530418147 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe79406a830 con 0x7fe77c0a3240 2024-09-19T09:24:10.397 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 10 on host 'smithi097' 2024-09-19T09:24:10.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:24:10.396+0000 7fe7991e9640 1 -- 172.21.15.97:0/3530418147 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe77c0a3240 msgr2=0x7fe77c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:10.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:24:10.396+0000 7fe7991e9640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe77c0a3240 0x7fe77c0a5700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe7880314e0 tx=0x7fe788000bc0 comp rx=0 tx=0).stop 2024-09-19T09:24:10.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:24:10.396+0000 7fe7991e9640 1 -- 172.21.15.97:0/3530418147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe794102690 msgr2=0x7fe79419e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:10.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:24:10.396+0000 7fe7991e9640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe794102690 0x7fe79419e2a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe78c007cb0 tx=0x7fe78c00e420 comp rx=0 tx=0).stop 2024-09-19T09:24:10.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:24:10.397+0000 7fe7991e9640 1 -- 172.21.15.97:0/3530418147 shutdown_connections 2024-09-19T09:24:10.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:24:10.397+0000 7fe7991e9640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe794102690 0x7fe79419e2a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:10.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:24:10.397+0000 7fe7991e9640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe794101cd0 0x7fe794197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:10.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:24:10.397+0000 7fe7991e9640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe77c0a3240 0x7fe77c0a5700 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:10.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:24:10.397+0000 7fe7991e9640 1 --2- 172.21.15.97:0/3530418147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe794100ad0 0x7fe794196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:10.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:24:10.397+0000 7fe7991e9640 1 -- 172.21.15.97:0/3530418147 >> 172.21.15.97:0/3530418147 conn(0x7fe7940fc280 msgr2=0x7fe794104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:10.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:24:10.397+0000 7fe7991e9640 1 -- 172.21.15.97:0/3530418147 shutdown_connections 2024-09-19T09:24:10.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-19T09:24:10.397+0000 7fe7991e9640 1 -- 172.21.15.97:0/3530418147 wait complete. 2024-09-19T09:24:10.977 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:10 smithi097 ceph-mon[34864]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:10.977 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:10 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:10.977 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:10 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:10.977 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:10 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:10.977 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:10 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:10.977 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:10 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:10.977 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:10 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:10.978 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:10 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:10.978 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:10 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:11.003 DEBUG:teuthology.orchestra.run.smithi097:osd.10> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T09:24:11.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:10 smithi016 ceph-mon[30979]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:11.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:10 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:11.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:10 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:11.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:10 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:11.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:10 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:11.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:10 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:11.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:10 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:11.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:10 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:11.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:10 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:11.047 INFO:tasks.cephadm:Deploying osd.11 on smithi142 with /dev/vg_nvme/lv_1... 2024-09-19T09:24:11.047 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T09:24:11.075 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:10 smithi142 ceph-mon[35032]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:11.075 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:10 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:11.075 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:10 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:11.075 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:10 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:11.075 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:10 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:11.075 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:10 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:11.075 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:10 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:11.075 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:10 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:11.075 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:10 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:11.303 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:24:12.706 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:12 smithi142 ceph-mon[35032]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:13.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:12 smithi016 ceph-mon[30979]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:13.010 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:12 smithi097 ceph-mon[34864]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:13.381 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:24:13 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:24:13.007+0000 7fd75f3d2640 -1 osd.10 0 log_to_monitors true 2024-09-19T09:24:13.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:13 smithi142 ceph-mon[35032]: from='osd.10 [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T09:24:13.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:13 smithi142 ceph-mon[35032]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T09:24:14.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:13 smithi016 ceph-mon[30979]: from='osd.10 [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T09:24:14.005 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:13 smithi016 ceph-mon[30979]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T09:24:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-19T09:24:14.032 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_1 2024-09-19T09:24:14.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:13 smithi097 ceph-mon[34864]: from='osd.10 [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T09:24:14.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:13 smithi097 ceph-mon[34864]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T09:24:14.319 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:24:14.689 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:14.689 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T09:24:14.689 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T09:24:14.689 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:14.689 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='osd.10 [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:24:14.689 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:24:14.689 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: Detected new or changed devices on smithi097 2024-09-19T09:24:14.689 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:14.689 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:14.690 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:14.690 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:14.690 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:14.690 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:14.690 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: Adjusting osd_memory_target on smithi097 to 4272M 2024-09-19T09:24:14.690 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:14.690 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:14.690 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:14.690 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:14.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.770+0000 7fa18ab34640 1 -- 172.21.15.142:0/2862464605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa184101d10 msgr2=0x7fa184102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:14.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.770+0000 7fa18ab34640 1 --2- 172.21.15.142:0/2862464605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa184101d10 0x7fa184102190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa170009900 tx=0x7fa17002a410 comp rx=0 tx=0).stop 2024-09-19T09:24:14.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.770+0000 7fa18ab34640 1 -- 172.21.15.142:0/2862464605 shutdown_connections 2024-09-19T09:24:14.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.770+0000 7fa18ab34640 1 --2- 172.21.15.142:0/2862464605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa184101d10 0x7fa184102190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:14.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.770+0000 7fa18ab34640 1 --2- 172.21.15.142:0/2862464605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1841026d0 0x7fa184108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:14.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.770+0000 7fa18ab34640 1 --2- 172.21.15.142:0/2862464605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa184100b10 0x7fa184100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:14.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.770+0000 7fa18ab34640 1 -- 172.21.15.142:0/2862464605 >> 172.21.15.142:0/2862464605 conn(0x7fa1840fc2a0 msgr2=0x7fa1840fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:14.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.770+0000 7fa18ab34640 1 -- 172.21.15.142:0/2862464605 shutdown_connections 2024-09-19T09:24:14.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.770+0000 7fa18ab34640 1 -- 172.21.15.142:0/2862464605 wait complete. 2024-09-19T09:24:14.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.771+0000 7fa18ab34640 1 Processor -- start 2024-09-19T09:24:14.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.771+0000 7fa18ab34640 1 -- start start 2024-09-19T09:24:14.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.771+0000 7fa18ab34640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa184100b10 0x7fa18419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:14.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.771+0000 7fa18ab34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa184101d10 0x7fa18419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:14.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.771+0000 7fa18ab34640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1841026d0 0x7fa1841a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:14.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.771+0000 7fa18ab34640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa18410c810 con 0x7fa184101d10 2024-09-19T09:24:14.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.771+0000 7fa18ab34640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa18410c690 con 0x7fa184100b10 2024-09-19T09:24:14.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.771+0000 7fa18ab34640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa18410c990 con 0x7fa1841026d0 2024-09-19T09:24:14.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.771+0000 7fa1890d1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1841026d0 0x7fa1841a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:14.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.771+0000 7fa183fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa184101d10 0x7fa18419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:14.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.771+0000 7fa1890d1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1841026d0 0x7fa1841a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45172/0 (socket says 172.21.15.142:45172) 2024-09-19T09:24:14.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.771+0000 7fa1890d1640 1 -- 172.21.15.142:0/1859908872 learned_addr learned my addr 172.21.15.142:0/1859908872 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-19T09:24:14.781 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.771+0000 7fa1888d0640 1 --2- 172.21.15.142:0/1859908872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa184100b10 0x7fa18419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:14.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.772+0000 7fa1890d1640 1 -- 172.21.15.142:0/1859908872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa184100b10 msgr2=0x7fa18419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:14.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.772+0000 7fa1890d1640 1 --2- 172.21.15.142:0/1859908872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa184100b10 0x7fa18419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:14.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.772+0000 7fa1890d1640 1 -- 172.21.15.142:0/1859908872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa184101d10 msgr2=0x7fa18419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:14.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.772+0000 7fa1890d1640 1 --2- 172.21.15.142:0/1859908872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa184101d10 0x7fa18419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:14.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.772+0000 7fa1890d1640 1 -- 172.21.15.142:0/1859908872 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1841a1e10 con 0x7fa1841026d0 2024-09-19T09:24:14.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.772+0000 7fa1890d1640 1 --2- 172.21.15.142:0/1859908872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1841026d0 0x7fa1841a17a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00b990 tx=0x7fa17c00be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:14.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.772+0000 7fa181ffb640 1 -- 172.21.15.142:0/1859908872 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c019070 con 0x7fa1841026d0 2024-09-19T09:24:14.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.772+0000 7fa181ffb640 1 -- 172.21.15.142:0/1859908872 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa17c00cba0 con 0x7fa1841026d0 2024-09-19T09:24:14.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.772+0000 7fa18ab34640 1 -- 172.21.15.142:0/1859908872 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1841a2100 con 0x7fa1841026d0 2024-09-19T09:24:14.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.772+0000 7fa181ffb640 1 -- 172.21.15.142:0/1859908872 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c007650 con 0x7fa1841026d0 2024-09-19T09:24:14.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.772+0000 7fa18ab34640 1 -- 172.21.15.142:0/1859908872 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1841a2690 con 0x7fa1841026d0 2024-09-19T09:24:14.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.773+0000 7fa18ab34640 1 -- 172.21.15.142:0/1859908872 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1841a2950 con 0x7fa1841026d0 2024-09-19T09:24:14.782 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.774+0000 7fa181ffb640 1 -- 172.21.15.142:0/1859908872 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa17c01c070 con 0x7fa1841026d0 2024-09-19T09:24:14.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.775+0000 7fa181ffb640 1 --2- 172.21.15.142:0/1859908872 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa1740a3270 0x7fa1740a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:14.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.775+0000 7fa181ffb640 1 -- 172.21.15.142:0/1859908872 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6414+0+0 (secure 0 0 0) 0x7fa17c0cec20 con 0x7fa1841026d0 2024-09-19T09:24:14.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.775+0000 7fa1888d0640 1 --2- 172.21.15.142:0/1859908872 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa1740a3270 0x7fa1740a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:14.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.776+0000 7fa1888d0640 1 --2- 172.21.15.142:0/1859908872 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa1740a3270 0x7fa1740a5730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa184101b70 tx=0x7fa178005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:14.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.778+0000 7fa181ffb640 1 -- 172.21.15.142:0/1859908872 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa17c09ba60 con 0x7fa1841026d0 2024-09-19T09:24:14.955 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:14.954+0000 7fa18ab34640 1 -- 172.21.15.142:0/1859908872 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa1841a2b20 con 0x7fa1740a3270 2024-09-19T09:24:15.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:15.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T09:24:15.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T09:24:15.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:15.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='osd.10 [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:24:15.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:24:15.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: Detected new or changed devices on smithi097 2024-09-19T09:24:15.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:15.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:15.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:15.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:15.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:15.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:15.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: Adjusting osd_memory_target on smithi097 to 4272M 2024-09-19T09:24:15.005 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:15.005 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:15.005 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:15.005 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:15.137 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:24:14.680+0000 7fd75ab32640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T09:24:15.137 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:24:14.694+0000 7fd75716c640 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:24:15.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:15.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T09:24:15.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: osdmap e64: 11 total, 10 up, 11 in 2024-09-19T09:24:15.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:15.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='osd.10 [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:24:15.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-19T09:24:15.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: Detected new or changed devices on smithi097 2024-09-19T09:24:15.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:15.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:15.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:15.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:15.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:15.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:15.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: Adjusting osd_memory_target on smithi097 to 4272M 2024-09-19T09:24:15.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:15.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:15.138 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:15.139 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:15.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:15 smithi142 ceph-mon[35032]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-19T09:24:15.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:15 smithi142 ceph-mon[35032]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T09:24:15.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:15 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:15.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:15 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:15.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:15 smithi142 ceph-mon[35032]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:15.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:15 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:24:15.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:15 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:24:15.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:15 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:16.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:15 smithi016 ceph-mon[30979]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-19T09:24:16.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:15 smithi016 ceph-mon[30979]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T09:24:16.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:15 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:16.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:15 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:16.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:15 smithi016 ceph-mon[30979]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:16.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:15 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:24:16.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:15 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:24:16.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:15 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:16.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:15 smithi097 ceph-mon[34864]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-19T09:24:16.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:15 smithi097 ceph-mon[34864]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T09:24:16.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:15 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:16.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:15 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:16.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:15 smithi097 ceph-mon[34864]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:16.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:15 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T09:24:16.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:15 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T09:24:16.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:15 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:16.852 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:16 smithi142 ceph-mon[35032]: purged_snaps scrub starts 2024-09-19T09:24:16.852 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:16 smithi142 ceph-mon[35032]: purged_snaps scrub ok 2024-09-19T09:24:16.852 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:16 smithi142 ceph-mon[35032]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:16.852 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:16 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:16.852 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:16 smithi142 ceph-mon[35032]: osd.10 [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486] boot 2024-09-19T09:24:16.852 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:16 smithi142 ceph-mon[35032]: osdmap e66: 11 total, 11 up, 11 in 2024-09-19T09:24:16.852 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:16 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:17.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:16 smithi016 ceph-mon[30979]: purged_snaps scrub starts 2024-09-19T09:24:17.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:16 smithi016 ceph-mon[30979]: purged_snaps scrub ok 2024-09-19T09:24:17.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:16 smithi016 ceph-mon[30979]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:17.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:16 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:17.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:16 smithi016 ceph-mon[30979]: osd.10 [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486] boot 2024-09-19T09:24:17.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:16 smithi016 ceph-mon[30979]: osdmap e66: 11 total, 11 up, 11 in 2024-09-19T09:24:17.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:16 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:17.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:16 smithi097 ceph-mon[34864]: purged_snaps scrub starts 2024-09-19T09:24:17.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:16 smithi097 ceph-mon[34864]: purged_snaps scrub ok 2024-09-19T09:24:17.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:16 smithi097 ceph-mon[34864]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-09-19T09:24:17.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:16 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:17.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:16 smithi097 ceph-mon[34864]: osd.10 [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486] boot 2024-09-19T09:24:17.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:16 smithi097 ceph-mon[34864]: osdmap e66: 11 total, 11 up, 11 in 2024-09-19T09:24:17.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:16 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T09:24:17.783 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:17 smithi142 ceph-mon[35032]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T09:24:17.783 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:17 smithi142 ceph-mon[35032]: from='client.? 172.21.15.142:0/3308677500' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d27f20e3-84ce-470c-9889-cd86906b158d"}]: dispatch 2024-09-19T09:24:17.783 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:17 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d27f20e3-84ce-470c-9889-cd86906b158d"}]: dispatch 2024-09-19T09:24:17.783 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:17 smithi142 ceph-mon[35032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d27f20e3-84ce-470c-9889-cd86906b158d"}]': finished 2024-09-19T09:24:17.783 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:17 smithi142 ceph-mon[35032]: osdmap e68: 12 total, 11 up, 12 in 2024-09-19T09:24:17.783 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:18.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:17 smithi016 ceph-mon[30979]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T09:24:18.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:17 smithi016 ceph-mon[30979]: from='client.? 172.21.15.142:0/3308677500' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d27f20e3-84ce-470c-9889-cd86906b158d"}]: dispatch 2024-09-19T09:24:18.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:17 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d27f20e3-84ce-470c-9889-cd86906b158d"}]: dispatch 2024-09-19T09:24:18.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:17 smithi016 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d27f20e3-84ce-470c-9889-cd86906b158d"}]': finished 2024-09-19T09:24:18.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:17 smithi016 ceph-mon[30979]: osdmap e68: 12 total, 11 up, 12 in 2024-09-19T09:24:18.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:18.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:17 smithi097 ceph-mon[34864]: osdmap e67: 11 total, 11 up, 11 in 2024-09-19T09:24:18.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:17 smithi097 ceph-mon[34864]: from='client.? 172.21.15.142:0/3308677500' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d27f20e3-84ce-470c-9889-cd86906b158d"}]: dispatch 2024-09-19T09:24:18.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:17 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d27f20e3-84ce-470c-9889-cd86906b158d"}]: dispatch 2024-09-19T09:24:18.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:17 smithi097 ceph-mon[34864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d27f20e3-84ce-470c-9889-cd86906b158d"}]': finished 2024-09-19T09:24:18.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:17 smithi097 ceph-mon[34864]: osdmap e68: 12 total, 11 up, 12 in 2024-09-19T09:24:18.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:19.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:18 smithi016 ceph-mon[30979]: pgmap v226: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T09:24:19.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:18 smithi016 ceph-mon[30979]: from='client.? 172.21.15.142:0/4271891046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:24:19.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:18 smithi016 ceph-mon[30979]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-19T09:24:19.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:18 smithi016 ceph-mon[30979]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T09:24:19.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:18 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:19.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:18 smithi142 ceph-mon[35032]: pgmap v226: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T09:24:19.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:18 smithi142 ceph-mon[35032]: from='client.? 172.21.15.142:0/4271891046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:24:19.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:18 smithi142 ceph-mon[35032]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-19T09:24:19.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:18 smithi142 ceph-mon[35032]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T09:24:19.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:18 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:19.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:18 smithi097 ceph-mon[34864]: pgmap v226: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T09:24:19.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:18 smithi097 ceph-mon[34864]: from='client.? 172.21.15.142:0/4271891046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T09:24:19.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:18 smithi097 ceph-mon[34864]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-19T09:24:19.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:18 smithi097 ceph-mon[34864]: osdmap e69: 12 total, 11 up, 12 in 2024-09-19T09:24:19.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:18 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:20.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:20 smithi142 ceph-mon[35032]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T09:24:20.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:20.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:20 smithi097 ceph-mon[34864]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T09:24:20.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:20.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:20 smithi016 ceph-mon[30979]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T09:24:20.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:21.530 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:21 smithi142 ceph-mon[35032]: pgmap v229: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T09:24:21.537 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:21 smithi097 ceph-mon[34864]: pgmap v229: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T09:24:21.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:21 smithi016 ceph-mon[30979]: pgmap v229: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T09:24:23.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:23 smithi142 ceph-mon[35032]: pgmap v230: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T09:24:23.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:23 smithi097 ceph-mon[34864]: pgmap v230: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T09:24:23.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:23 smithi016 ceph-mon[30979]: pgmap v230: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-19T09:24:24.508 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:24 smithi142 ceph-mon[35032]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-19T09:24:24.508 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:24 smithi142 ceph-mon[35032]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-19T09:24:24.508 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:24 smithi142 ceph-mon[35032]: Cluster is now healthy 2024-09-19T09:24:24.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:24 smithi097 ceph-mon[34864]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-19T09:24:24.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:24 smithi097 ceph-mon[34864]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-19T09:24:24.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:24 smithi097 ceph-mon[34864]: Cluster is now healthy 2024-09-19T09:24:24.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:24 smithi016 ceph-mon[30979]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-19T09:24:24.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:24 smithi016 ceph-mon[30979]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-19T09:24:24.679 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:24 smithi016 ceph-mon[30979]: Cluster is now healthy 2024-09-19T09:24:26.435 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:26 smithi142 ceph-mon[35032]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T09:24:26.436 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T09:24:26.436 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:26.436 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:26 smithi142 ceph-mon[35032]: Deploying daemon osd.11 on smithi142 2024-09-19T09:24:26.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:26 smithi097 ceph-mon[34864]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T09:24:26.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T09:24:26.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:26.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:26 smithi097 ceph-mon[34864]: Deploying daemon osd.11 on smithi142 2024-09-19T09:24:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:26 smithi016 ceph-mon[30979]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T09:24:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T09:24:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:26 smithi016 ceph-mon[30979]: Deploying daemon osd.11 on smithi142 2024-09-19T09:24:28.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:28 smithi142 ceph-mon[35032]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T09:24:28.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:28 smithi097 ceph-mon[34864]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T09:24:28.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:28 smithi016 ceph-mon[30979]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T09:24:30.485 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:30 smithi142 ceph-mon[35032]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T09:24:30.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:30 smithi097 ceph-mon[34864]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T09:24:30.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:30 smithi016 ceph-mon[30979]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T09:24:32.469 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:32 smithi142 ceph-mon[35032]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:24:32.469 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:32 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:32.469 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:32 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:32.469 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:32 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:32.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:32 smithi097 ceph-mon[34864]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:24:32.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:32 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:32.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:32 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:32.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:32 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:32.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:32 smithi016 ceph-mon[30979]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:24:32.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:32 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:32.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:32 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:32.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:32 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.054 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:35.053+0000 7fa181ffb640 1 -- 172.21.15.142:0/1859908872 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa1841a2b20 con 0x7fa1740a3270 2024-09-19T09:24:35.055 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 11 on host 'smithi142' 2024-09-19T09:24:35.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:35.056+0000 7fa18ab34640 1 -- 172.21.15.142:0/1859908872 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa1740a3270 msgr2=0x7fa1740a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:35.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:35.056+0000 7fa18ab34640 1 --2- 172.21.15.142:0/1859908872 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa1740a3270 0x7fa1740a5730 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa184101b70 tx=0x7fa178005f90 comp rx=0 tx=0).stop 2024-09-19T09:24:35.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:35.056+0000 7fa18ab34640 1 -- 172.21.15.142:0/1859908872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1841026d0 msgr2=0x7fa1841a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:35.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:35.056+0000 7fa18ab34640 1 --2- 172.21.15.142:0/1859908872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1841026d0 0x7fa1841a17a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00b990 tx=0x7fa17c00be50 comp rx=0 tx=0).stop 2024-09-19T09:24:35.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:35.056+0000 7fa18ab34640 1 -- 172.21.15.142:0/1859908872 shutdown_connections 2024-09-19T09:24:35.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:35.056+0000 7fa18ab34640 1 --2- 172.21.15.142:0/1859908872 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa1740a3270 0x7fa1740a5730 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:35.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:35.056+0000 7fa18ab34640 1 --2- 172.21.15.142:0/1859908872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1841026d0 0x7fa1841a17a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:35.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:35.056+0000 7fa18ab34640 1 --2- 172.21.15.142:0/1859908872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa184101d10 0x7fa18419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:35.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:35.056+0000 7fa18ab34640 1 --2- 172.21.15.142:0/1859908872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa184100b10 0x7fa18419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:35.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:35.056+0000 7fa18ab34640 1 -- 172.21.15.142:0/1859908872 >> 172.21.15.142:0/1859908872 conn(0x7fa1840fc2a0 msgr2=0x7fa18406a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:35.058 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:35.056+0000 7fa18ab34640 1 -- 172.21.15.142:0/1859908872 shutdown_connections 2024-09-19T09:24:35.059 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:35.056+0000 7fa18ab34640 1 -- 172.21.15.142:0/1859908872 wait complete. 2024-09-19T09:24:35.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:35 smithi142 ceph-mon[35032]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:24:35.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:35 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:35 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:35 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:35.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:35 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:35.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:35 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:35 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:35.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:35 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:35 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:35 smithi016 ceph-mon[30979]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:24:35.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:35 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:35 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:35 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:35.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:35 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:35.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:35 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:35 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:35.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:35 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:35 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:35 smithi097 ceph-mon[34864]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T09:24:35.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:35 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:35 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:35 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:35.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:35 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:35.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:35 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:35 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:35.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:35 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:35 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:35.781 DEBUG:teuthology.orchestra.run.smithi142:osd.11> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T09:24:35.784 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T09:24:35.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T09:24:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:36.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.504+0000 7f6901aae640 1 -- 172.21.15.16:0/710787242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc1026a0 msgr2=0x7f68fc104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:36.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.504+0000 7f6901aae640 1 --2- 172.21.15.16:0/710787242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc1026a0 0x7f68fc104a90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009900 tx=0x7f68ec02a410 comp rx=0 tx=0).stop 2024-09-19T09:24:36.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.504+0000 7f6901aae640 1 -- 172.21.15.16:0/710787242 shutdown_connections 2024-09-19T09:24:36.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.504+0000 7f6901aae640 1 --2- 172.21.15.16:0/710787242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68fc107900 0x7f68fc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:36.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.505+0000 7f6901aae640 1 --2- 172.21.15.16:0/710787242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68fc104fd0 0x7f68fc1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:36.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.505+0000 7f6901aae640 1 --2- 172.21.15.16:0/710787242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc1026a0 0x7f68fc104a90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:36.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.505+0000 7f6901aae640 1 -- 172.21.15.16:0/710787242 >> 172.21.15.16:0/710787242 conn(0x7f68fc0fc2a0 msgr2=0x7f68fc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:36.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.505+0000 7f6901aae640 1 -- 172.21.15.16:0/710787242 shutdown_connections 2024-09-19T09:24:36.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.505+0000 7f6901aae640 1 -- 172.21.15.16:0/710787242 wait complete. 2024-09-19T09:24:36.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.505+0000 7f6901aae640 1 Processor -- start 2024-09-19T09:24:36.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.505+0000 7f6901aae640 1 -- start start 2024-09-19T09:24:36.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f6901aae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68fc1026a0 0x7f68fc19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:36.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f6901aae640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68fc104fd0 0x7f68fc19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:36.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f6901aae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc107900 0x7f68fc1a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:36.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f6901aae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc10d8b0 con 0x7f68fc107900 2024-09-19T09:24:36.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f6901aae640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc10d730 con 0x7f68fc1026a0 2024-09-19T09:24:36.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f6901aae640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc10da30 con 0x7f68fc104fd0 2024-09-19T09:24:36.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc107900 0x7f68fc1a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:36.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68faffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68fc104fd0 0x7f68fc19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:36.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68fb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68fc1026a0 0x7f68fc19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:36.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc107900 0x7f68fc1a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56374/0 (socket says 172.21.15.16:56374) 2024-09-19T09:24:36.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68fbfff640 1 -- 172.21.15.16:0/1582079309 learned_addr learned my addr 172.21.15.16:0/1582079309 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:24:36.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68faffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68fc104fd0 0x7f68fc19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:48698/0 (socket says 172.21.15.16:48698) 2024-09-19T09:24:36.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68fbfff640 1 -- 172.21.15.16:0/1582079309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68fc104fd0 msgr2=0x7f68fc19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:36.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68fbfff640 1 --2- 172.21.15.16:0/1582079309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68fc104fd0 0x7f68fc19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:36.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68fbfff640 1 -- 172.21.15.16:0/1582079309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68fc1026a0 msgr2=0x7f68fc19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:36.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68fbfff640 1 --2- 172.21.15.16:0/1582079309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68fc1026a0 0x7f68fc19a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:36.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68fbfff640 1 -- 172.21.15.16:0/1582079309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc1a1db0 con 0x7f68fc107900 2024-09-19T09:24:36.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68fbfff640 1 --2- 172.21.15.16:0/1582079309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc107900 0x7f68fc1a1740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009fd0 tx=0x7f68f400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:36.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68f8ff9640 1 -- 172.21.15.16:0/1582079309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4017070 con 0x7f68fc107900 2024-09-19T09:24:36.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.506+0000 7f68f8ff9640 1 -- 172.21.15.16:0/1582079309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68f4003e50 con 0x7f68fc107900 2024-09-19T09:24:36.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.507+0000 7f6901aae640 1 -- 172.21.15.16:0/1582079309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68fc1a20a0 con 0x7f68fc107900 2024-09-19T09:24:36.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.507+0000 7f68f8ff9640 1 -- 172.21.15.16:0/1582079309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f40106c0 con 0x7f68fc107900 2024-09-19T09:24:36.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.507+0000 7f6901aae640 1 -- 172.21.15.16:0/1582079309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68fc1a25e0 con 0x7f68fc107900 2024-09-19T09:24:36.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.508+0000 7f6901aae640 1 -- 172.21.15.16:0/1582079309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68fc109450 con 0x7f68fc107900 2024-09-19T09:24:36.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.509+0000 7f68f8ff9640 1 -- 172.21.15.16:0/1582079309 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f68f4003970 con 0x7f68fc107900 2024-09-19T09:24:36.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.510+0000 7f68f8ff9640 1 --2- 172.21.15.16:0/1582079309 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f68e40a32f0 0x7f68e40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:36.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.510+0000 7f68fb7fe640 1 --2- 172.21.15.16:0/1582079309 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f68e40a32f0 0x7f68e40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:36.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.510+0000 7f68f8ff9640 1 -- 172.21.15.16:0/1582079309 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6809+0+0 (secure 0 0 0) 0x7f68f40cef50 con 0x7f68fc107900 2024-09-19T09:24:36.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.510+0000 7f68fb7fe640 1 --2- 172.21.15.16:0/1582079309 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f68e40a32f0 0x7f68e40a57b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f68ec0314e0 tx=0x7f68ec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:36.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.513+0000 7f68f8ff9640 1 -- 172.21.15.16:0/1582079309 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f68f409bc80 con 0x7f68fc107900 2024-09-19T09:24:36.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.691+0000 7f6901aae640 1 -- 172.21.15.16:0/1582079309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f68fc062df0 con 0x7f68fc107900 2024-09-19T09:24:36.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.692+0000 7f68f8ff9640 1 -- 172.21.15.16:0/1582079309 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f68f409ff80 con 0x7f68fc107900 2024-09-19T09:24:36.694 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:24:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.695+0000 7f6901aae640 1 -- 172.21.15.16:0/1582079309 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f68e40a32f0 msgr2=0x7f68e40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.695+0000 7f6901aae640 1 --2- 172.21.15.16:0/1582079309 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f68e40a32f0 0x7f68e40a57b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f68ec0314e0 tx=0x7f68ec000bc0 comp rx=0 tx=0).stop 2024-09-19T09:24:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.695+0000 7f6901aae640 1 -- 172.21.15.16:0/1582079309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc107900 msgr2=0x7f68fc1a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.695+0000 7f6901aae640 1 --2- 172.21.15.16:0/1582079309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc107900 0x7f68fc1a1740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009fd0 tx=0x7f68f400b260 comp rx=0 tx=0).stop 2024-09-19T09:24:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.695+0000 7f6901aae640 1 -- 172.21.15.16:0/1582079309 shutdown_connections 2024-09-19T09:24:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.695+0000 7f6901aae640 1 --2- 172.21.15.16:0/1582079309 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f68e40a32f0 0x7f68e40a57b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.695+0000 7f6901aae640 1 --2- 172.21.15.16:0/1582079309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68fc104fd0 0x7f68fc19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.695+0000 7f6901aae640 1 --2- 172.21.15.16:0/1582079309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc107900 0x7f68fc1a1740 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.695+0000 7f6901aae640 1 --2- 172.21.15.16:0/1582079309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68fc1026a0 0x7f68fc19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.695+0000 7f6901aae640 1 -- 172.21.15.16:0/1582079309 >> 172.21.15.16:0/1582079309 conn(0x7f68fc0fc2a0 msgr2=0x7f68fc101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.695+0000 7f6901aae640 1 -- 172.21.15.16:0/1582079309 shutdown_connections 2024-09-19T09:24:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:36.695+0000 7f6901aae640 1 -- 172.21.15.16:0/1582079309 wait complete. 2024-09-19T09:24:37.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:37 smithi016 ceph-mon[30979]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:24:37.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:37 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1582079309' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:24:37.431 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1726737855,"num_in_osds":12,"osd_in_since":1726737857,"num_remapped_pgs":0} 2024-09-19T09:24:37.511 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:37 smithi142 ceph-mon[35032]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:24:37.511 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:37 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1582079309' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:24:37.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:37 smithi097 ceph-mon[34864]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:24:37.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:37 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1582079309' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:24:37.836 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:24:37 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:24:37.509+0000 7ff14db63640 -1 osd.11 0 log_to_monitors true 2024-09-19T09:24:38.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:38 smithi016 ceph-mon[30979]: from='osd.11 [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T09:24:38.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:38 smithi016 ceph-mon[30979]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T09:24:38.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T09:24:38.454 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:38 smithi142 ceph-mon[35032]: from='osd.11 [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T09:24:38.454 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:38 smithi142 ceph-mon[35032]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T09:24:38.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:38 smithi097 ceph-mon[34864]: from='osd.11 [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T09:24:38.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:38 smithi097 ceph-mon[34864]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T09:24:38.693 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.103+0000 7f41f8fa1640 1 -- 172.21.15.16:0/1507496095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f4101d10 msgr2=0x7f41f4102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.103+0000 7f41f8fa1640 1 --2- 172.21.15.16:0/1507496095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f4101d10 0x7f41f4102190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009900 tx=0x7f41dc02a410 comp rx=0 tx=0).stop 2024-09-19T09:24:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.104+0000 7f41f8fa1640 1 -- 172.21.15.16:0/1507496095 shutdown_connections 2024-09-19T09:24:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.104+0000 7f41f8fa1640 1 --2- 172.21.15.16:0/1507496095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f41026d0 0x7f41f4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.104+0000 7f41f8fa1640 1 --2- 172.21.15.16:0/1507496095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f4101d10 0x7f41f4102190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.104+0000 7f41f8fa1640 1 --2- 172.21.15.16:0/1507496095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f4100b10 0x7f41f4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.104+0000 7f41f8fa1640 1 -- 172.21.15.16:0/1507496095 >> 172.21.15.16:0/1507496095 conn(0x7f41f40fc2a0 msgr2=0x7f41f40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.104+0000 7f41f8fa1640 1 -- 172.21.15.16:0/1507496095 shutdown_connections 2024-09-19T09:24:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.104+0000 7f41f8fa1640 1 -- 172.21.15.16:0/1507496095 wait complete. 2024-09-19T09:24:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.105+0000 7f41f8fa1640 1 Processor -- start 2024-09-19T09:24:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.105+0000 7f41f8fa1640 1 -- start start 2024-09-19T09:24:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.105+0000 7f41f8fa1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f4100b10 0x7f41f410b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.105+0000 7f41f8fa1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f4101d10 0x7f41f4109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.105+0000 7f41f8fa1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f41026d0 0x7f41f410a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.105+0000 7f41f8fa1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41f410d8c0 con 0x7f41f4101d10 2024-09-19T09:24:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.105+0000 7f41f8fa1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41f410d740 con 0x7f41f41026d0 2024-09-19T09:24:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.105+0000 7f41f8fa1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41f410da40 con 0x7f41f4100b10 2024-09-19T09:24:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.105+0000 7f41ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f4101d10 0x7f41f4109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.105+0000 7f41ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f4101d10 0x7f41f4109ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56388/0 (socket says 172.21.15.16:56388) 2024-09-19T09:24:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.105+0000 7f41ebfff640 1 -- 172.21.15.16:0/651575436 learned_addr learned my addr 172.21.15.16:0/651575436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:24:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41f259c640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f4100b10 0x7f41f410b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41f2d9d640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f41026d0 0x7f41f410a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41ebfff640 1 -- 172.21.15.16:0/651575436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f4100b10 msgr2=0x7f41f410b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41ebfff640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f4100b10 0x7f41f410b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41ebfff640 1 -- 172.21.15.16:0/651575436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f41026d0 msgr2=0x7f41f410a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41ebfff640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f41026d0 0x7f41f410a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41ebfff640 1 -- 172.21.15.16:0/651575436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f410a8e0 con 0x7f41f4101d10 2024-09-19T09:24:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41f259c640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f4100b10 0x7f41f410b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41f2d9d640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f41026d0 0x7f41f410a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41ebfff640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f4101d10 0x7f41f4109ae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009230 tx=0x7f41dc004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41eb7fe640 1 -- 172.21.15.16:0/651575436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc03f030 con 0x7f41f4101d10 2024-09-19T09:24:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41eb7fe640 1 -- 172.21.15.16:0/651575436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41dc003ce0 con 0x7f41f4101d10 2024-09-19T09:24:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41f8fa1640 1 -- 172.21.15.16:0/651575436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f41a2050 con 0x7f41f4101d10 2024-09-19T09:24:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41eb7fe640 1 -- 172.21.15.16:0/651575436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc035070 con 0x7f41f4101d10 2024-09-19T09:24:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.106+0000 7f41f8fa1640 1 -- 172.21.15.16:0/651575436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f41a2500 con 0x7f41f4101d10 2024-09-19T09:24:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.107+0000 7f41f8fa1640 1 -- 172.21.15.16:0/651575436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f4107f30 con 0x7f41f4101d10 2024-09-19T09:24:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.108+0000 7f41eb7fe640 1 -- 172.21.15.16:0/651575436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f41dc042030 con 0x7f41f4101d10 2024-09-19T09:24:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.109+0000 7f41eb7fe640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41e00a3240 0x7f41e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.109+0000 7f41f259c640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41e00a3240 0x7f41e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.109+0000 7f41eb7fe640 1 -- 172.21.15.16:0/651575436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6831+0+0 (secure 0 0 0) 0x7f41dc0efe80 con 0x7f41f4101d10 2024-09-19T09:24:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.109+0000 7f41f259c640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41e00a3240 0x7f41e00a5700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f41e40058c0 tx=0x7f41e4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.113+0000 7f41eb7fe640 1 -- 172.21.15.16:0/651575436 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f41dc0bd0d0 con 0x7f41f4101d10 2024-09-19T09:24:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.292+0000 7f41f8fa1640 1 -- 172.21.15.16:0/651575436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f41f4062df0 con 0x7f41f4101d10 2024-09-19T09:24:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.292+0000 7f41eb7fe640 1 -- 172.21.15.16:0/651575436 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f41dc0bc9d0 con 0x7f41f4101d10 2024-09-19T09:24:39.293 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:24:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.294+0000 7f41f8fa1640 1 -- 172.21.15.16:0/651575436 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41e00a3240 msgr2=0x7f41e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.294+0000 7f41f8fa1640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41e00a3240 0x7f41e00a5700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f41e40058c0 tx=0x7f41e4005850 comp rx=0 tx=0).stop 2024-09-19T09:24:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.294+0000 7f41f8fa1640 1 -- 172.21.15.16:0/651575436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f4101d10 msgr2=0x7f41f4109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.294+0000 7f41f8fa1640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f4101d10 0x7f41f4109ae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009230 tx=0x7f41dc004360 comp rx=0 tx=0).stop 2024-09-19T09:24:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.294+0000 7f41f8fa1640 1 -- 172.21.15.16:0/651575436 shutdown_connections 2024-09-19T09:24:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.294+0000 7f41f8fa1640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f41026d0 0x7f41f410a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.294+0000 7f41f8fa1640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f4101d10 0x7f41f4109ae0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.294+0000 7f41f8fa1640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41e00a3240 0x7f41e00a5700 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.294+0000 7f41f8fa1640 1 --2- 172.21.15.16:0/651575436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f4100b10 0x7f41f410b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.294+0000 7f41f8fa1640 1 -- 172.21.15.16:0/651575436 >> 172.21.15.16:0/651575436 conn(0x7f41f40fc2a0 msgr2=0x7f41f4105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.294+0000 7f41f8fa1640 1 -- 172.21.15.16:0/651575436 shutdown_connections 2024-09-19T09:24:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:39.294+0000 7f41f8fa1640 1 -- 172.21.15.16:0/651575436 wait complete. 2024-09-19T09:24:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:24:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T09:24:39.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T09:24:39.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:39.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='osd.11 [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:24:39.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:24:39.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:39.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:39.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:39.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:39.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:39.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:39.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:39.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:39.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:39.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:39 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:39.628 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:24:39.193+0000 7ff1492c3640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T09:24:39.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:24:39.207+0000 7ff1458fd640 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T09:24:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:24:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T09:24:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T09:24:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='osd.11 [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:24:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:24:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:39.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:39.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:39.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:39.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:39.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:39.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:39.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:39 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:24:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T09:24:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T09:24:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='osd.11 [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:24:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-09-19T09:24:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:39.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:39.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:39.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T09:24:39.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:39.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:39.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:39.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:39 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:40.082 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726737855,"num_in_osds":12,"osd_in_since":1726737857,"num_remapped_pgs":0} 2024-09-19T09:24:40.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:40 smithi016 ceph-mon[30979]: Detected new or changed devices on smithi142 2024-09-19T09:24:40.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:40 smithi016 ceph-mon[30979]: Adjusting osd_memory_target on smithi142 to 4272M 2024-09-19T09:24:40.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:40 smithi016 ceph-mon[30979]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-19T09:24:40.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:40 smithi016 ceph-mon[30979]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T09:24:40.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:40 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:40.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:40 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:40.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:40 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/651575436' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:24:40.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:40 smithi142 ceph-mon[35032]: Detected new or changed devices on smithi142 2024-09-19T09:24:40.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:40 smithi142 ceph-mon[35032]: Adjusting osd_memory_target on smithi142 to 4272M 2024-09-19T09:24:40.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:40 smithi142 ceph-mon[35032]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-19T09:24:40.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:40 smithi142 ceph-mon[35032]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T09:24:40.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:40 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:40.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:40 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:40.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:40 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/651575436' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:24:40.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:40 smithi097 ceph-mon[34864]: Detected new or changed devices on smithi142 2024-09-19T09:24:40.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:40 smithi097 ceph-mon[34864]: Adjusting osd_memory_target on smithi142 to 4272M 2024-09-19T09:24:40.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:40 smithi097 ceph-mon[34864]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-09-19T09:24:40.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:40 smithi097 ceph-mon[34864]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T09:24:40.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:40 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:40.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:40 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:40.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:40 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/651575436' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:24:41.084 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T09:24:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:41.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:41 smithi016 ceph-mon[30979]: purged_snaps scrub starts 2024-09-19T09:24:41.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:41 smithi016 ceph-mon[30979]: purged_snaps scrub ok 2024-09-19T09:24:41.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:41 smithi016 ceph-mon[30979]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:24:41.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:41 smithi016 ceph-mon[30979]: osd.11 [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878] boot 2024-09-19T09:24:41.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:41 smithi016 ceph-mon[30979]: osdmap e73: 12 total, 12 up, 12 in 2024-09-19T09:24:41.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:41.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:41 smithi142 ceph-mon[35032]: purged_snaps scrub starts 2024-09-19T09:24:41.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:41 smithi142 ceph-mon[35032]: purged_snaps scrub ok 2024-09-19T09:24:41.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:41 smithi142 ceph-mon[35032]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:24:41.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:41 smithi142 ceph-mon[35032]: osd.11 [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878] boot 2024-09-19T09:24:41.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:41 smithi142 ceph-mon[35032]: osdmap e73: 12 total, 12 up, 12 in 2024-09-19T09:24:41.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:41.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:41 smithi097 ceph-mon[34864]: purged_snaps scrub starts 2024-09-19T09:24:41.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:41 smithi097 ceph-mon[34864]: purged_snaps scrub ok 2024-09-19T09:24:41.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:41 smithi097 ceph-mon[34864]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-19T09:24:41.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:41 smithi097 ceph-mon[34864]: osd.11 [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878] boot 2024-09-19T09:24:41.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:41 smithi097 ceph-mon[34864]: osdmap e73: 12 total, 12 up, 12 in 2024-09-19T09:24:41.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T09:24:41.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.943+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/3508748846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c241026d0 msgr2=0x7f7c24108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:41.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.943+0000 7f7c2a1f3640 1 --2- 172.21.15.16:0/3508748846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c241026d0 0x7f7c24108f60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009ca0 tx=0x7f7c1c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.943+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/3508748846 shutdown_connections 2024-09-19T09:24:41.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.943+0000 7f7c2a1f3640 1 --2- 172.21.15.16:0/3508748846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c24101d10 0x7f7c24102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.943+0000 7f7c2a1f3640 1 --2- 172.21.15.16:0/3508748846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c241026d0 0x7f7c24108f60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.943+0000 7f7c2a1f3640 1 --2- 172.21.15.16:0/3508748846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c24100b10 0x7f7c24100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.943+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/3508748846 >> 172.21.15.16:0/3508748846 conn(0x7f7c240fc2a0 msgr2=0x7f7c240fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:41.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.943+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/3508748846 shutdown_connections 2024-09-19T09:24:41.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.943+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/3508748846 wait complete. 2024-09-19T09:24:41.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.943+0000 7f7c2a1f3640 1 Processor -- start 2024-09-19T09:24:41.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.943+0000 7f7c2a1f3640 1 -- start start 2024-09-19T09:24:41.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c2a1f3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c24100b10 0x7f7c2419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c2a1f3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c24101d10 0x7f7c2419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c2a1f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c241026d0 0x7f7c241a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c2a1f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2410c7f0 con 0x7f7c241026d0 2024-09-19T09:24:41.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c2a1f3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2410c670 con 0x7f7c24100b10 2024-09-19T09:24:41.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c2a1f3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2410c970 con 0x7f7c24101d10 2024-09-19T09:24:41.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c23fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c241026d0 0x7f7c241a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c23fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c241026d0 0x7f7c241a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56408/0 (socket says 172.21.15.16:56408) 2024-09-19T09:24:41.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c23fff640 1 -- 172.21.15.16:0/4282176390 learned_addr learned my addr 172.21.15.16:0/4282176390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:24:41.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c22ffd640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c24101d10 0x7f7c2419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c237fe640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c24100b10 0x7f7c2419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c23fff640 1 -- 172.21.15.16:0/4282176390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c24101d10 msgr2=0x7f7c2419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:41.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c23fff640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c24101d10 0x7f7c2419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c23fff640 1 -- 172.21.15.16:0/4282176390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c24100b10 msgr2=0x7f7c2419a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:41.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c23fff640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c24100b10 0x7f7c2419a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:41.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c23fff640 1 -- 172.21.15.16:0/4282176390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c241a1d90 con 0x7f7c241026d0 2024-09-19T09:24:41.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c237fe640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c24100b10 0x7f7c2419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:41.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c23fff640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c241026d0 0x7f7c241a1720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0098d0 tx=0x7f7c1c02f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:41.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.944+0000 7f7c20ff9640 1 -- 172.21.15.16:0/4282176390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c02fc30 con 0x7f7c241026d0 2024-09-19T09:24:41.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.945+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/4282176390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c241a2020 con 0x7f7c241026d0 2024-09-19T09:24:41.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.945+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/4282176390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c241a2630 con 0x7f7c241026d0 2024-09-19T09:24:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.945+0000 7f7c20ff9640 1 -- 172.21.15.16:0/4282176390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c1c003e50 con 0x7f7c241026d0 2024-09-19T09:24:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.945+0000 7f7c20ff9640 1 -- 172.21.15.16:0/4282176390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c004810 con 0x7f7c241026d0 2024-09-19T09:24:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.946+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/4282176390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c10005290 con 0x7f7c241026d0 2024-09-19T09:24:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.947+0000 7f7c20ff9640 1 -- 172.21.15.16:0/4282176390 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7c1c0305c0 con 0x7f7c241026d0 2024-09-19T09:24:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.948+0000 7f7c20ff9640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7c0c0a32f0 0x7f7c0c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.948+0000 7f7c237fe640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7c0c0a32f0 0x7f7c0c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.948+0000 7f7c20ff9640 1 -- 172.21.15.16:0/4282176390 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f7c1c0f0860 con 0x7f7c241026d0 2024-09-19T09:24:41.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.948+0000 7f7c237fe640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7c0c0a32f0 0x7f7c0c0a57b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7c24101b70 tx=0x7f7c1400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:41.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:41.952+0000 7f7c20ff9640 1 -- 172.21.15.16:0/4282176390 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c1c0bda20 con 0x7f7c241026d0 2024-09-19T09:24:42.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.129+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/4282176390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7c10005850 con 0x7f7c241026d0 2024-09-19T09:24:42.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.129+0000 7f7c20ff9640 1 -- 172.21.15.16:0/4282176390 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f7c1c0bd320 con 0x7f7c241026d0 2024-09-19T09:24:42.130 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:24:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.131+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/4282176390 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7c0c0a32f0 msgr2=0x7f7c0c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.131+0000 7f7c2a1f3640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7c0c0a32f0 0x7f7c0c0a57b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7c24101b70 tx=0x7f7c1400a560 comp rx=0 tx=0).stop 2024-09-19T09:24:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.131+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/4282176390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c241026d0 msgr2=0x7f7c241a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.131+0000 7f7c2a1f3640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c241026d0 0x7f7c241a1720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0098d0 tx=0x7f7c1c02f840 comp rx=0 tx=0).stop 2024-09-19T09:24:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.131+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/4282176390 shutdown_connections 2024-09-19T09:24:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.131+0000 7f7c2a1f3640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7c0c0a32f0 0x7f7c0c0a57b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.131+0000 7f7c2a1f3640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c24101d10 0x7f7c2419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.131+0000 7f7c2a1f3640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c241026d0 0x7f7c241a1720 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.131+0000 7f7c2a1f3640 1 --2- 172.21.15.16:0/4282176390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c24100b10 0x7f7c2419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.131+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/4282176390 >> 172.21.15.16:0/4282176390 conn(0x7f7c240fc2a0 msgr2=0x7f7c240fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.132+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/4282176390 shutdown_connections 2024-09-19T09:24:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:42.132+0000 7f7c2a1f3640 1 -- 172.21.15.16:0/4282176390 wait complete. 2024-09-19T09:24:42.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:42 smithi016 ceph-mon[30979]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T09:24:42.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:42 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/4282176390' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:24:42.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:42 smithi142 ceph-mon[35032]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T09:24:42.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:42 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/4282176390' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:24:42.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:42 smithi097 ceph-mon[34864]: osdmap e74: 12 total, 12 up, 12 in 2024-09-19T09:24:42.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:42 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/4282176390' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T09:24:42.779 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"num_osds":12,"num_up_osds":12,"osd_up_since":1726737880,"num_in_osds":12,"osd_in_since":1726737857,"num_remapped_pgs":0} 2024-09-19T09:24:42.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:24:43.033 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:43.275 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:43 smithi016 ceph-mon[30979]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.493+0000 7fd08647e640 1 -- 172.21.15.16:0/781904587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd080102ef0 msgr2=0x7fd08010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.493+0000 7fd08647e640 1 --2- 172.21.15.16:0/781904587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd080102ef0 0x7fd08010da40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd078009ca0 tx=0x7fd07802a950 comp rx=0 tx=0).stop 2024-09-19T09:24:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.493+0000 7fd08647e640 1 -- 172.21.15.16:0/781904587 shutdown_connections 2024-09-19T09:24:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.493+0000 7fd08647e640 1 --2- 172.21.15.16:0/781904587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd080102570 0x7fd0801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.493+0000 7fd08647e640 1 --2- 172.21.15.16:0/781904587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd080102ef0 0x7fd08010da40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.493+0000 7fd08647e640 1 --2- 172.21.15.16:0/781904587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd080100e20 0x7fd080101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.493+0000 7fd08647e640 1 -- 172.21.15.16:0/781904587 >> 172.21.15.16:0/781904587 conn(0x7fd0800fc7d0 msgr2=0x7fd0800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.493+0000 7fd08647e640 1 -- 172.21.15.16:0/781904587 shutdown_connections 2024-09-19T09:24:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.493+0000 7fd08647e640 1 -- 172.21.15.16:0/781904587 wait complete. 2024-09-19T09:24:43.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.493+0000 7fd08647e640 1 Processor -- start 2024-09-19T09:24:43.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd08647e640 1 -- start start 2024-09-19T09:24:43.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd08647e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd080100e20 0x7fd080197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:43.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd08647e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd080102570 0x7fd080197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:43.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd08647e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd080102ef0 0x7fd08019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:43.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd08647e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd080110f80 con 0x7fd080100e20 2024-09-19T09:24:43.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd08647e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd080110e00 con 0x7fd080102ef0 2024-09-19T09:24:43.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd08647e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd080111100 con 0x7fd080102570 2024-09-19T09:24:43.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd07ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd080100e20 0x7fd080197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:43.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd07ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd080100e20 0x7fd080197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56420/0 (socket says 172.21.15.16:56420) 2024-09-19T09:24:43.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd07ffff640 1 -- 172.21.15.16:0/1439584236 learned_addr learned my addr 172.21.15.16:0/1439584236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:24:43.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd077fff640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd080102570 0x7fd080197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:43.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd084a1b640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd080102ef0 0x7fd08019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:43.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd07ffff640 1 -- 172.21.15.16:0/1439584236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd080102570 msgr2=0x7fd080197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:43.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd07ffff640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd080102570 0x7fd080197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:43.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd07ffff640 1 -- 172.21.15.16:0/1439584236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd080102ef0 msgr2=0x7fd08019bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:43.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd07ffff640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd080102ef0 0x7fd08019bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:43.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd07ffff640 1 -- 172.21.15.16:0/1439584236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08019c2b0 con 0x7fd080100e20 2024-09-19T09:24:43.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd084a1b640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd080102ef0 0x7fd08019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:43.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.494+0000 7fd077fff640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd080102570 0x7fd080197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:43.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.495+0000 7fd07ffff640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd080100e20 0x7fd080197300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd07000d950 tx=0x7fd07000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:43.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.495+0000 7fd07dffb640 1 -- 172.21.15.16:0/1439584236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070017070 con 0x7fd080100e20 2024-09-19T09:24:43.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.495+0000 7fd08647e640 1 -- 172.21.15.16:0/1439584236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08019c5a0 con 0x7fd080100e20 2024-09-19T09:24:43.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.495+0000 7fd07dffb640 1 -- 172.21.15.16:0/1439584236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd070009c30 con 0x7fd080100e20 2024-09-19T09:24:43.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.495+0000 7fd07dffb640 1 -- 172.21.15.16:0/1439584236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07000f6b0 con 0x7fd080100e20 2024-09-19T09:24:43.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.495+0000 7fd08647e640 1 -- 172.21.15.16:0/1439584236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0801a3e80 con 0x7fd080100e20 2024-09-19T09:24:43.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.496+0000 7fd08647e640 1 -- 172.21.15.16:0/1439584236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd080101200 con 0x7fd080100e20 2024-09-19T09:24:43.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.497+0000 7fd07dffb640 1 -- 172.21.15.16:0/1439584236 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd0700109d0 con 0x7fd080100e20 2024-09-19T09:24:43.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.498+0000 7fd07dffb640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd06c0a3240 0x7fd06c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:43.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.498+0000 7fd077fff640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd06c0a3240 0x7fd06c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:43.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.498+0000 7fd07dffb640 1 -- 172.21.15.16:0/1439584236 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fd0700cf560 con 0x7fd080100e20 2024-09-19T09:24:43.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.498+0000 7fd077fff640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd06c0a3240 0x7fd06c0a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd0800fec10 tx=0x7fd06800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:43.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.502+0000 7fd07dffb640 1 -- 172.21.15.16:0/1439584236 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd070009de0 con 0x7fd080100e20 2024-09-19T09:24:43.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:43 smithi142 ceph-mon[35032]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:43.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:43 smithi097 ceph-mon[34864]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:43.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.678+0000 7fd08647e640 1 -- 172.21.15.16:0/1439584236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0801a40b0 con 0x7fd080100e20 2024-09-19T09:24:43.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.680+0000 7fd07dffb640 1 -- 172.21.15.16:0/1439584236 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+18431 (secure 0 0 0) 0x7fd07009c680 con 0x7fd080100e20 2024-09-19T09:24:43.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:24:43.681 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","created":"2024-09-19T09:17:48.242238+0000","modified":"2024-09-19T09:24:41.201000+0000","last_up_change":"2024-09-19T09:24:40.190289+0000","last_in_change":"2024-09-19T09:24:17.190380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:20:57.748686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbf3b28b-2282-4057-9e65-19ceb78ff19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6803","nonce":1578957623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6805","nonce":1578957623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6809","nonce":1578957623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6807","nonce":1578957623}]},"public_addr":"172.21.15.16:6803/1578957623","cluster_addr":"172.21.15.16:6805/1578957623","heartbeat_back_addr":"172.21.15.16:6809/1578957623","heartbeat_front_addr":"172.21.15.16:6807/1578957623","state":["exists","up"]},{"osd":1,"uuid":"c2517aeb-1d79-41ad-a180-369eecb9732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6801","nonce":3804152349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6803","nonce":3804152349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6807","nonce":3804152349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6805","nonce":3804152349}]},"public_addr":"172.21.15.97:6801/3804152349","cluster_addr":"172.21.15.97:6803/3804152349","heartbeat_back_addr":"172.21.15.97:6807/3804152349","heartbeat_front_addr":"172.21.15.97:6805/3804152349","state":["exists","up"]},{"osd":2,"uuid":"3c367518-5085-471a-98cb-ec5bf5facdf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6801","nonce":2692613951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6803","nonce":2692613951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6807","nonce":2692613951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6805","nonce":2692613951}]},"public_addr":"172.21.15.142:6801/2692613951","cluster_addr":"172.21.15.142:6803/2692613951","heartbeat_back_addr":"172.21.15.142:6807/2692613951","heartbeat_front_addr":"172.21.15.142:6805/2692613951","state":["exists","up"]},{"osd":3,"uuid":"e7118925-f0fa-46b3-8552-3e3bb76a6489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6811","nonce":3169290004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6813","nonce":3169290004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6817","nonce":3169290004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6815","nonce":3169290004}]},"public_addr":"172.21.15.16:6811/3169290004","cluster_addr":"172.21.15.16:6813/3169290004","heartbeat_back_addr":"172.21.15.16:6817/3169290004","heartbeat_front_addr":"172.21.15.16:6815/3169290004","state":["exists","up"]},{"osd":4,"uuid":"dc1964f7-af9c-4853-843b-9818f87c7768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6809","nonce":3491353825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6811","nonce":3491353825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6815","nonce":3491353825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6813","nonce":3491353825}]},"public_addr":"172.21.15.97:6809/3491353825","cluster_addr":"172.21.15.97:6811/3491353825","heartbeat_back_addr":"172.21.15.97:6815/3491353825","heartbeat_front_addr":"172.21.15.97:6813/3491353825","state":["exists","up"]},{"osd":5,"uuid":"4d7fbafc-d9ba-4036-a89d-aa295b8a23a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6809","nonce":577099986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6811","nonce":577099986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6815","nonce":577099986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6813","nonce":577099986}]},"public_addr":"172.21.15.142:6809/577099986","cluster_addr":"172.21.15.142:6811/577099986","heartbeat_back_addr":"172.21.15.142:6815/577099986","heartbeat_front_addr":"172.21.15.142:6813/577099986","state":["exists","up"]},{"osd":6,"uuid":"e3f346fc-c027-44f8-bd37-bccf6ec09e3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6819","nonce":4053539536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6821","nonce":4053539536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6825","nonce":4053539536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6823","nonce":4053539536}]},"public_addr":"172.21.15.16:6819/4053539536","cluster_addr":"172.21.15.16:6821/4053539536","heartbeat_back_addr":"172.21.15.16:6825/4053539536","heartbeat_front_addr":"172.21.15.16:6823/4053539536","state":["exists","up"]},{"osd":7,"uuid":"d390b547-8949-4d4b-8481-c49d6ae7e424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6817","nonce":1157122670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6819","nonce":1157122670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6823","nonce":1157122670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6821","nonce":1157122670}]},"public_addr":"172.21.15.97:6817/1157122670","cluster_addr":"172.21.15.97:6819/1157122670","heartbeat_back_addr":"172.21.15.97:6823/1157122670","heartbeat_front_addr":"172.21.15.97:6821/1157122670","state":["exists","up"]},{"osd":8,"uuid":"78a476c4-5deb-4f9e-a266-db4493850732","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6817","nonce":643795004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6819","nonce":643795004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6823","nonce":643795004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6821","nonce":643795004}]},"public_addr":"172.21.15.142:6817/643795004","cluster_addr":"172.21.15.142:6819/643795004","heartbeat_back_addr":"172.21.15.142:6823/643795004","heartbeat_front_addr":"172.21.15.142:6821/643795004","state":["exists","up"]},{"osd":9,"uuid":"0415d187-4a77-42f7-b8bb-2d0b52e9cb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6827","nonce":2651132360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6829","nonce":2651132360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2651132360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6831","nonce":2651132360}]},"public_addr":"172.21.15.16:6827/2651132360","cluster_addr":"172.21.15.16:6829/2651132360","heartbeat_back_addr":"172.21.15.16:6833/2651132360","heartbeat_front_addr":"172.21.15.16:6831/2651132360","state":["exists","up"]},{"osd":10,"uuid":"6fa78f96-cdfe-42a0-8d27-0705a0c7df8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6825","nonce":1848371486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6827","nonce":1848371486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6831","nonce":1848371486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6829","nonce":1848371486}]},"public_addr":"172.21.15.97:6825/1848371486","cluster_addr":"172.21.15.97:6827/1848371486","heartbeat_back_addr":"172.21.15.97:6831/1848371486","heartbeat_front_addr":"172.21.15.97:6829/1848371486","state":["exists","up"]},{"osd":11,"uuid":"d27f20e3-84ce-470c-9889-cd86906b158d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6825","nonce":155083878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6827","nonce":155083878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6831","nonce":155083878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6829","nonce":155083878}]},"public_addr":"172.21.15.142:6825/155083878","cluster_addr":"172.21.15.142:6827/155083878","heartbeat_back_addr":"172.21.15.142:6831/155083878","heartbeat_front_addr":"172.21.15.142:6829/155083878","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:05.088764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:29.928529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:55.236930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:20.379471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:44.984151+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:10.410074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:35.233553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:59.928700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:24.503069+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:49.853073+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:14.006841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:38.546467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3710267992":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3740275561":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/2755421444":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/357390846":"2024-09-20T09:18:17.070034+0000","172.21.15.16:6800/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3104087402":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3546947161":"2024-09-20T09:18:17.070034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:24:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.683+0000 7fd08647e640 1 -- 172.21.15.16:0/1439584236 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd06c0a3240 msgr2=0x7fd06c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.683+0000 7fd08647e640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd06c0a3240 0x7fd06c0a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd0800fec10 tx=0x7fd06800a5c0 comp rx=0 tx=0).stop 2024-09-19T09:24:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.683+0000 7fd08647e640 1 -- 172.21.15.16:0/1439584236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd080100e20 msgr2=0x7fd080197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.683+0000 7fd08647e640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd080100e20 0x7fd080197300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd07000d950 tx=0x7fd07000de10 comp rx=0 tx=0).stop 2024-09-19T09:24:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.683+0000 7fd08647e640 1 -- 172.21.15.16:0/1439584236 shutdown_connections 2024-09-19T09:24:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.683+0000 7fd08647e640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd06c0a3240 0x7fd06c0a5700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.683+0000 7fd08647e640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd080102570 0x7fd080197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.683+0000 7fd08647e640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd080102ef0 0x7fd08019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.683+0000 7fd08647e640 1 --2- 172.21.15.16:0/1439584236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd080100e20 0x7fd080197300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.683+0000 7fd08647e640 1 -- 172.21.15.16:0/1439584236 >> 172.21.15.16:0/1439584236 conn(0x7fd0800fc7d0 msgr2=0x7fd0801067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.683+0000 7fd08647e640 1 -- 172.21.15.16:0/1439584236 shutdown_connections 2024-09-19T09:24:43.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:43.683+0000 7fd08647e640 1 -- 172.21.15.16:0/1439584236 wait complete. 2024-09-19T09:24:44.259 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:44 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1439584236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:24:44.294 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T09:20:57.748686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T09:24:44.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T09:24:44.549 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:44.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:44 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1439584236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:24:44.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:44 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1439584236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:24:45.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.019+0000 7fd48b4ed640 1 -- 172.21.15.16:0/466854890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd48410e480 msgr2=0x7fd484110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:45.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.019+0000 7fd48b4ed640 1 --2- 172.21.15.16:0/466854890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd48410e480 0x7fd484110870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009ca0 tx=0x7fd47c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.019+0000 7fd48b4ed640 1 -- 172.21.15.16:0/466854890 shutdown_connections 2024-09-19T09:24:45.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.019+0000 7fd48b4ed640 1 --2- 172.21.15.16:0/466854890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd484101280 0x7fd48410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.019+0000 7fd48b4ed640 1 --2- 172.21.15.16:0/466854890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd48410e480 0x7fd484110870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.019+0000 7fd48b4ed640 1 --2- 172.21.15.16:0/466854890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4841008d0 0x7fd484100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.019+0000 7fd48b4ed640 1 -- 172.21.15.16:0/466854890 >> 172.21.15.16:0/466854890 conn(0x7fd4840fc7a0 msgr2=0x7fd4840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.019+0000 7fd48b4ed640 1 -- 172.21.15.16:0/466854890 shutdown_connections 2024-09-19T09:24:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.019+0000 7fd48b4ed640 1 -- 172.21.15.16:0/466854890 wait complete. 2024-09-19T09:24:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.020+0000 7fd48b4ed640 1 Processor -- start 2024-09-19T09:24:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.020+0000 7fd48b4ed640 1 -- start start 2024-09-19T09:24:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.020+0000 7fd48b4ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4841008d0 0x7fd4841a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.020+0000 7fd48b4ed640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd484101280 0x7fd4841a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.020+0000 7fd48b4ed640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd48410e480 0x7fd48419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.020+0000 7fd48b4ed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd484114240 con 0x7fd4841008d0 2024-09-19T09:24:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.020+0000 7fd48b4ed640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841140c0 con 0x7fd484101280 2024-09-19T09:24:45.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.020+0000 7fd48b4ed640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841143c0 con 0x7fd48410e480 2024-09-19T09:24:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.020+0000 7fd489289640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4841008d0 0x7fd4841a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.020+0000 7fd489289640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4841008d0 0x7fd4841a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56438/0 (socket says 172.21.15.16:56438) 2024-09-19T09:24:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.020+0000 7fd489289640 1 -- 172.21.15.16:0/92652898 learned_addr learned my addr 172.21.15.16:0/92652898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:24:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.020+0000 7fd489a8a640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd48410e480 0x7fd48419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.020+0000 7fd488a88640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd484101280 0x7fd4841a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.021+0000 7fd489289640 1 -- 172.21.15.16:0/92652898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd48410e480 msgr2=0x7fd48419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.021+0000 7fd489289640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd48410e480 0x7fd48419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.021+0000 7fd489289640 1 -- 172.21.15.16:0/92652898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd484101280 msgr2=0x7fd4841a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.021+0000 7fd489289640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd484101280 0x7fd4841a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.021+0000 7fd489289640 1 -- 172.21.15.16:0/92652898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48419ad20 con 0x7fd4841008d0 2024-09-19T09:24:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.021+0000 7fd488a88640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd484101280 0x7fd4841a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:45.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.021+0000 7fd489289640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4841008d0 0x7fd4841a0430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd48000b4a0 tx=0x7fd48000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:45.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.021+0000 7fd47a7fc640 1 -- 172.21.15.16:0/92652898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480014070 con 0x7fd4841008d0 2024-09-19T09:24:45.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.021+0000 7fd47a7fc640 1 -- 172.21.15.16:0/92652898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd480010af0 con 0x7fd4841008d0 2024-09-19T09:24:45.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.021+0000 7fd48b4ed640 1 -- 172.21.15.16:0/92652898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48419b010 con 0x7fd4841008d0 2024-09-19T09:24:45.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.021+0000 7fd47a7fc640 1 -- 172.21.15.16:0/92652898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480017d10 con 0x7fd4841008d0 2024-09-19T09:24:45.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.021+0000 7fd48b4ed640 1 -- 172.21.15.16:0/92652898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4841a7300 con 0x7fd4841008d0 2024-09-19T09:24:45.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.022+0000 7fd48b4ed640 1 -- 172.21.15.16:0/92652898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd484100d30 con 0x7fd4841008d0 2024-09-19T09:24:45.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.023+0000 7fd47a7fc640 1 -- 172.21.15.16:0/92652898 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd480017070 con 0x7fd4841008d0 2024-09-19T09:24:45.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.024+0000 7fd47a7fc640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4700a32c0 0x7fd4700a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:45.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.024+0000 7fd488a88640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4700a32c0 0x7fd4700a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:45.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.025+0000 7fd47a7fc640 1 -- 172.21.15.16:0/92652898 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fd4800cecd0 con 0x7fd4841008d0 2024-09-19T09:24:45.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.025+0000 7fd488a88640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4700a32c0 0x7fd4700a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd474006fd0 tx=0x7fd474008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.028+0000 7fd47a7fc640 1 -- 172.21.15.16:0/92652898 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd48009bdf0 con 0x7fd4841008d0 2024-09-19T09:24:45.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.205+0000 7fd48b4ed640 1 -- 172.21.15.16:0/92652898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd484109080 con 0x7fd4841008d0 2024-09-19T09:24:45.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.206+0000 7fd47a7fc640 1 -- 172.21.15.16:0/92652898 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7fd48009b6f0 con 0x7fd4841008d0 2024-09-19T09:24:45.207 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-19T09:24:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.208+0000 7fd48b4ed640 1 -- 172.21.15.16:0/92652898 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4700a32c0 msgr2=0x7fd4700a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.208+0000 7fd48b4ed640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4700a32c0 0x7fd4700a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd474006fd0 tx=0x7fd474008040 comp rx=0 tx=0).stop 2024-09-19T09:24:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.208+0000 7fd48b4ed640 1 -- 172.21.15.16:0/92652898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4841008d0 msgr2=0x7fd4841a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:45.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.208+0000 7fd48b4ed640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4841008d0 0x7fd4841a0430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd48000b4a0 tx=0x7fd48000b960 comp rx=0 tx=0).stop 2024-09-19T09:24:45.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.208+0000 7fd48b4ed640 1 -- 172.21.15.16:0/92652898 shutdown_connections 2024-09-19T09:24:45.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.208+0000 7fd48b4ed640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4700a32c0 0x7fd4700a5780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.208+0000 7fd48b4ed640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd48410e480 0x7fd48419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.208+0000 7fd48b4ed640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd484101280 0x7fd4841a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.208+0000 7fd48b4ed640 1 --2- 172.21.15.16:0/92652898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4841008d0 0x7fd4841a0430 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.208+0000 7fd48b4ed640 1 -- 172.21.15.16:0/92652898 >> 172.21.15.16:0/92652898 conn(0x7fd4840fc7a0 msgr2=0x7fd4840feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.208+0000 7fd48b4ed640 1 -- 172.21.15.16:0/92652898 shutdown_connections 2024-09-19T09:24:45.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:45.208+0000 7fd48b4ed640 1 -- 172.21.15.16:0/92652898 wait complete. 2024-09-19T09:24:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:45 smithi016 ceph-mon[30979]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:45 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/92652898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:24:45.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:45 smithi142 ceph-mon[35032]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:45.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:45 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/92652898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:24:45.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:45 smithi097 ceph-mon[34864]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:45.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:45 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/92652898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:24:45.845 INFO:tasks.cephadm:Adding mds.a on smithi016 2024-09-19T09:24:45.845 INFO:tasks.cephadm:Adding mds.d on smithi016 2024-09-19T09:24:45.845 INFO:tasks.cephadm:Adding mds.g on smithi016 2024-09-19T09:24:45.845 INFO:tasks.cephadm:Adding mds.j on smithi016 2024-09-19T09:24:45.845 INFO:tasks.cephadm:Adding mds.b on smithi097 2024-09-19T09:24:45.845 INFO:tasks.cephadm:Adding mds.e on smithi097 2024-09-19T09:24:45.845 INFO:tasks.cephadm:Adding mds.h on smithi097 2024-09-19T09:24:45.845 INFO:tasks.cephadm:Adding mds.k on smithi097 2024-09-19T09:24:45.845 INFO:tasks.cephadm:Adding mds.c on smithi142 2024-09-19T09:24:45.845 INFO:tasks.cephadm:Adding mds.f on smithi142 2024-09-19T09:24:45.846 INFO:tasks.cephadm:Adding mds.i on smithi142 2024-09-19T09:24:45.846 INFO:tasks.cephadm:Adding mds.l on smithi142 2024-09-19T09:24:45.846 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l' 2024-09-19T09:24:46.106 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:24:46.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.580+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1253831625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54103be0 msgr2=0x7fec540ffc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.580+0000 7fec5a7c1640 1 --2- 172.21.15.142:0/1253831625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54103be0 0x7fec540ffc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fec48009900 tx=0x7fec4802a410 comp rx=0 tx=0).stop 2024-09-19T09:24:46.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.580+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1253831625 shutdown_connections 2024-09-19T09:24:46.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.580+0000 7fec5a7c1640 1 --2- 172.21.15.142:0/1253831625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54103be0 0x7fec540ffc90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.580+0000 7fec5a7c1640 1 --2- 172.21.15.142:0/1253831625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec54100350 0x7fec5410a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.584 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.580+0000 7fec5a7c1640 1 --2- 172.21.15.142:0/1253831625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec54103210 0x7fec54103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.580+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1253831625 >> 172.21.15.142:0/1253831625 conn(0x7fec540fba40 msgr2=0x7fec540fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:46.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.580+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1253831625 shutdown_connections 2024-09-19T09:24:46.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.580+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1253831625 wait complete. 2024-09-19T09:24:46.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.581+0000 7fec5a7c1640 1 Processor -- start 2024-09-19T09:24:46.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.581+0000 7fec5a7c1640 1 -- start start 2024-09-19T09:24:46.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.581+0000 7fec5a7c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec54100350 0x7fec54102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.581+0000 7fec5a7c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec54103210 0x7fec54100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.581+0000 7fec5a7c1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54103be0 0x7fec541014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.581+0000 7fec5a7c1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec5410d920 con 0x7fec54103210 2024-09-19T09:24:46.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.581+0000 7fec5a7c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fec5410d7a0 con 0x7fec54100350 2024-09-19T09:24:46.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.581+0000 7fec5a7c1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec5410daa0 con 0x7fec54103be0 2024-09-19T09:24:46.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.581+0000 7fec58d5e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54103be0 0x7fec541014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.581+0000 7fec58d5e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54103be0 0x7fec541014a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54122/0 (socket says 172.21.15.142:54122) 2024-09-19T09:24:46.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.581+0000 7fec58d5e640 1 -- 172.21.15.142:0/1400947996 learned_addr learned my addr 172.21.15.142:0/1400947996 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-19T09:24:46.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec537fe640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec54103210 0x7fec54100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec53fff640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec54100350 0x7fec54102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec58d5e640 1 -- 172.21.15.142:0/1400947996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec54100350 msgr2=0x7fec54102910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec58d5e640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec54100350 0x7fec54102910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec58d5e640 1 -- 172.21.15.142:0/1400947996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec54103210 msgr2=0x7fec54100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec58d5e640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec54103210 0x7fec54100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec58d5e640 1 -- 172.21.15.142:0/1400947996 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec54101d60 con 0x7fec54103be0 2024-09-19T09:24:46.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec53fff640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec54100350 0x7fec54102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:46.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec537fe640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec54103210 0x7fec54100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:46.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec58d5e640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54103be0 0x7fec541014a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00cce0 tx=0x7fec4c00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:46.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec517fa640 1 -- 172.21.15.142:0/1400947996 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c00f450 con 0x7fec54103be0 2024-09-19T09:24:46.586 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec517fa640 1 -- 172.21.15.142:0/1400947996 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec4c00fad0 con 0x7fec54103be0 2024-09-19T09:24:46.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1400947996 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec541a2140 con 0x7fec54103be0 2024-09-19T09:24:46.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec517fa640 1 -- 172.21.15.142:0/1400947996 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c018c10 con 0x7fec54103be0 2024-09-19T09:24:46.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.582+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1400947996 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec541a2680 con 0x7fec54103be0 2024-09-19T09:24:46.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.583+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1400947996 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec541035f0 con 0x7fec54103be0 2024-09-19T09:24:46.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.585+0000 7fec517fa640 1 -- 172.21.15.142:0/1400947996 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fec4c00f5f0 con 0x7fec54103be0 2024-09-19T09:24:46.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.585+0000 7fec517fa640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fec3c0a3240 0x7fec3c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:46.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.585+0000 7fec517fa640 1 -- 172.21.15.142:0/1400947996 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fec4c0cfc30 con 0x7fec54103be0 2024-09-19T09:24:46.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.586+0000 7fec53fff640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fec3c0a3240 0x7fec3c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:46.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.586+0000 7fec53fff640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fec3c0a3240 0x7fec3c0a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec5406a490 tx=0x7fec44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:46.589 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.588+0000 7fec517fa640 1 -- 172.21.15.142:0/1400947996 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fec4c014030 con 0x7fec54103be0 2024-09-19T09:24:46.764 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.763+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1400947996 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}) -- 0x7fec54062630 con 0x7fec3c0a3240 2024-09-19T09:24:46.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.771+0000 7fec517fa640 1 -- 172.21.15.142:0/1400947996 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fec54062630 con 0x7fec3c0a3240 2024-09-19T09:24:46.772 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mds.all update... 2024-09-19T09:24:46.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.773+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1400947996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fec3c0a3240 msgr2=0x7fec3c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.773+0000 7fec5a7c1640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fec3c0a3240 0x7fec3c0a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec5406a490 tx=0x7fec44008040 comp rx=0 tx=0).stop 2024-09-19T09:24:46.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.773+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1400947996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54103be0 msgr2=0x7fec541014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:46.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.773+0000 7fec5a7c1640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54103be0 0x7fec541014a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00cce0 tx=0x7fec4c00a620 comp rx=0 tx=0).stop 2024-09-19T09:24:46.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.773+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1400947996 shutdown_connections 2024-09-19T09:24:46.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.773+0000 7fec5a7c1640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fec3c0a3240 0x7fec3c0a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.773+0000 7fec5a7c1640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec54103be0 0x7fec541014a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.773+0000 7fec5a7c1640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec54103210 0x7fec54100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.773+0000 7fec5a7c1640 1 --2- 172.21.15.142:0/1400947996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec54100350 0x7fec54102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:46.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.773+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1400947996 >> 172.21.15.142:0/1400947996 conn(0x7fec540fba40 msgr2=0x7fec54106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:46.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.773+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1400947996 shutdown_connections 2024-09-19T09:24:46.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-19T09:24:46.773+0000 7fec5a7c1640 1 -- 172.21.15.142:0/1400947996 wait complete. 2024-09-19T09:24:47.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:47 smithi142 ceph-mon[35032]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:47.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:47.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:47.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:47.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:47.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:47.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:24:47.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:24:47.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:47 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:47.484 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:47 smithi016 ceph-mon[30979]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:47.484 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:47.484 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:47.484 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:47.485 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:47.485 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:47.485 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:24:47.485 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:24:47.485 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:47 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:47.495 DEBUG:teuthology.orchestra.run.smithi016:mds.a> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T09:24:47.498 DEBUG:teuthology.orchestra.run.smithi016:mds.d> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T09:24:47.542 DEBUG:teuthology.orchestra.run.smithi016:mds.g> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T09:24:47.586 DEBUG:teuthology.orchestra.run.smithi016:mds.j> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T09:24:47.629 DEBUG:teuthology.orchestra.run.smithi097:mds.b> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T09:24:47.632 DEBUG:teuthology.orchestra.run.smithi097:mds.e> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T09:24:47.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:47 smithi097 ceph-mon[34864]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:47.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:47.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:24:47.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:47.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:24:47.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:47.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:24:47.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:24:47.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:47 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:47.637 DEBUG:teuthology.orchestra.run.smithi097:mds.h> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T09:24:47.680 DEBUG:teuthology.orchestra.run.smithi097:mds.k> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T09:24:47.723 DEBUG:teuthology.orchestra.run.smithi142:mds.c> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T09:24:47.726 DEBUG:teuthology.orchestra.run.smithi142:mds.f> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T09:24:47.769 DEBUG:teuthology.orchestra.run.smithi142:mds.i> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T09:24:47.812 DEBUG:teuthology.orchestra.run.smithi142:mds.l> sudo journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T09:24:47.855 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T09:24:47.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:24:48.114 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:48.341 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:48 smithi142 ceph-mon[35032]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:48.341 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:48 smithi142 ceph-mon[35032]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-09-19T09:24:48.341 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:48 smithi142 ceph-mon[35032]: Deploying daemon mds.l on smithi142 2024-09-19T09:24:48.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:48 smithi016 ceph-mon[30979]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:48.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:48 smithi016 ceph-mon[30979]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-09-19T09:24:48.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:48 smithi016 ceph-mon[30979]: Deploying daemon mds.l on smithi142 2024-09-19T09:24:48.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.581+0000 7f223c898640 1 -- 172.21.15.16:0/1715485446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2238102790 msgr2=0x7f22381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:48.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.581+0000 7f223c898640 1 --2- 172.21.15.16:0/1715485446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2238102790 0x7f22381067e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f222c009900 tx=0x7f222c02a3f0 comp rx=0 tx=0).stop 2024-09-19T09:24:48.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.581+0000 7f223c898640 1 -- 172.21.15.16:0/1715485446 shutdown_connections 2024-09-19T09:24:48.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.581+0000 7f223c898640 1 --2- 172.21.15.16:0/1715485446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2238106f10 0x7f223810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.581+0000 7f223c898640 1 --2- 172.21.15.16:0/1715485446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2238102790 0x7f22381067e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.581+0000 7f223c898640 1 --2- 172.21.15.16:0/1715485446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2238101de0 0x7f22381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.581+0000 7f223c898640 1 -- 172.21.15.16:0/1715485446 >> 172.21.15.16:0/1715485446 conn(0x7f22380fd8c0 msgr2=0x7f22380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.581+0000 7f223c898640 1 -- 172.21.15.16:0/1715485446 shutdown_connections 2024-09-19T09:24:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.581+0000 7f223c898640 1 -- 172.21.15.16:0/1715485446 wait complete. 2024-09-19T09:24:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.582+0000 7f223c898640 1 Processor -- start 2024-09-19T09:24:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.582+0000 7f223c898640 1 -- start start 2024-09-19T09:24:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f223c898640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2238101de0 0x7f2238198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f223c898640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2238102790 0x7f2238198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f223c898640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2238106f10 0x7f223819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f223c898640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2238111f30 con 0x7f2238102790 2024-09-19T09:24:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f223c898640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2238111db0 con 0x7f2238106f10 2024-09-19T09:24:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f223c898640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f22381120b0 con 0x7f2238101de0 2024-09-19T09:24:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f2235d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2238102790 0x7f2238198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f223659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2238101de0 0x7f2238198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f2235d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2238102790 0x7f2238198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40894/0 (socket says 172.21.15.16:40894) 2024-09-19T09:24:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f223659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2238101de0 0x7f2238198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:60856/0 (socket says 172.21.15.16:60856) 2024-09-19T09:24:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f2235d9b640 1 -- 172.21.15.16:0/718370499 learned_addr learned my addr 172.21.15.16:0/718370499 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:24:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f2236d9d640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2238106f10 0x7f223819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f2235d9b640 1 -- 172.21.15.16:0/718370499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2238101de0 msgr2=0x7f2238198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f2235d9b640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2238101de0 0x7f2238198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f2235d9b640 1 -- 172.21.15.16:0/718370499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2238106f10 msgr2=0x7f223819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f2235d9b640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2238106f10 0x7f223819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f2235d9b640 1 -- 172.21.15.16:0/718370499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223819d2c0 con 0x7f2238102790 2024-09-19T09:24:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f2236d9d640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2238106f10 0x7f223819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:24:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.583+0000 7f2235d9b640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2238102790 0x7f2238198850 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f222c005ab0 tx=0x7f222c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.584+0000 7f22277fe640 1 -- 172.21.15.16:0/718370499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c03f030 con 0x7f2238102790 2024-09-19T09:24:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.584+0000 7f22277fe640 1 -- 172.21.15.16:0/718370499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222c003ce0 con 0x7f2238102790 2024-09-19T09:24:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.584+0000 7f223c898640 1 -- 172.21.15.16:0/718370499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223819d550 con 0x7f2238102790 2024-09-19T09:24:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.584+0000 7f22277fe640 1 -- 172.21.15.16:0/718370499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c035070 con 0x7f2238102790 2024-09-19T09:24:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.584+0000 7f223c898640 1 -- 172.21.15.16:0/718370499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22381a4e90 con 0x7f2238102790 2024-09-19T09:24:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.585+0000 7f223c898640 1 -- 172.21.15.16:0/718370499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22381021c0 con 0x7f2238102790 2024-09-19T09:24:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.586+0000 7f22277fe640 1 -- 172.21.15.16:0/718370499 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f222c042030 con 0x7f2238102790 2024-09-19T09:24:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.587+0000 7f22277fe640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f22200a3240 0x7f22200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.587+0000 7f223659c640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f22200a3240 0x7f22200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.587+0000 7f22277fe640 1 -- 172.21.15.16:0/718370499 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f222c0f0030 con 0x7f2238102790 2024-09-19T09:24:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.587+0000 7f223659c640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f22200a3240 0x7f22200a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f222800abe0 tx=0x7f222800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.590+0000 7f22277fe640 1 -- 172.21.15.16:0/718370499 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f222c0bd1f0 con 0x7f2238102790 2024-09-19T09:24:48.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:48 smithi097 ceph-mon[34864]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:24:48.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:48 smithi097 ceph-mon[34864]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-09-19T09:24:48.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:48 smithi097 ceph-mon[34864]: Deploying daemon mds.l on smithi142 2024-09-19T09:24:48.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.801+0000 7f223c898640 1 -- 172.21.15.16:0/718370499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f223810a0f0 con 0x7f2238102790 2024-09-19T09:24:48.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.802+0000 7f22277fe640 1 -- 172.21.15.16:0/718370499 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f222c0bcaf0 con 0x7f2238102790 2024-09-19T09:24:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:24:48.803 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":1,"btime":"2024-09-19T09:17:48:242055+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T09:24:48.803 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 1 2024-09-19T09:24:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.804+0000 7f223c898640 1 -- 172.21.15.16:0/718370499 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f22200a3240 msgr2=0x7f22200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.804+0000 7f223c898640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f22200a3240 0x7f22200a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f222800abe0 tx=0x7f222800a2d0 comp rx=0 tx=0).stop 2024-09-19T09:24:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.804+0000 7f223c898640 1 -- 172.21.15.16:0/718370499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2238102790 msgr2=0x7f2238198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.804+0000 7f223c898640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2238102790 0x7f2238198850 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f222c005ab0 tx=0x7f222c0042f0 comp rx=0 tx=0).stop 2024-09-19T09:24:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.804+0000 7f223c898640 1 -- 172.21.15.16:0/718370499 shutdown_connections 2024-09-19T09:24:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.804+0000 7f223c898640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2238106f10 0x7f223819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.804+0000 7f223c898640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2238102790 0x7f2238198850 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.804+0000 7f223c898640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f22200a3240 0x7f22200a5700 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.804+0000 7f223c898640 1 --2- 172.21.15.16:0/718370499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2238101de0 0x7f2238198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.804+0000 7f223c898640 1 -- 172.21.15.16:0/718370499 >> 172.21.15.16:0/718370499 conn(0x7f22380fd8c0 msgr2=0x7f22380febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.804+0000 7f223c898640 1 -- 172.21.15.16:0/718370499 shutdown_connections 2024-09-19T09:24:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:48.804+0000 7f223c898640 1 -- 172.21.15.16:0/718370499 wait complete. 2024-09-19T09:24:49.230 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:24:49 smithi142 systemd[1]: Starting Ceph mds.l for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:24:49.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:49 smithi016 ceph-mon[30979]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:49.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:49 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/718370499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:49.520 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:49 smithi142 ceph-mon[35032]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:49.520 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:49 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/718370499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:49.520 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:24:49 smithi142 podman[64392]: 2024-09-19 09:24:49.284456363 +0000 UTC m=+0.106804296 container create bda487c244015e5f98a8a0eed424f1ae9f756f9f3fdf5b9b10ad342ab74d6ac9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T09:24:49.521 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:24:49 smithi142 podman[64392]: 2024-09-19 09:24:49.19370005 +0000 UTC m=+0.016047984 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:24:49.521 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:24:49 smithi142 podman[64392]: 2024-09-19 09:24:49.520165801 +0000 UTC m=+0.342513732 container init bda487c244015e5f98a8a0eed424f1ae9f756f9f3fdf5b9b10ad342ab74d6ac9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T09:24:49.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:49 smithi097 ceph-mon[34864]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:49.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:49 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/718370499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:49.878 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:24:49 smithi142 podman[64392]: 2024-09-19 09:24:49.523101658 +0000 UTC m=+0.345449591 container start bda487c244015e5f98a8a0eed424f1ae9f756f9f3fdf5b9b10ad342ab74d6ac9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:24:49.879 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:24:49 smithi142 ceph-mds[64411]: -- 172.21.15.142:0/2379647105 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56503c55b860 con 0x56503c539000 2024-09-19T09:24:49.879 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:24:49 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-l[64407]: starting mds.l at 2024-09-19T09:24:49.879 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:24:49 smithi142 bash[64392]: bda487c244015e5f98a8a0eed424f1ae9f756f9f3fdf5b9b10ad342ab74d6ac9 2024-09-19T09:24:49.879 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:24:49 smithi142 systemd[1]: Started Ceph mds.l for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:24:50.761 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:50 smithi097 ceph-mon[34864]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:50.761 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:50 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] up:boot 2024-09-19T09:24:50.762 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:50 smithi097 ceph-mon[34864]: fsmap 1 up:standby 2024-09-19T09:24:50.762 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:24:50.762 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:50.762 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:50.762 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:50.762 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:24:50.762 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:24:50.762 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:50 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:50.762 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:50 smithi097 ceph-mon[34864]: Deploying daemon mds.k on smithi097 2024-09-19T09:24:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:50 smithi142 ceph-mon[35032]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:50 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] up:boot 2024-09-19T09:24:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:50 smithi142 ceph-mon[35032]: fsmap 1 up:standby 2024-09-19T09:24:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:24:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:24:50.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:24:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:50 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:50.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:50 smithi142 ceph-mon[35032]: Deploying daemon mds.k on smithi097 2024-09-19T09:24:51.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:50 smithi016 ceph-mon[30979]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:51.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:50 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] up:boot 2024-09-19T09:24:51.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:50 smithi016 ceph-mon[30979]: fsmap 1 up:standby 2024-09-19T09:24:51.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:24:51.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:51.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:24:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:24:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:50 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:51.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:50 smithi016 ceph-mon[30979]: Deploying daemon mds.k on smithi097 2024-09-19T09:24:51.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:24:51.747 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:51.852 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:24:51 smithi097 systemd[1]: Starting Ceph mds.k for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:24:52.119 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:24:51 smithi097 podman[63855]: 2024-09-19 09:24:51.753434821 +0000 UTC m=+0.015688499 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:24:52.119 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:24:51 smithi097 podman[63855]: 2024-09-19 09:24:51.873158432 +0000 UTC m=+0.135412108 container create baea3117dccf740f217320a3c96dc82b12648c011903e48f1982988dfeecc670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T09:24:52.119 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:24:52 smithi097 podman[63855]: 2024-09-19 09:24:52.091802046 +0000 UTC m=+0.354055727 container init baea3117dccf740f217320a3c96dc82b12648c011903e48f1982988dfeecc670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-k, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T09:24:52.119 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:24:52 smithi097 podman[63855]: 2024-09-19 09:24:52.094694782 +0000 UTC m=+0.356948467 container start baea3117dccf740f217320a3c96dc82b12648c011903e48f1982988dfeecc670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True) 2024-09-19T09:24:52.119 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:24:52 smithi097 ceph-mds[63874]: -- 172.21.15.97:0/1716642287 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56173c169860 con 0x56173c147000 2024-09-19T09:24:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.204+0000 7f49cbc48640 1 -- 172.21.15.16:0/256554119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c4102d40 msgr2=0x7f49c4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.204+0000 7f49cbc48640 1 --2- 172.21.15.16:0/256554119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c4102d40 0x7f49c4103120 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009900 tx=0x7f49c002a480 comp rx=0 tx=0).stop 2024-09-19T09:24:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.204+0000 7f49cbc48640 1 -- 172.21.15.16:0/256554119 shutdown_connections 2024-09-19T09:24:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.204+0000 7f49cbc48640 1 --2- 172.21.15.16:0/256554119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4103660 0x7f49c410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.204+0000 7f49cbc48640 1 --2- 172.21.15.16:0/256554119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c410c170 0x7f49c410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.204+0000 7f49cbc48640 1 --2- 172.21.15.16:0/256554119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c4102d40 0x7f49c4103120 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.204+0000 7f49cbc48640 1 -- 172.21.15.16:0/256554119 >> 172.21.15.16:0/256554119 conn(0x7f49c40fc7a0 msgr2=0x7f49c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.204+0000 7f49cbc48640 1 -- 172.21.15.16:0/256554119 shutdown_connections 2024-09-19T09:24:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.204+0000 7f49cbc48640 1 -- 172.21.15.16:0/256554119 wait complete. 2024-09-19T09:24:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49cbc48640 1 Processor -- start 2024-09-19T09:24:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49cbc48640 1 -- start start 2024-09-19T09:24:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49cbc48640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c4102d40 0x7f49c4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49cbc48640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4103660 0x7f49c4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49cbc48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c410c170 0x7f49c419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49cbc48640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49c4111f70 con 0x7f49c410c170 2024-09-19T09:24:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49cbc48640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f49c4111df0 con 0x7f49c4102d40 2024-09-19T09:24:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49cbc48640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41120f0 con 0x7f49c4103660 2024-09-19T09:24:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49ca1e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c410c170 0x7f49c419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49ca1e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c410c170 0x7f49c419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40906/0 (socket says 172.21.15.16:40906) 2024-09-19T09:24:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49ca1e5640 1 -- 172.21.15.16:0/411038421 learned_addr learned my addr 172.21.15.16:0/411038421 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:24:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49c99e4640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c4102d40 0x7f49c4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49c91e3640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4103660 0x7f49c4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49ca1e5640 1 -- 172.21.15.16:0/411038421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4103660 msgr2=0x7f49c4198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49ca1e5640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4103660 0x7f49c4198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49ca1e5640 1 -- 172.21.15.16:0/411038421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c4102d40 msgr2=0x7f49c4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49ca1e5640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c4102d40 0x7f49c4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49ca1e5640 1 -- 172.21.15.16:0/411038421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c419d3b0 con 0x7f49c410c170 2024-09-19T09:24:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.205+0000 7f49c91e3640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4103660 0x7f49c4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:24:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.206+0000 7f49ca1e5640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c410c170 0x7f49c419ccd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00eaa0 tx=0x7f49bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.206+0000 7f49baffd640 1 -- 172.21.15.16:0/411038421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc00cdb0 con 0x7f49c410c170 2024-09-19T09:24:52.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.206+0000 7f49baffd640 1 -- 172.21.15.16:0/411038421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49bc005020 con 0x7f49c410c170 2024-09-19T09:24:52.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.206+0000 7f49cbc48640 1 -- 172.21.15.16:0/411038421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c419d6a0 con 0x7f49c410c170 2024-09-19T09:24:52.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.206+0000 7f49baffd640 1 -- 172.21.15.16:0/411038421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0116b0 con 0x7f49c410c170 2024-09-19T09:24:52.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.206+0000 7f49cbc48640 1 -- 172.21.15.16:0/411038421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c41a4f80 con 0x7f49c410c170 2024-09-19T09:24:52.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.207+0000 7f49cbc48640 1 -- 172.21.15.16:0/411038421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c4103120 con 0x7f49c410c170 2024-09-19T09:24:52.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.208+0000 7f49baffd640 1 -- 172.21.15.16:0/411038421 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f49bc005560 con 0x7f49c410c170 2024-09-19T09:24:52.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.209+0000 7f49baffd640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f49b00a3240 0x7f49b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:52.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.209+0000 7f49c99e4640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f49b00a3240 0x7f49b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:52.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.209+0000 7f49baffd640 1 -- 172.21.15.16:0/411038421 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f49bc0cfa30 con 0x7f49c410c170 2024-09-19T09:24:52.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.209+0000 7f49c99e4640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f49b00a3240 0x7f49b00a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009230 tx=0x7f49c00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:52.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.212+0000 7f49baffd640 1 -- 172.21.15.16:0/411038421 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f49bc016030 con 0x7f49c410c170 2024-09-19T09:24:52.386 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:24:52 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-k[63870]: starting mds.k at 2024-09-19T09:24:52.386 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:24:52 smithi097 bash[63855]: baea3117dccf740f217320a3c96dc82b12648c011903e48f1982988dfeecc670 2024-09-19T09:24:52.386 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:24:52 smithi097 systemd[1]: Started Ceph mds.k for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:24:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.443+0000 7f49cbc48640 1 -- 172.21.15.16:0/411038421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49c4199760 con 0x7f49c410c170 2024-09-19T09:24:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.443+0000 7f49baffd640 1 -- 172.21.15.16:0/411038421 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f49bc09cbf0 con 0x7f49c410c170 2024-09-19T09:24:52.445 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:24:52.445 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"btime":"2024-09-19T09:24:49:550864+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T09:24:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-09-19T09:24:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.446+0000 7f49cbc48640 1 -- 172.21.15.16:0/411038421 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f49b00a3240 msgr2=0x7f49b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.446+0000 7f49cbc48640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f49b00a3240 0x7f49b00a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009230 tx=0x7f49c00058d0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.446+0000 7f49cbc48640 1 -- 172.21.15.16:0/411038421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c410c170 msgr2=0x7f49c419ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.446+0000 7f49cbc48640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c410c170 0x7f49c419ccd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00eaa0 tx=0x7f49bc00ef60 comp rx=0 tx=0).stop 2024-09-19T09:24:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.446+0000 7f49cbc48640 1 -- 172.21.15.16:0/411038421 shutdown_connections 2024-09-19T09:24:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.446+0000 7f49cbc48640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f49b00a3240 0x7f49b00a5700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.446+0000 7f49cbc48640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c4103660 0x7f49c4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.446+0000 7f49cbc48640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c410c170 0x7f49c419ccd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.446+0000 7f49cbc48640 1 --2- 172.21.15.16:0/411038421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c4102d40 0x7f49c4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.446+0000 7f49cbc48640 1 -- 172.21.15.16:0/411038421 >> 172.21.15.16:0/411038421 conn(0x7f49c40fc7a0 msgr2=0x7f49c40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.446+0000 7f49cbc48640 1 -- 172.21.15.16:0/411038421 shutdown_connections 2024-09-19T09:24:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:52.446+0000 7f49cbc48640 1 -- 172.21.15.16:0/411038421 wait complete. 2024-09-19T09:24:53.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:53 smithi016 ceph-mon[30979]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:53.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:53.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:53.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:53.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:24:53.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:24:53.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:53.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:53 smithi016 ceph-mon[30979]: Deploying daemon mds.j on smithi016 2024-09-19T09:24:53.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:53 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/411038421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:53.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:53 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] up:boot 2024-09-19T09:24:53.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:53 smithi016 ceph-mon[30979]: fsmap 2 up:standby 2024-09-19T09:24:53.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:53 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:24:53.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:53 smithi097 ceph-mon[34864]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:53.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:53 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:53.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:53 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:53.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:53 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:53.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:53 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:24:53.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:53 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:24:53.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:53 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:53.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:53 smithi097 ceph-mon[34864]: Deploying daemon mds.j on smithi016 2024-09-19T09:24:53.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:53 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/411038421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:53.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:53 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] up:boot 2024-09-19T09:24:53.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:53 smithi097 ceph-mon[34864]: fsmap 2 up:standby 2024-09-19T09:24:53.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:53 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:24:53.582 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:53 smithi142 ceph-mon[35032]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:53.582 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:53 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:53.582 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:53 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:53.582 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:53 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:53.583 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:53 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:24:53.583 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:53 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:24:53.583 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:53 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:53.583 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:53 smithi142 ceph-mon[35032]: Deploying daemon mds.j on smithi016 2024-09-19T09:24:53.583 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:53 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/411038421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:53.583 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:53 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] up:boot 2024-09-19T09:24:53.583 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:53 smithi142 ceph-mon[35032]: fsmap 2 up:standby 2024-09-19T09:24:53.583 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:53 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:24:54.951 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:24:54 smithi016 systemd[1]: Starting Ceph mds.j for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:24:55.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:24:55.202 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:24:54 smithi016 podman[68153]: 2024-09-19 09:24:54.853162727 +0000 UTC m=+0.016228369 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:24:55.202 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:24:54 smithi016 podman[68153]: 2024-09-19 09:24:54.962081701 +0000 UTC m=+0.125147334 container create f9a7443f2ede2962d5683a7fb03ac444daedf9b3a6d2e52f7fe302cb0409fe69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-j, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-19T09:24:55.202 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:24:55 smithi016 podman[68153]: 2024-09-19 09:24:55.198532691 +0000 UTC m=+0.361598337 container init f9a7443f2ede2962d5683a7fb03ac444daedf9b3a6d2e52f7fe302cb0409fe69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T09:24:55.203 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:24:55 smithi016 podman[68153]: 2024-09-19 09:24:55.201663438 +0000 UTC m=+0.364729067 container start f9a7443f2ede2962d5683a7fb03ac444daedf9b3a6d2e52f7fe302cb0409fe69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-j, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T09:24:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:55.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:55 smithi016 ceph-mon[30979]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:55.503 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:24:55 smithi016 ceph-mds[68188]: -- 172.21.15.16:0/86306747 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55953fad1860 con 0x55953faaf400 2024-09-19T09:24:55.503 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:24:55 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-j[68168]: starting mds.j at 2024-09-19T09:24:55.503 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:24:55 smithi016 bash[68153]: f9a7443f2ede2962d5683a7fb03ac444daedf9b3a6d2e52f7fe302cb0409fe69 2024-09-19T09:24:55.503 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:24:55 smithi016 systemd[1]: Started Ceph mds.j for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:24:55.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:55 smithi142 ceph-mon[35032]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:55.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:55 smithi097 ceph-mon[34864]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:55.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.893+0000 7f402e408640 1 -- 172.21.15.16:0/2778806734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40281002f0 msgr2=0x7f40281006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:55.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.893+0000 7f402e408640 1 --2- 172.21.15.16:0/2778806734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40281002f0 0x7f40281006d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4018009930 tx=0x7f401802a420 comp rx=0 tx=0).stop 2024-09-19T09:24:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.893+0000 7f402e408640 1 -- 172.21.15.16:0/2778806734 shutdown_connections 2024-09-19T09:24:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.893+0000 7f402e408640 1 --2- 172.21.15.16:0/2778806734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4028106f50 0x7f402810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.893+0000 7f402e408640 1 --2- 172.21.15.16:0/2778806734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4028100ca0 0x7f4028106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.893+0000 7f402e408640 1 --2- 172.21.15.16:0/2778806734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40281002f0 0x7f40281006d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.893+0000 7f402e408640 1 -- 172.21.15.16:0/2778806734 >> 172.21.15.16:0/2778806734 conn(0x7f4028077210 msgr2=0x7f40280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.893+0000 7f402e408640 1 -- 172.21.15.16:0/2778806734 shutdown_connections 2024-09-19T09:24:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.893+0000 7f402e408640 1 -- 172.21.15.16:0/2778806734 wait complete. 2024-09-19T09:24:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.894+0000 7f402e408640 1 Processor -- start 2024-09-19T09:24:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.894+0000 7f402e408640 1 -- start start 2024-09-19T09:24:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f402e408640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40281002f0 0x7f402810df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f402e408640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4028100ca0 0x7f402810e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f402e408640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4028106f50 0x7f40281a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f402e408640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4028111fa0 con 0x7f4028100ca0 2024-09-19T09:24:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f402e408640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4028111e20 con 0x7f4028106f50 2024-09-19T09:24:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f402e408640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4028112120 con 0x7f40281002f0 2024-09-19T09:24:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f40277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4028100ca0 0x7f402810e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f40277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4028100ca0 0x7f402810e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52980/0 (socket says 172.21.15.16:52980) 2024-09-19T09:24:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f40277fe640 1 -- 172.21.15.16:0/2207969814 learned_addr learned my addr 172.21.15.16:0/2207969814 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:24:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f4027fff640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40281002f0 0x7f402810df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f402c9a5640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4028106f50 0x7f40281a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f40277fe640 1 -- 172.21.15.16:0/2207969814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40281002f0 msgr2=0x7f402810df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f40277fe640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40281002f0 0x7f402810df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f40277fe640 1 -- 172.21.15.16:0/2207969814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4028106f50 msgr2=0x7f40281a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f40277fe640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4028106f50 0x7f40281a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f40277fe640 1 -- 172.21.15.16:0/2207969814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40281a3b80 con 0x7f4028100ca0 2024-09-19T09:24:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.895+0000 7f40277fe640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4028100ca0 0x7f402810e450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f401c00d770 tx=0x7f401c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.896+0000 7f40257fa640 1 -- 172.21.15.16:0/2207969814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c017070 con 0x7f4028100ca0 2024-09-19T09:24:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.896+0000 7f40257fa640 1 -- 172.21.15.16:0/2207969814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f401c009c30 con 0x7f4028100ca0 2024-09-19T09:24:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.896+0000 7f402e408640 1 -- 172.21.15.16:0/2207969814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40281a3e70 con 0x7f4028100ca0 2024-09-19T09:24:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.896+0000 7f40257fa640 1 -- 172.21.15.16:0/2207969814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c00f620 con 0x7f4028100ca0 2024-09-19T09:24:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.896+0000 7f402e408640 1 -- 172.21.15.16:0/2207969814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40281ab700 con 0x7f4028100ca0 2024-09-19T09:24:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.897+0000 7f402e408640 1 -- 172.21.15.16:0/2207969814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40281033a0 con 0x7f4028100ca0 2024-09-19T09:24:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.898+0000 7f40257fa640 1 -- 172.21.15.16:0/2207969814 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f401c01a070 con 0x7f4028100ca0 2024-09-19T09:24:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.898+0000 7f40257fa640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f40100a3240 0x7f40100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.899+0000 7f4027fff640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f40100a3240 0x7f40100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.899+0000 7f40257fa640 1 -- 172.21.15.16:0/2207969814 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f401c0cef80 con 0x7f4028100ca0 2024-09-19T09:24:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.899+0000 7f4027fff640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f40100a3240 0x7f40100a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f402810efa0 tx=0x7f40180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:55.902+0000 7f40257fa640 1 -- 172.21.15.16:0/2207969814 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f401c009de0 con 0x7f4028100ca0 2024-09-19T09:24:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.118+0000 7f402e408640 1 -- 172.21.15.16:0/2207969814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f402819feb0 con 0x7f4028100ca0 2024-09-19T09:24:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.118+0000 7f40257fa640 1 -- 172.21.15.16:0/2207969814 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2260 (secure 0 0 0) 0x7f401c09ba50 con 0x7f4028100ca0 2024-09-19T09:24:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:24:56.120 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"btime":"2024-09-19T09:24:52:555147+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T09:24:56.120 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2024-09-19T09:24:56.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.121+0000 7f402e408640 1 -- 172.21.15.16:0/2207969814 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f40100a3240 msgr2=0x7f40100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:56.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.121+0000 7f402e408640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f40100a3240 0x7f40100a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f402810efa0 tx=0x7f40180058d0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.121+0000 7f402e408640 1 -- 172.21.15.16:0/2207969814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4028100ca0 msgr2=0x7f402810e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:56.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.121+0000 7f402e408640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4028100ca0 0x7f402810e450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f401c00d770 tx=0x7f401c00dc30 comp rx=0 tx=0).stop 2024-09-19T09:24:56.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.121+0000 7f402e408640 1 -- 172.21.15.16:0/2207969814 shutdown_connections 2024-09-19T09:24:56.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.121+0000 7f402e408640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4028106f50 0x7f40281a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.121+0000 7f402e408640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4028100ca0 0x7f402810e450 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.121+0000 7f402e408640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f40100a3240 0x7f40100a5700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.121+0000 7f402e408640 1 --2- 172.21.15.16:0/2207969814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40281002f0 0x7f402810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.121+0000 7f402e408640 1 -- 172.21.15.16:0/2207969814 >> 172.21.15.16:0/2207969814 conn(0x7f4028077210 msgr2=0x7f40281024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.121+0000 7f402e408640 1 -- 172.21.15.16:0/2207969814 shutdown_connections 2024-09-19T09:24:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:56.121+0000 7f402e408640 1 -- 172.21.15.16:0/2207969814 wait complete. 2024-09-19T09:24:56.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:56 smithi016 ceph-mon[30979]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:56.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:56.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:56.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:56.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:24:56.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:24:56.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:56.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:56 smithi016 ceph-mon[30979]: Deploying daemon mds.d on smithi016 2024-09-19T09:24:56.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:56 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2207969814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:56.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:56 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.16:6834/3514257756,v1:172.21.15.16:6835/3514257756] up:boot 2024-09-19T09:24:56.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:56 smithi016 ceph-mon[30979]: fsmap 3 up:standby 2024-09-19T09:24:56.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:56 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T09:24:56.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:56 smithi142 ceph-mon[35032]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:56.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:56.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:56.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:56.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:24:56.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:24:56.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:56.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:56 smithi142 ceph-mon[35032]: Deploying daemon mds.d on smithi016 2024-09-19T09:24:56.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:56 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2207969814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:56.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:56 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.16:6834/3514257756,v1:172.21.15.16:6835/3514257756] up:boot 2024-09-19T09:24:56.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:56 smithi142 ceph-mon[35032]: fsmap 3 up:standby 2024-09-19T09:24:56.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:56 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T09:24:56.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:56 smithi097 ceph-mon[34864]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:56.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:56.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:56.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:56.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:24:56.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:24:56.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:56.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:56 smithi097 ceph-mon[34864]: Deploying daemon mds.d on smithi016 2024-09-19T09:24:56.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:56 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2207969814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:24:56.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:56 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.16:6834/3514257756,v1:172.21.15.16:6835/3514257756] up:boot 2024-09-19T09:24:56.638 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:56 smithi097 ceph-mon[34864]: fsmap 3 up:standby 2024-09-19T09:24:56.638 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:56 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T09:24:58.482 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:58 smithi016 ceph-mon[30979]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:58.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:58 smithi142 ceph-mon[35032]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:58.635 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:58 smithi097 ceph-mon[34864]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:24:58.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:24:58.932 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 19 09:24:58 smithi016 systemd[1]: Started Ceph mds.d for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:24:59.169 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:24:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.610+0000 7f257a7be640 1 -- 172.21.15.16:0/1650017780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2574102d40 msgr2=0x7f2574103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.610+0000 7f257a7be640 1 --2- 172.21.15.16:0/1650017780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2574102d40 0x7f2574103120 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2564009900 tx=0x7f256402a480 comp rx=0 tx=0).stop 2024-09-19T09:24:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.610+0000 7f257a7be640 1 -- 172.21.15.16:0/1650017780 shutdown_connections 2024-09-19T09:24:59.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.610+0000 7f257a7be640 1 --2- 172.21.15.16:0/1650017780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2574103660 0x7f257410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.610+0000 7f257a7be640 1 --2- 172.21.15.16:0/1650017780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f257410c170 0x7f257410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.610+0000 7f257a7be640 1 --2- 172.21.15.16:0/1650017780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2574102d40 0x7f2574103120 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.610+0000 7f257a7be640 1 -- 172.21.15.16:0/1650017780 >> 172.21.15.16:0/1650017780 conn(0x7f25740fc7a0 msgr2=0x7f25740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.610+0000 7f257a7be640 1 -- 172.21.15.16:0/1650017780 shutdown_connections 2024-09-19T09:24:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.610+0000 7f257a7be640 1 -- 172.21.15.16:0/1650017780 wait complete. 2024-09-19T09:24:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.611+0000 7f257a7be640 1 Processor -- start 2024-09-19T09:24:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.611+0000 7f257a7be640 1 -- start start 2024-09-19T09:24:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.611+0000 7f257a7be640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2574102d40 0x7f2574198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.611+0000 7f257a7be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2574103660 0x7f2574198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.611+0000 7f257a7be640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f257410c170 0x7f257419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.611+0000 7f257a7be640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2574111fb0 con 0x7f2574103660 2024-09-19T09:24:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.611+0000 7f257a7be640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2574111e30 con 0x7f2574102d40 2024-09-19T09:24:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.611+0000 7f257a7be640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2574112130 con 0x7f257410c170 2024-09-19T09:24:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.611+0000 7f256bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2574103660 0x7f2574198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.611+0000 7f256bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2574103660 0x7f2574198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53018/0 (socket says 172.21.15.16:53018) 2024-09-19T09:24:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.611+0000 7f256bfff640 1 -- 172.21.15.16:0/3663518946 learned_addr learned my addr 172.21.15.16:0/3663518946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:24:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.611+0000 7f2573fff640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2574102d40 0x7f2574198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:59.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.612+0000 7f2578d5b640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f257410c170 0x7f257419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:59.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.612+0000 7f256bfff640 1 -- 172.21.15.16:0/3663518946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f257410c170 msgr2=0x7f257419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:59.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.612+0000 7f256bfff640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f257410c170 0x7f257419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:59.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.612+0000 7f256bfff640 1 -- 172.21.15.16:0/3663518946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2574102d40 msgr2=0x7f2574198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:59.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.612+0000 7f256bfff640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2574102d40 0x7f2574198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:59.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.612+0000 7f256bfff640 1 -- 172.21.15.16:0/3663518946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257419d520 con 0x7f2574103660 2024-09-19T09:24:59.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.612+0000 7f256bfff640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2574103660 0x7f2574198ab0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f255c00b3f0 tx=0x7f255c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:59.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.612+0000 7f2571ffb640 1 -- 172.21.15.16:0/3663518946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c015020 con 0x7f2574103660 2024-09-19T09:24:59.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.612+0000 7f2571ffb640 1 -- 172.21.15.16:0/3663518946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f255c011d10 con 0x7f2574103660 2024-09-19T09:24:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.612+0000 7f257a7be640 1 -- 172.21.15.16:0/3663518946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257419d810 con 0x7f2574103660 2024-09-19T09:24:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.612+0000 7f2571ffb640 1 -- 172.21.15.16:0/3663518946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c00f2e0 con 0x7f2574103660 2024-09-19T09:24:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.612+0000 7f257a7be640 1 -- 172.21.15.16:0/3663518946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25741a50f0 con 0x7f2574103660 2024-09-19T09:24:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.613+0000 7f257a7be640 1 -- 172.21.15.16:0/3663518946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2574103120 con 0x7f2574103660 2024-09-19T09:24:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.614+0000 7f2571ffb640 1 -- 172.21.15.16:0/3663518946 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f255c018020 con 0x7f2574103660 2024-09-19T09:24:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.615+0000 7f2571ffb640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f25600a32c0 0x7f25600a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:24:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.615+0000 7f2573fff640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f25600a32c0 0x7f25600a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:24:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.615+0000 7f2571ffb640 1 -- 172.21.15.16:0/3663518946 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f255c0cf230 con 0x7f2574103660 2024-09-19T09:24:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.616+0000 7f2573fff640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f25600a32c0 0x7f25600a5780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2564000c00 tx=0x7f25640058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:24:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.618+0000 7f2571ffb640 1 -- 172.21.15.16:0/3663518946 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f255c01f020 con 0x7f2574103660 2024-09-19T09:24:59.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.833+0000 7f257a7be640 1 -- 172.21.15.16:0/3663518946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25741998d0 con 0x7f2574103660 2024-09-19T09:24:59.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:59 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:59.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:59 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:59.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:59 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:24:59.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:59 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:24:59.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:59 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:24:59.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:59 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:24:59.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:59 smithi016 ceph-mon[30979]: Deploying daemon mds.e on smithi097 2024-09-19T09:24:59.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:59 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.16:6836/2828730634,v1:172.21.15.16:6837/2828730634] up:boot 2024-09-19T09:24:59.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:59 smithi016 ceph-mon[30979]: fsmap 4 up:standby 2024-09-19T09:24:59.836 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:24:59 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:24:59.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.834+0000 7f2571ffb640 1 -- 172.21.15.16:0/3663518946 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7f25741998d0 con 0x7f2574103660 2024-09-19T09:24:59.837 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:24:59.837 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"btime":"2024-09-19T09:24:59:369933+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T09:24:59.838 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 5 2024-09-19T09:24:59.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.836+0000 7f257a7be640 1 -- 172.21.15.16:0/3663518946 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f25600a32c0 msgr2=0x7f25600a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:59.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.836+0000 7f257a7be640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f25600a32c0 0x7f25600a5780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2564000c00 tx=0x7f25640058d0 comp rx=0 tx=0).stop 2024-09-19T09:24:59.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.836+0000 7f257a7be640 1 -- 172.21.15.16:0/3663518946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2574103660 msgr2=0x7f2574198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:24:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.836+0000 7f257a7be640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2574103660 0x7f2574198ab0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f255c00b3f0 tx=0x7f255c00b8b0 comp rx=0 tx=0).stop 2024-09-19T09:24:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.836+0000 7f257a7be640 1 -- 172.21.15.16:0/3663518946 shutdown_connections 2024-09-19T09:24:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.836+0000 7f257a7be640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f25600a32c0 0x7f25600a5780 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.836+0000 7f257a7be640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f257410c170 0x7f257419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.836+0000 7f257a7be640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2574103660 0x7f2574198ab0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.836+0000 7f257a7be640 1 --2- 172.21.15.16:0/3663518946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2574102d40 0x7f2574198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:24:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.836+0000 7f257a7be640 1 -- 172.21.15.16:0/3663518946 >> 172.21.15.16:0/3663518946 conn(0x7f25740fc7a0 msgr2=0x7f25740fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:24:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.836+0000 7f257a7be640 1 -- 172.21.15.16:0/3663518946 shutdown_connections 2024-09-19T09:24:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:24:59.836+0000 7f257a7be640 1 -- 172.21.15.16:0/3663518946 wait complete. 2024-09-19T09:25:00.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:59 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:00.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:59 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:00.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:59 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:00.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:59 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:00.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:59 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:00.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:59 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:00.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:59 smithi142 ceph-mon[35032]: Deploying daemon mds.e on smithi097 2024-09-19T09:25:00.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:59 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.16:6836/2828730634,v1:172.21.15.16:6837/2828730634] up:boot 2024-09-19T09:25:00.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:59 smithi142 ceph-mon[35032]: fsmap 4 up:standby 2024-09-19T09:25:00.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:24:59 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:25:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:59 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:59 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:59 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:59 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:59 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:59 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:59 smithi097 ceph-mon[34864]: Deploying daemon mds.e on smithi097 2024-09-19T09:25:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:59 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.16:6836/2828730634,v1:172.21.15.16:6837/2828730634] up:boot 2024-09-19T09:25:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:59 smithi097 ceph-mon[34864]: fsmap 4 up:standby 2024-09-19T09:25:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:24:59 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T09:25:00.789 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:25:00 smithi097 systemd[1]: Starting Ceph mds.e for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:25:01.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:00 smithi016 ceph-mon[30979]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:01.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:00 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3663518946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:01.048 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:00 smithi097 ceph-mon[34864]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:01.049 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:00 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3663518946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:01.049 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:25:00 smithi097 podman[64170]: 2024-09-19 09:25:00.799938311 +0000 UTC m=+0.017332369 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:25:01.052 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:25:00 smithi097 podman[64170]: 2024-09-19 09:25:00.917587562 +0000 UTC m=+0.134981654 container create d0bae33a229bff2a68d7421a5409592b9aa65b5b5934367888f0dbfa41a04f98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T09:25:01.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:00 smithi142 ceph-mon[35032]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:01.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:00 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3663518946' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:01.636 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:25:01 smithi097 podman[64170]: 2024-09-19 09:25:01.136586866 +0000 UTC m=+0.353980918 container init d0bae33a229bff2a68d7421a5409592b9aa65b5b5934367888f0dbfa41a04f98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-e, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:25:01.636 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:25:01 smithi097 podman[64170]: 2024-09-19 09:25:01.139349252 +0000 UTC m=+0.356743299 container start d0bae33a229bff2a68d7421a5409592b9aa65b5b5934367888f0dbfa41a04f98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T09:25:01.636 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:25:01 smithi097 ceph-mds[64189]: -- 172.21.15.97:0/515572358 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c3e8895860 con 0x55c3e8872800 2024-09-19T09:25:01.636 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:25:01 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-e[64185]: starting mds.e at 2024-09-19T09:25:01.636 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:25:01 smithi097 bash[64170]: d0bae33a229bff2a68d7421a5409592b9aa65b5b5934367888f0dbfa41a04f98 2024-09-19T09:25:01.636 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:25:01 smithi097 systemd[1]: Started Ceph mds.e for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:25:02.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:25:02.609 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:02.609 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:02.609 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:02.609 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:02.609 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:02.610 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:02.610 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:02 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.97:6834/4135704970,v1:172.21.15.97:6835/4135704970] up:boot 2024-09-19T09:25:02.610 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:02 smithi142 ceph-mon[35032]: fsmap 5 up:standby 2024-09-19T09:25:02.610 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:02 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:25:02.623 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:02.623 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:02.623 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:02.623 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:02.623 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:02.623 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:02.624 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:02 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.97:6834/4135704970,v1:172.21.15.97:6835/4135704970] up:boot 2024-09-19T09:25:02.624 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:02 smithi016 ceph-mon[30979]: fsmap 5 up:standby 2024-09-19T09:25:02.624 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:02 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:25:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:02 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.97:6834/4135704970,v1:172.21.15.97:6835/4135704970] up:boot 2024-09-19T09:25:02.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:02 smithi097 ceph-mon[34864]: fsmap 5 up:standby 2024-09-19T09:25:02.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:02 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:25:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:03.277 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:25:03 smithi142 systemd[1]: Starting Ceph mds.f for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:25:03.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.503+0000 7efd1d4c7640 1 -- 172.21.15.16:0/3521627655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18103660 msgr2=0x7efd1810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:03.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.503+0000 7efd1d4c7640 1 --2- 172.21.15.16:0/3521627655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18103660 0x7efd1810bb50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009900 tx=0x7efd0c02a480 comp rx=0 tx=0).stop 2024-09-19T09:25:03.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.503+0000 7efd1d4c7640 1 -- 172.21.15.16:0/3521627655 shutdown_connections 2024-09-19T09:25:03.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.503+0000 7efd1d4c7640 1 --2- 172.21.15.16:0/3521627655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd1810c170 0x7efd1810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:03.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.503+0000 7efd1d4c7640 1 --2- 172.21.15.16:0/3521627655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18103660 0x7efd1810bb50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:03.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.503+0000 7efd1d4c7640 1 --2- 172.21.15.16:0/3521627655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd18102d40 0x7efd18103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.503+0000 7efd1d4c7640 1 -- 172.21.15.16:0/3521627655 >> 172.21.15.16:0/3521627655 conn(0x7efd180fc7a0 msgr2=0x7efd180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.503+0000 7efd1d4c7640 1 -- 172.21.15.16:0/3521627655 shutdown_connections 2024-09-19T09:25:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.503+0000 7efd1d4c7640 1 -- 172.21.15.16:0/3521627655 wait complete. 2024-09-19T09:25:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.504+0000 7efd1d4c7640 1 Processor -- start 2024-09-19T09:25:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.504+0000 7efd1d4c7640 1 -- start start 2024-09-19T09:25:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.504+0000 7efd1d4c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18102d40 0x7efd181009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.504+0000 7efd1d4c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd18103660 0x7efd18100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.504+0000 7efd1d4c7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1810c170 0x7efd18101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.504+0000 7efd1d4c7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd18111f70 con 0x7efd18102d40 2024-09-19T09:25:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.504+0000 7efd1d4c7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd18111df0 con 0x7efd1810c170 2024-09-19T09:25:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.504+0000 7efd1d4c7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efd181120f0 con 0x7efd18103660 2024-09-19T09:25:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18102d40 0x7efd181009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:03.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18102d40 0x7efd181009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53024/0 (socket says 172.21.15.16:53024) 2024-09-19T09:25:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd16ffd640 1 -- 172.21.15.16:0/2901596357 learned_addr learned my addr 172.21.15.16:0/2901596357 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd167fc640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd18103660 0x7efd18100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd177fe640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1810c170 0x7efd18101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd16ffd640 1 -- 172.21.15.16:0/2901596357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd18103660 msgr2=0x7efd18100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd16ffd640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd18103660 0x7efd18100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd16ffd640 1 -- 172.21.15.16:0/2901596357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1810c170 msgr2=0x7efd18101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd16ffd640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1810c170 0x7efd18101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd16ffd640 1 -- 172.21.15.16:0/2901596357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd181a6950 con 0x7efd18102d40 2024-09-19T09:25:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd177fe640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1810c170 0x7efd18101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:03.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd16ffd640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18102d40 0x7efd181009a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd0800c890 tx=0x7efd0800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:03.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efcfffff640 1 -- 172.21.15.16:0/2901596357 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08004220 con 0x7efd18102d40 2024-09-19T09:25:03.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd1d4c7640 1 -- 172.21.15.16:0/2901596357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd181a6c40 con 0x7efd18102d40 2024-09-19T09:25:03.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efcfffff640 1 -- 172.21.15.16:0/2901596357 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd080043c0 con 0x7efd18102d40 2024-09-19T09:25:03.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efcfffff640 1 -- 172.21.15.16:0/2901596357 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08004a00 con 0x7efd18102d40 2024-09-19T09:25:03.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.505+0000 7efd1d4c7640 1 -- 172.21.15.16:0/2901596357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd181a71a0 con 0x7efd18102d40 2024-09-19T09:25:03.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.506+0000 7efd1d4c7640 1 -- 172.21.15.16:0/2901596357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd18103120 con 0x7efd18102d40 2024-09-19T09:25:03.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.508+0000 7efcfffff640 1 -- 172.21.15.16:0/2901596357 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7efd080079b0 con 0x7efd18102d40 2024-09-19T09:25:03.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.508+0000 7efcfffff640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7efd000a3240 0x7efd000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:03.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.508+0000 7efd167fc640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7efd000a3240 0x7efd000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:03.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.509+0000 7efcfffff640 1 -- 172.21.15.16:0/2901596357 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7efd080cf540 con 0x7efd18102d40 2024-09-19T09:25:03.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.509+0000 7efd167fc640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7efd000a3240 0x7efd000a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd0c000c00 tx=0x7efd0c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:03.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.512+0000 7efcfffff640 1 -- 172.21.15.16:0/2901596357 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efd08014030 con 0x7efd18102d40 2024-09-19T09:25:03.531 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:03 smithi142 ceph-mon[35032]: Deploying daemon mds.f on smithi142 2024-09-19T09:25:03.531 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:03 smithi142 ceph-mon[35032]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:03.531 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:25:03 smithi142 podman[64691]: 2024-09-19 09:25:03.178275795 +0000 UTC m=+0.015980385 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:25:03.531 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:25:03 smithi142 podman[64691]: 2024-09-19 09:25:03.277125428 +0000 UTC m=+0.114830009 container create 0317d9ea27f71e73ed21cbc1a93872cda640b35af7b09beb67ef01bc28253f8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T09:25:03.531 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:25:03 smithi142 podman[64691]: 2024-09-19 09:25:03.504540499 +0000 UTC m=+0.342245080 container init 0317d9ea27f71e73ed21cbc1a93872cda640b35af7b09beb67ef01bc28253f8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T09:25:03.531 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:25:03 smithi142 podman[64691]: 2024-09-19 09:25:03.507316091 +0000 UTC m=+0.345020675 container start 0317d9ea27f71e73ed21cbc1a93872cda640b35af7b09beb67ef01bc28253f8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T09:25:03.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:03 smithi097 ceph-mon[34864]: Deploying daemon mds.f on smithi142 2024-09-19T09:25:03.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:03 smithi097 ceph-mon[34864]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:03.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.725+0000 7efd1d4c7640 1 -- 172.21.15.16:0/2901596357 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd18101ed0 con 0x7efd18102d40 2024-09-19T09:25:03.726 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:03 smithi016 ceph-mon[30979]: Deploying daemon mds.f on smithi142 2024-09-19T09:25:03.726 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:03 smithi016 ceph-mon[30979]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.725+0000 7efcfffff640 1 -- 172.21.15.16:0/2901596357 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7efd0809c700 con 0x7efd18102d40 2024-09-19T09:25:03.727 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:03.727 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"btime":"2024-09-19T09:25:01:788301+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T09:25:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-09-19T09:25:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.728+0000 7efd1d4c7640 1 -- 172.21.15.16:0/2901596357 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7efd000a3240 msgr2=0x7efd000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.728+0000 7efd1d4c7640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7efd000a3240 0x7efd000a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd0c000c00 tx=0x7efd0c0058d0 comp rx=0 tx=0).stop 2024-09-19T09:25:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.728+0000 7efd1d4c7640 1 -- 172.21.15.16:0/2901596357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18102d40 msgr2=0x7efd181009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.728+0000 7efd1d4c7640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18102d40 0x7efd181009a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd0800c890 tx=0x7efd0800cd50 comp rx=0 tx=0).stop 2024-09-19T09:25:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.728+0000 7efd1d4c7640 1 -- 172.21.15.16:0/2901596357 shutdown_connections 2024-09-19T09:25:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.728+0000 7efd1d4c7640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7efd000a3240 0x7efd000a5700 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.728+0000 7efd1d4c7640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd18103660 0x7efd18100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.728+0000 7efd1d4c7640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1810c170 0x7efd18101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.728+0000 7efd1d4c7640 1 --2- 172.21.15.16:0/2901596357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd18102d40 0x7efd181009a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.728+0000 7efd1d4c7640 1 -- 172.21.15.16:0/2901596357 >> 172.21.15.16:0/2901596357 conn(0x7efd180fc7a0 msgr2=0x7efd180fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:03.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.728+0000 7efd1d4c7640 1 -- 172.21.15.16:0/2901596357 shutdown_connections 2024-09-19T09:25:03.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:03.729+0000 7efd1d4c7640 1 -- 172.21.15.16:0/2901596357 wait complete. 2024-09-19T09:25:03.878 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:25:03 smithi142 ceph-mds[64710]: -- 172.21.15.142:0/3751607628 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562231209860 con 0x5622311e6800 2024-09-19T09:25:03.878 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:25:03 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-f[64706]: starting mds.f at 2024-09-19T09:25:03.879 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:25:03 smithi142 bash[64691]: 0317d9ea27f71e73ed21cbc1a93872cda640b35af7b09beb67ef01bc28253f8b 2024-09-19T09:25:03.879 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:25:03 smithi142 systemd[1]: Started Ceph mds.f for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:25:04.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:04 smithi016 ceph-mon[30979]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:04.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:04 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:04.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:04 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:04.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:04 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:04.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:04 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:04.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:04 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:04.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:04 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:04.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:04 smithi016 ceph-mon[30979]: Deploying daemon mds.c on smithi142 2024-09-19T09:25:04.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:04 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2901596357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:04.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:04 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.142:6834/2657443138,v1:172.21.15.142:6835/2657443138] up:boot 2024-09-19T09:25:04.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:04 smithi016 ceph-mon[30979]: fsmap 6 up:standby 2024-09-19T09:25:04.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:04 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:25:04.891 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:04 smithi097 ceph-mon[34864]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:04.891 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:04 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:04.891 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:04 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:04.891 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:04 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:04.891 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:04 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:04.891 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:04 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:04.891 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:04 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:04.891 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:04 smithi097 ceph-mon[34864]: Deploying daemon mds.c on smithi142 2024-09-19T09:25:04.891 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:04 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2901596357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:04.891 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:04 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.142:6834/2657443138,v1:172.21.15.142:6835/2657443138] up:boot 2024-09-19T09:25:04.892 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:04 smithi097 ceph-mon[34864]: fsmap 6 up:standby 2024-09-19T09:25:04.892 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:04 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:25:05.098 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:04 smithi142 ceph-mon[35032]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:05.098 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:04 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:05.098 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:04 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:05.098 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:04 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:05.099 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:04 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:05.099 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:04 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:05.099 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:04 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:05.099 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:04 smithi142 ceph-mon[35032]: Deploying daemon mds.c on smithi142 2024-09-19T09:25:05.099 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:04 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2901596357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:05.099 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:04 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.142:6834/2657443138,v1:172.21.15.142:6835/2657443138] up:boot 2024-09-19T09:25:05.099 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:04 smithi142 ceph-mon[35032]: fsmap 6 up:standby 2024-09-19T09:25:05.099 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:04 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:25:05.628 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:25:05 smithi142 systemd[1]: Starting Ceph mds.c for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:25:06.007 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:25:05 smithi142 podman[64995]: 2024-09-19 09:25:05.659010215 +0000 UTC m=+0.015797200 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:25:06.007 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:25:05 smithi142 podman[64995]: 2024-09-19 09:25:05.771036693 +0000 UTC m=+0.127823708 container create e18c3349f4b65171044f3ffcfd197b6469ab5bd7c698c228effdc94290679c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-c, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T09:25:06.007 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:25:06 smithi142 podman[64995]: 2024-09-19 09:25:06.006855849 +0000 UTC m=+0.363642821 container init e18c3349f4b65171044f3ffcfd197b6469ab5bd7c698c228effdc94290679c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T09:25:06.379 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:25:06 smithi142 podman[64995]: 2024-09-19 09:25:06.009655073 +0000 UTC m=+0.366442043 container start e18c3349f4b65171044f3ffcfd197b6469ab5bd7c698c228effdc94290679c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True) 2024-09-19T09:25:06.379 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:25:06 smithi142 ceph-mds[65014]: -- 172.21.15.142:0/289386496 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56399f451860 con 0x56399f42f400 2024-09-19T09:25:06.379 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:25:06 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-c[65010]: starting mds.c at 2024-09-19T09:25:06.379 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:25:06 smithi142 bash[64995]: e18c3349f4b65171044f3ffcfd197b6469ab5bd7c698c228effdc94290679c44 2024-09-19T09:25:06.379 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:25:06 smithi142 systemd[1]: Started Ceph mds.c for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:25:06.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:25:06.780 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:07.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:07 smithi016 ceph-mon[30979]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:07 smithi016 ceph-mon[30979]: Deploying daemon mds.h on smithi097 2024-09-19T09:25:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:07 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.142:6836/81091251,v1:172.21.15.142:6837/81091251] up:boot 2024-09-19T09:25:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:07 smithi016 ceph-mon[30979]: fsmap 7 up:standby 2024-09-19T09:25:07.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:07 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:25:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.243+0000 7f37ef59e640 1 -- 172.21.15.16:0/1799911231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f0102b90 msgr2=0x7f37f0102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.243+0000 7f37ef59e640 1 --2- 172.21.15.16:0/1799911231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f0102b90 0x7f37f0102f70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009ca0 tx=0x7f37e002a950 comp rx=0 tx=0).stop 2024-09-19T09:25:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.244+0000 7f37ef59e640 1 -- 172.21.15.16:0/1799911231 shutdown_connections 2024-09-19T09:25:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.244+0000 7f37ef59e640 1 --2- 172.21.15.16:0/1799911231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37f010bfc0 0x7f37f010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.244+0000 7f37ef59e640 1 --2- 172.21.15.16:0/1799911231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37f01034b0 0x7f37f010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.244+0000 7f37ef59e640 1 --2- 172.21.15.16:0/1799911231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f0102b90 0x7f37f0102f70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.244+0000 7f37ef59e640 1 -- 172.21.15.16:0/1799911231 >> 172.21.15.16:0/1799911231 conn(0x7f37f00fc610 msgr2=0x7f37f00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.244+0000 7f37ef59e640 1 -- 172.21.15.16:0/1799911231 shutdown_connections 2024-09-19T09:25:07.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.244+0000 7f37ef59e640 1 -- 172.21.15.16:0/1799911231 wait complete. 2024-09-19T09:25:07.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.244+0000 7f37ef59e640 1 Processor -- start 2024-09-19T09:25:07.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.244+0000 7f37ef59e640 1 -- start start 2024-09-19T09:25:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.245+0000 7f37ef59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37f0102b90 0x7f37f01007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.245+0000 7f37ef59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f01034b0 0x7f37f0100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.245+0000 7f37edd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f01034b0 0x7f37f0100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.245+0000 7f37edd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f01034b0 0x7f37f0100d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40686/0 (socket says 172.21.15.16:40686) 2024-09-19T09:25:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.245+0000 7f37ef59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37f010bfc0 0x7f37f0101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.245+0000 7f37edd9b640 1 -- 172.21.15.16:0/238377640 learned_addr learned my addr 172.21.15.16:0/238377640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.245+0000 7f37ef59e640 1 -- 172.21.15.16:0/238377640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0111dc0 con 0x7f37f01034b0 2024-09-19T09:25:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.245+0000 7f37ef59e640 1 -- 172.21.15.16:0/238377640 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0111c40 con 0x7f37f010bfc0 2024-09-19T09:25:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.245+0000 7f37ef59e640 1 -- 172.21.15.16:0/238377640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0111f40 con 0x7f37f0102b90 2024-09-19T09:25:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.245+0000 7f37ee59c640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37f0102b90 0x7f37f01007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.245+0000 7f37eed9d640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37f010bfc0 0x7f37f0101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:07.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.245+0000 7f37edd9b640 1 -- 172.21.15.16:0/238377640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37f0102b90 msgr2=0x7f37f01007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.246+0000 7f37edd9b640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37f0102b90 0x7f37f01007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.246+0000 7f37edd9b640 1 -- 172.21.15.16:0/238377640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37f010bfc0 msgr2=0x7f37f0101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.246+0000 7f37edd9b640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37f010bfc0 0x7f37f0101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.246+0000 7f37edd9b640 1 -- 172.21.15.16:0/238377640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f019c450 con 0x7f37f01034b0 2024-09-19T09:25:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.246+0000 7f37ee59c640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37f0102b90 0x7f37f01007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.246+0000 7f37edd9b640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f01034b0 0x7f37f0100d30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f37e401d8d0 tx=0x7f37e401dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.246+0000 7f37df7fe640 1 -- 172.21.15.16:0/238377640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4025070 con 0x7f37f01034b0 2024-09-19T09:25:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.246+0000 7f37ef59e640 1 -- 172.21.15.16:0/238377640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f019c740 con 0x7f37f01034b0 2024-09-19T09:25:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.246+0000 7f37df7fe640 1 -- 172.21.15.16:0/238377640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e4004440 con 0x7f37f01034b0 2024-09-19T09:25:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.246+0000 7f37df7fe640 1 -- 172.21.15.16:0/238377640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4003e00 con 0x7f37f01034b0 2024-09-19T09:25:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.246+0000 7f37ef59e640 1 -- 172.21.15.16:0/238377640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f019cc80 con 0x7f37f01034b0 2024-09-19T09:25:07.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.247+0000 7f37ef59e640 1 -- 172.21.15.16:0/238377640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f006a850 con 0x7f37f01034b0 2024-09-19T09:25:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.248+0000 7f37df7fe640 1 -- 172.21.15.16:0/238377640 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f37e4028070 con 0x7f37f01034b0 2024-09-19T09:25:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.249+0000 7f37df7fe640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f37d80a3240 0x7f37d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.249+0000 7f37df7fe640 1 -- 172.21.15.16:0/238377640 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f37e40dff80 con 0x7f37f01034b0 2024-09-19T09:25:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.249+0000 7f37ee59c640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f37d80a3240 0x7f37d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:07.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.250+0000 7f37ee59c640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f37d80a3240 0x7f37d80a5700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f37e0000f80 tx=0x7f37e002fcf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:07.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.253+0000 7f37df7fe640 1 -- 172.21.15.16:0/238377640 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f37e40acaf0 con 0x7f37f01034b0 2024-09-19T09:25:07.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:07 smithi097 ceph-mon[34864]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:07.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:07.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:07.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:07.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:07.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:07.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:07.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:07 smithi097 ceph-mon[34864]: Deploying daemon mds.h on smithi097 2024-09-19T09:25:07.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:07 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.142:6836/81091251,v1:172.21.15.142:6837/81091251] up:boot 2024-09-19T09:25:07.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:07 smithi097 ceph-mon[34864]: fsmap 7 up:standby 2024-09-19T09:25:07.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:07 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:25:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.465+0000 7f37ef59e640 1 -- 172.21.15.16:0/238377640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37f0107060 con 0x7f37f01034b0 2024-09-19T09:25:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.465+0000 7f37df7fe640 1 -- 172.21.15.16:0/238377640 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6385 (secure 0 0 0) 0x7f37e40b0df0 con 0x7f37f01034b0 2024-09-19T09:25:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:07.467 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-19T09:25:06:674768+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T09:25:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-19T09:25:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.467+0000 7f37ef59e640 1 -- 172.21.15.16:0/238377640 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f37d80a3240 msgr2=0x7f37d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.467+0000 7f37ef59e640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f37d80a3240 0x7f37d80a5700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f37e0000f80 tx=0x7f37e002fcf0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.467+0000 7f37ef59e640 1 -- 172.21.15.16:0/238377640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f01034b0 msgr2=0x7f37f0100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.467+0000 7f37ef59e640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f01034b0 0x7f37f0100d30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f37e401d8d0 tx=0x7f37e401dd90 comp rx=0 tx=0).stop 2024-09-19T09:25:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.467+0000 7f37ef59e640 1 -- 172.21.15.16:0/238377640 shutdown_connections 2024-09-19T09:25:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.467+0000 7f37ef59e640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37f010bfc0 0x7f37f0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.467+0000 7f37ef59e640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f01034b0 0x7f37f0100d30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.467+0000 7f37ef59e640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f37d80a3240 0x7f37d80a5700 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.467+0000 7f37ef59e640 1 --2- 172.21.15.16:0/238377640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37f0102b90 0x7f37f01007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:07.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.467+0000 7f37ef59e640 1 -- 172.21.15.16:0/238377640 >> 172.21.15.16:0/238377640 conn(0x7f37f00fc610 msgr2=0x7f37f00fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:07.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.468+0000 7f37ef59e640 1 -- 172.21.15.16:0/238377640 shutdown_connections 2024-09-19T09:25:07.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:07.468+0000 7f37ef59e640 1 -- 172.21.15.16:0/238377640 wait complete. 2024-09-19T09:25:07.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:07 smithi142 ceph-mon[35032]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:07.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:07.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:07.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:07.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:07.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:07.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:07.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:07 smithi142 ceph-mon[35032]: Deploying daemon mds.h on smithi097 2024-09-19T09:25:07.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:07 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.142:6836/81091251,v1:172.21.15.142:6837/81091251] up:boot 2024-09-19T09:25:07.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:07 smithi142 ceph-mon[35032]: fsmap 7 up:standby 2024-09-19T09:25:07.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:07 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:25:08.136 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:25:08 smithi097 systemd[1]: Starting Ceph mds.h for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:25:08.448 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:08 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/238377640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:08.449 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:25:08 smithi097 podman[64473]: 2024-09-19 09:25:08.117489908 +0000 UTC m=+0.015649721 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:25:08.449 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:25:08 smithi097 podman[64473]: 2024-09-19 09:25:08.221061919 +0000 UTC m=+0.119221778 container create 08718aa33904b0abd3e5c1dbaf45c8888471181097dabf92e0fc95fb31770dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T09:25:08.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:08 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/238377640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:08.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:08 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/238377640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:08.886 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:25:08 smithi097 podman[64473]: 2024-09-19 09:25:08.448475745 +0000 UTC m=+0.346635568 container init 08718aa33904b0abd3e5c1dbaf45c8888471181097dabf92e0fc95fb31770dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T09:25:08.886 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:25:08 smithi097 podman[64473]: 2024-09-19 09:25:08.451178469 +0000 UTC m=+0.349338283 container start 08718aa33904b0abd3e5c1dbaf45c8888471181097dabf92e0fc95fb31770dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:25:08.886 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:25:08 smithi097 ceph-mds[64492]: -- 172.21.15.97:0/4293567712 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b35e15860 con 0x563b35df2800 2024-09-19T09:25:08.886 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:25:08 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h[64488]: starting mds.h at 2024-09-19T09:25:08.886 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:25:08 smithi097 bash[64473]: 08718aa33904b0abd3e5c1dbaf45c8888471181097dabf92e0fc95fb31770dff 2024-09-19T09:25:08.886 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:25:08 smithi097 systemd[1]: Started Ceph mds.h for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:25:09.285 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:09 smithi142 ceph-mon[35032]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:09.285 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:09 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:09.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:09 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:09.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:09 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:09.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:09 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:09.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:09 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:09.286 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:09 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:09 smithi016 ceph-mon[30979]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:09 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:09 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:09 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:09 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:09.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:09 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:09.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:09 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:09.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:09 smithi097 ceph-mon[34864]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:09.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:09 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:09.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:09 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:09.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:09 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:09.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:09 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:09.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:09 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:09.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:09 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:09.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:25:10.261 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:10.292 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:10 smithi142 ceph-mon[35032]: Deploying daemon mds.i on smithi142 2024-09-19T09:25:10.292 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:10 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] up:boot 2024-09-19T09:25:10.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:10 smithi142 ceph-mon[35032]: fsmap 8 up:standby 2024-09-19T09:25:10.293 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:10 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:25:10.382 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:10 smithi016 ceph-mon[30979]: Deploying daemon mds.i on smithi142 2024-09-19T09:25:10.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:10 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] up:boot 2024-09-19T09:25:10.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:10 smithi016 ceph-mon[30979]: fsmap 8 up:standby 2024-09-19T09:25:10.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:10 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:25:10.629 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:25:10 smithi142 systemd[1]: Starting Ceph mds.i for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:25:10.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:10 smithi097 ceph-mon[34864]: Deploying daemon mds.i on smithi142 2024-09-19T09:25:10.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:10 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] up:boot 2024-09-19T09:25:10.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:10 smithi097 ceph-mon[34864]: fsmap 8 up:standby 2024-09-19T09:25:10.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:10 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:25:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.735+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/1221979591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec8101410 msgr2=0x7f2ec810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.735+0000 7f2ecfa1d640 1 --2- 172.21.15.16:0/1221979591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec8101410 0x7f2ec810d940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009cd0 tx=0x7f2ec002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.735+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/1221979591 shutdown_connections 2024-09-19T09:25:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.735+0000 7f2ecfa1d640 1 --2- 172.21.15.16:0/1221979591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ec8100a70 0x7f2ec8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.735+0000 7f2ecfa1d640 1 --2- 172.21.15.16:0/1221979591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec8101410 0x7f2ec810d940 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.735+0000 7f2ecfa1d640 1 --2- 172.21.15.16:0/1221979591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec8106a70 0x7f2ec8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.735+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/1221979591 >> 172.21.15.16:0/1221979591 conn(0x7f2ec80fc790 msgr2=0x7f2ec80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.735+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/1221979591 shutdown_connections 2024-09-19T09:25:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.735+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/1221979591 wait complete. 2024-09-19T09:25:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.736+0000 7f2ecfa1d640 1 Processor -- start 2024-09-19T09:25:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.736+0000 7f2ecfa1d640 1 -- start start 2024-09-19T09:25:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.736+0000 7f2ecfa1d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec8100a70 0x7f2ec8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.736+0000 7f2ecfa1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec8101410 0x7f2ec8197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.736+0000 7f2ecfa1d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ec8106a70 0x7f2ec819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.736+0000 7f2ecfa1d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec8110e70 con 0x7f2ec8101410 2024-09-19T09:25:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.736+0000 7f2ecfa1d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec8110cf0 con 0x7f2ec8100a70 2024-09-19T09:25:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.736+0000 7f2ecfa1d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec8110ff0 con 0x7f2ec8106a70 2024-09-19T09:25:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.736+0000 7f2eccfb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec8101410 0x7f2ec8197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.736+0000 7f2eccfb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec8101410 0x7f2ec8197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40702/0 (socket says 172.21.15.16:40702) 2024-09-19T09:25:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.736+0000 7f2eccfb8640 1 -- 172.21.15.16:0/610041423 learned_addr learned my addr 172.21.15.16:0/610041423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.736+0000 7f2ecdfba640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ec8106a70 0x7f2ec819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.736+0000 7f2ecd7b9640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec8100a70 0x7f2ec8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.737+0000 7f2eccfb8640 1 -- 172.21.15.16:0/610041423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ec8106a70 msgr2=0x7f2ec819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.737+0000 7f2eccfb8640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ec8106a70 0x7f2ec819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.737+0000 7f2eccfb8640 1 -- 172.21.15.16:0/610041423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec8100a70 msgr2=0x7f2ec8197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.737+0000 7f2eccfb8640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec8100a70 0x7f2ec8197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.737+0000 7f2eccfb8640 1 -- 172.21.15.16:0/610041423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec819c2f0 con 0x7f2ec8101410 2024-09-19T09:25:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.737+0000 7f2eccfb8640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec8101410 0x7f2ec8197880 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800e9f0 tx=0x7f2eb800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.737+0000 7f2ebe7fc640 1 -- 172.21.15.16:0/610041423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb800cd80 con 0x7f2ec8101410 2024-09-19T09:25:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.737+0000 7f2ebe7fc640 1 -- 172.21.15.16:0/610041423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb8005500 con 0x7f2ec8101410 2024-09-19T09:25:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.737+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/610041423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec819c5e0 con 0x7f2ec8101410 2024-09-19T09:25:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.737+0000 7f2ebe7fc640 1 -- 172.21.15.16:0/610041423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8010690 con 0x7f2ec8101410 2024-09-19T09:25:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.737+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/610041423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec81a3ec0 con 0x7f2ec8101410 2024-09-19T09:25:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.738+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/610041423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec8100ed0 con 0x7f2ec8101410 2024-09-19T09:25:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.739+0000 7f2ebe7fc640 1 -- 172.21.15.16:0/610041423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2eb80046c0 con 0x7f2ec8101410 2024-09-19T09:25:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.740+0000 7f2ebe7fc640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2eb40a32c0 0x7f2eb40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.740+0000 7f2ecd7b9640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2eb40a32c0 0x7f2eb40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.740+0000 7f2ebe7fc640 1 -- 172.21.15.16:0/610041423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f2eb80cf940 con 0x7f2ec8101410 2024-09-19T09:25:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.740+0000 7f2ecd7b9640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2eb40a32c0 0x7f2eb40a5780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4005d90 tx=0x7f2ec4005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.744+0000 7f2ebe7fc640 1 -- 172.21.15.16:0/610041423 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2eb809ca60 con 0x7f2ec8101410 2024-09-19T09:25:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.954+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/610041423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ec802d7c0 con 0x7f2ec8101410 2024-09-19T09:25:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.954+0000 7f2ebe7fc640 1 -- 172.21.15.16:0/610041423 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7210 (secure 0 0 0) 0x7f2eb809c360 con 0x7f2ec8101410 2024-09-19T09:25:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:10.956 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-09-19T09:25:09:154266+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T09:25:10.956 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-19T09:25:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.956+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/610041423 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2eb40a32c0 msgr2=0x7f2eb40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.956+0000 7f2ecfa1d640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2eb40a32c0 0x7f2eb40a5780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4005d90 tx=0x7f2ec4005d20 comp rx=0 tx=0).stop 2024-09-19T09:25:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.956+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/610041423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec8101410 msgr2=0x7f2ec8197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.956+0000 7f2ecfa1d640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec8101410 0x7f2ec8197880 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800e9f0 tx=0x7f2eb800edf0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.957+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/610041423 shutdown_connections 2024-09-19T09:25:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.957+0000 7f2ecfa1d640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2eb40a32c0 0x7f2eb40a5780 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.957+0000 7f2ecfa1d640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ec8106a70 0x7f2ec819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.957+0000 7f2ecfa1d640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec8101410 0x7f2ec8197880 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.957+0000 7f2ecfa1d640 1 --2- 172.21.15.16:0/610041423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec8100a70 0x7f2ec8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.957+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/610041423 >> 172.21.15.16:0/610041423 conn(0x7f2ec80fc790 msgr2=0x7f2ec80fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.957+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/610041423 shutdown_connections 2024-09-19T09:25:10.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:10.957+0000 7f2ecfa1d640 1 -- 172.21.15.16:0/610041423 wait complete. 2024-09-19T09:25:10.969 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:25:10 smithi142 podman[65294]: 2024-09-19 09:25:10.60149076 +0000 UTC m=+0.016098972 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:25:10.969 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:25:10 smithi142 podman[65294]: 2024-09-19 09:25:10.72330586 +0000 UTC m=+0.137914063 container create bcc57a0f2b71fab51245af68cdc31302e5f4f43f898f6468637324441fd12b28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-i, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T09:25:10.969 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:25:10 smithi142 podman[65294]: 2024-09-19 09:25:10.942258549 +0000 UTC m=+0.356866751 container init bcc57a0f2b71fab51245af68cdc31302e5f4f43f898f6468637324441fd12b28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T09:25:10.969 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:25:10 smithi142 podman[65294]: 2024-09-19 09:25:10.945114677 +0000 UTC m=+0.359722886 container start bcc57a0f2b71fab51245af68cdc31302e5f4f43f898f6468637324441fd12b28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-i, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T09:25:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:11 smithi016 ceph-mon[30979]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:11 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/610041423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:11.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:11 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:11.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:11 smithi142 ceph-mon[35032]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:11 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/610041423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:11.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:11 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:11.379 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:25:10 smithi142 ceph-mds[65313]: -- 172.21.15.142:0/4140608254 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb41061c20 con 0x55cb4103f400 2024-09-19T09:25:11.379 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:25:10 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-i[65309]: starting mds.i at 2024-09-19T09:25:11.380 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:25:11 smithi142 bash[65294]: bcc57a0f2b71fab51245af68cdc31302e5f4f43f898f6468637324441fd12b28 2024-09-19T09:25:11.380 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:25:11 smithi142 systemd[1]: Started Ceph mds.i for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:25:11.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:11 smithi097 ceph-mon[34864]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:11.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:11 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/610041423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:11.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:11.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:11.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:11.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:11.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:11.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:11 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:12.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:12 smithi016 ceph-mon[30979]: Deploying daemon mds.g on smithi016 2024-09-19T09:25:12.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:12 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] up:boot 2024-09-19T09:25:12.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:12 smithi016 ceph-mon[30979]: fsmap 9 up:standby 2024-09-19T09:25:12.430 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:12 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:25:12.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:12 smithi142 ceph-mon[35032]: Deploying daemon mds.g on smithi016 2024-09-19T09:25:12.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:12 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] up:boot 2024-09-19T09:25:12.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:12 smithi142 ceph-mon[35032]: fsmap 9 up:standby 2024-09-19T09:25:12.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:12 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:25:12.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:12 smithi097 ceph-mon[34864]: Deploying daemon mds.g on smithi016 2024-09-19T09:25:12.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:12 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] up:boot 2024-09-19T09:25:12.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:12 smithi097 ceph-mon[34864]: fsmap 9 up:standby 2024-09-19T09:25:12.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:12 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:25:13.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:13 smithi016 ceph-mon[30979]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:13.254 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:25:13 smithi016 systemd[1]: Starting Ceph mds.g for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:25:13.509 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:25:13 smithi016 podman[69336]: 2024-09-19 09:25:13.299784462 +0000 UTC m=+0.016227444 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:25:13.509 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:25:13 smithi016 podman[69336]: 2024-09-19 09:25:13.399296192 +0000 UTC m=+0.115739195 container create 2f96fd20c7fcddb008e1209143faccb8e2da30ab51ef6a3b47dd3fbd11fdc9d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T09:25:13.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:25:13.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:13 smithi142 ceph-mon[35032]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:13.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:13 smithi097 ceph-mon[34864]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:13.867 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:13.990 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:25:13 smithi016 podman[69336]: 2024-09-19 09:25:13.677212167 +0000 UTC m=+0.393655139 container init 2f96fd20c7fcddb008e1209143faccb8e2da30ab51ef6a3b47dd3fbd11fdc9d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T09:25:13.990 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:25:13 smithi016 podman[69336]: 2024-09-19 09:25:13.680946103 +0000 UTC m=+0.397389065 container start 2f96fd20c7fcddb008e1209143faccb8e2da30ab51ef6a3b47dd3fbd11fdc9d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-g, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:25:13.990 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:25:13 smithi016 ceph-mds[69373]: -- 172.21.15.16:0/561373775 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5632a7d8d860 con 0x5632a7d6b400 2024-09-19T09:25:13.990 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:25:13 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-g[69351]: starting mds.g at 2024-09-19T09:25:13.990 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:25:13 smithi016 bash[69336]: 2f96fd20c7fcddb008e1209143faccb8e2da30ab51ef6a3b47dd3fbd11fdc9d8 2024-09-19T09:25:13.990 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:25:13 smithi016 systemd[1]: Started Ceph mds.g for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:25:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.321+0000 7f13266c1640 1 -- 172.21.15.16:0/3596119583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1320102da0 msgr2=0x7f1320103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.321+0000 7f13266c1640 1 --2- 172.21.15.16:0/3596119583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1320102da0 0x7f1320103180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1310009900 tx=0x7f131002a480 comp rx=0 tx=0).stop 2024-09-19T09:25:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.321+0000 7f13266c1640 1 -- 172.21.15.16:0/3596119583 shutdown_connections 2024-09-19T09:25:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.321+0000 7f13266c1640 1 --2- 172.21.15.16:0/3596119583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f132010c180 0x7f132010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.321+0000 7f13266c1640 1 --2- 172.21.15.16:0/3596119583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13201036c0 0x7f132010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.321+0000 7f13266c1640 1 --2- 172.21.15.16:0/3596119583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1320102da0 0x7f1320103180 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.321+0000 7f13266c1640 1 -- 172.21.15.16:0/3596119583 >> 172.21.15.16:0/3596119583 conn(0x7f13200fc7a0 msgr2=0x7f13200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.321+0000 7f13266c1640 1 -- 172.21.15.16:0/3596119583 shutdown_connections 2024-09-19T09:25:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.321+0000 7f13266c1640 1 -- 172.21.15.16:0/3596119583 wait complete. 2024-09-19T09:25:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.321+0000 7f13266c1640 1 Processor -- start 2024-09-19T09:25:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.321+0000 7f13266c1640 1 -- start start 2024-09-19T09:25:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f13266c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1320102da0 0x7f1320198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f13266c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13201036c0 0x7f1320198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f13266c1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f132010c180 0x7f132019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f13266c1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1320111f60 con 0x7f13201036c0 2024-09-19T09:25:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f13266c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1320111de0 con 0x7f1320102da0 2024-09-19T09:25:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f13266c1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f13201120e0 con 0x7f132010c180 2024-09-19T09:25:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13201036c0 0x7f1320198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13201036c0 0x7f1320198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40742/0 (socket says 172.21.15.16:40742) 2024-09-19T09:25:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f131f7fe640 1 -- 172.21.15.16:0/3502441899 learned_addr learned my addr 172.21.15.16:0/3502441899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f131ffff640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1320102da0 0x7f1320198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f1324c5e640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f132010c180 0x7f132019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f131f7fe640 1 -- 172.21.15.16:0/3502441899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f132010c180 msgr2=0x7f132019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f131f7fe640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f132010c180 0x7f132019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f131f7fe640 1 -- 172.21.15.16:0/3502441899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1320102da0 msgr2=0x7f1320198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f131f7fe640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1320102da0 0x7f1320198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f131f7fe640 1 -- 172.21.15.16:0/3502441899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132019d3b0 con 0x7f13201036c0 2024-09-19T09:25:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.322+0000 7f131f7fe640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13201036c0 0x7f1320198940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f131400d950 tx=0x7f131400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.323+0000 7f131d7fa640 1 -- 172.21.15.16:0/3502441899 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314017070 con 0x7f13201036c0 2024-09-19T09:25:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.323+0000 7f131d7fa640 1 -- 172.21.15.16:0/3502441899 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1314009c30 con 0x7f13201036c0 2024-09-19T09:25:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.323+0000 7f131d7fa640 1 -- 172.21.15.16:0/3502441899 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131400f6b0 con 0x7f13201036c0 2024-09-19T09:25:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.323+0000 7f13266c1640 1 -- 172.21.15.16:0/3502441899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132019d6a0 con 0x7f13201036c0 2024-09-19T09:25:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.323+0000 7f13266c1640 1 -- 172.21.15.16:0/3502441899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13201a4f80 con 0x7f13201036c0 2024-09-19T09:25:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.324+0000 7f13266c1640 1 -- 172.21.15.16:0/3502441899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1320103180 con 0x7f13201036c0 2024-09-19T09:25:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.325+0000 7f131d7fa640 1 -- 172.21.15.16:0/3502441899 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f13140109d0 con 0x7f13201036c0 2024-09-19T09:25:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.326+0000 7f131d7fa640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f13080a32c0 0x7f13080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.326+0000 7f131ffff640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f13080a32c0 0x7f13080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.326+0000 7f131d7fa640 1 -- 172.21.15.16:0/3502441899 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f13140cf050 con 0x7f13201036c0 2024-09-19T09:25:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.326+0000 7f131ffff640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f13080a32c0 0x7f13080a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1310009230 tx=0x7f13100058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.330+0000 7f131d7fa640 1 -- 172.21.15.16:0/3502441899 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1314009de0 con 0x7f13201036c0 2024-09-19T09:25:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.549+0000 7f13266c1640 1 -- 172.21.15.16:0/3502441899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1320107280 con 0x7f13201036c0 2024-09-19T09:25:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.549+0000 7f131d7fa640 1 -- 172.21.15.16:0/3502441899 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8866 (secure 0 0 0) 0x7f131409bb20 con 0x7f13201036c0 2024-09-19T09:25:14.550 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:14.551 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":11,"btime":"2024-09-19T09:25:14:179174+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T09:25:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 11 2024-09-19T09:25:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.552+0000 7f13266c1640 1 -- 172.21.15.16:0/3502441899 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f13080a32c0 msgr2=0x7f13080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.552+0000 7f13266c1640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f13080a32c0 0x7f13080a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1310009230 tx=0x7f13100058d0 comp rx=0 tx=0).stop 2024-09-19T09:25:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.552+0000 7f13266c1640 1 -- 172.21.15.16:0/3502441899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13201036c0 msgr2=0x7f1320198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.552+0000 7f13266c1640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13201036c0 0x7f1320198940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f131400d950 tx=0x7f131400de10 comp rx=0 tx=0).stop 2024-09-19T09:25:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.552+0000 7f13266c1640 1 -- 172.21.15.16:0/3502441899 shutdown_connections 2024-09-19T09:25:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.552+0000 7f13266c1640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f13080a32c0 0x7f13080a5780 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.552+0000 7f13266c1640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f132010c180 0x7f132019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.552+0000 7f13266c1640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13201036c0 0x7f1320198940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.552+0000 7f13266c1640 1 --2- 172.21.15.16:0/3502441899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1320102da0 0x7f1320198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.552+0000 7f13266c1640 1 -- 172.21.15.16:0/3502441899 >> 172.21.15.16:0/3502441899 conn(0x7f13200fc7a0 msgr2=0x7f13200fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.552+0000 7f13266c1640 1 -- 172.21.15.16:0/3502441899 shutdown_connections 2024-09-19T09:25:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:14.552+0000 7f13266c1640 1 -- 172.21.15.16:0/3502441899 wait complete. 2024-09-19T09:25:15.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:14 smithi016 ceph-mon[30979]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:15.087 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:15.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:15.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:15.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:15.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:15.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:15.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:14 smithi016 ceph-mon[30979]: Deploying daemon mds.a on smithi016 2024-09-19T09:25:15.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:14 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.16:6838/3573603820,v1:172.21.15.16:6839/3573603820] up:boot 2024-09-19T09:25:15.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:14 smithi016 ceph-mon[30979]: fsmap 10 up:standby 2024-09-19T09:25:15.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:14 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:25:15.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:14 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3502441899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:15.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:14 smithi142 ceph-mon[35032]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:15.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:15.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:15.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:15.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:15.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:15.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:15.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:14 smithi142 ceph-mon[35032]: Deploying daemon mds.a on smithi016 2024-09-19T09:25:15.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:14 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.16:6838/3573603820,v1:172.21.15.16:6839/3573603820] up:boot 2024-09-19T09:25:15.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:14 smithi142 ceph-mon[35032]: fsmap 10 up:standby 2024-09-19T09:25:15.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:14 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:25:15.130 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:14 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3502441899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:15.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:14 smithi097 ceph-mon[34864]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:15.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:15.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:15.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:15.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:15.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:15.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:15.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:14 smithi097 ceph-mon[34864]: Deploying daemon mds.a on smithi016 2024-09-19T09:25:15.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:14 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.16:6838/3573603820,v1:172.21.15.16:6839/3573603820] up:boot 2024-09-19T09:25:15.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:14 smithi097 ceph-mon[34864]: fsmap 10 up:standby 2024-09-19T09:25:15.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:14 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:25:15.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:14 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3502441899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:16.503 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:25:16 smithi016 systemd[1]: Starting Ceph mds.a for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:25:16.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:16 smithi016 ceph-mon[30979]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:16.879 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:25:16 smithi016 podman[69751]: 2024-09-19 09:25:16.52048702 +0000 UTC m=+0.016399122 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:25:16.880 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:25:16 smithi016 podman[69751]: 2024-09-19 09:25:16.617794235 +0000 UTC m=+0.113706326 container create b920f858065034c9d7b04dccd4efa5a4f32d9bd46dd6495a4737a31f25f61b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-a, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T09:25:16.880 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:25:16 smithi016 podman[69751]: 2024-09-19 09:25:16.787197662 +0000 UTC m=+0.283109756 container init b920f858065034c9d7b04dccd4efa5a4f32d9bd46dd6495a4737a31f25f61b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-a, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T09:25:16.880 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:25:16 smithi016 podman[69751]: 2024-09-19 09:25:16.790067741 +0000 UTC m=+0.285979832 container start b920f858065034c9d7b04dccd4efa5a4f32d9bd46dd6495a4737a31f25f61b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T09:25:16.880 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:25:16 smithi016 ceph-mds[69770]: -- 172.21.15.16:0/2869239182 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55de6576d860 con 0x55de6574b400 2024-09-19T09:25:16.880 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:25:16 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-a[69766]: starting mds.a at 2024-09-19T09:25:16.880 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:25:16 smithi016 bash[69751]: b920f858065034c9d7b04dccd4efa5a4f32d9bd46dd6495a4737a31f25f61b63 2024-09-19T09:25:17.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:25:17.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:16 smithi142 ceph-mon[35032]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:17.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:16 smithi097 ceph-mon[34864]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:17.143 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:25:16 smithi016 systemd[1]: Started Ceph mds.a for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:25:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.806+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2530330451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c38100e20 msgr2=0x7f9c38101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.806+0000 7f9c3d5c6640 1 --2- 172.21.15.16:0/2530330451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c38100e20 0x7f9c38101200 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009900 tx=0x7f9c2802a480 comp rx=0 tx=0).stop 2024-09-19T09:25:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.806+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2530330451 shutdown_connections 2024-09-19T09:25:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.806+0000 7f9c3d5c6640 1 --2- 172.21.15.16:0/2530330451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c38102ef0 0x7f9c3810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.806+0000 7f9c3d5c6640 1 --2- 172.21.15.16:0/2530330451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c38102570 0x7f9c381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.806+0000 7f9c3d5c6640 1 --2- 172.21.15.16:0/2530330451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c38100e20 0x7f9c38101200 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.806+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2530330451 >> 172.21.15.16:0/2530330451 conn(0x7f9c380fc7d0 msgr2=0x7f9c380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.806+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2530330451 shutdown_connections 2024-09-19T09:25:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.806+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2530330451 wait complete. 2024-09-19T09:25:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c3d5c6640 1 Processor -- start 2024-09-19T09:25:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c3d5c6640 1 -- start start 2024-09-19T09:25:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c3d5c6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c38100e20 0x7f9c381984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c3d5c6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c38102570 0x7f9c381989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c3d5c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c38102ef0 0x7f9c3819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c3d5c6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c381120a0 con 0x7f9c38102ef0 2024-09-19T09:25:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c3d5c6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c38111f20 con 0x7f9c38102570 2024-09-19T09:25:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c3d5c6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9c38112220 con 0x7f9c38100e20 2024-09-19T09:25:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c38102ef0 0x7f9c3819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c38102ef0 0x7f9c3819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37424/0 (socket says 172.21.15.16:37424) 2024-09-19T09:25:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c377fe640 1 -- 172.21.15.16:0/2907003318 learned_addr learned my addr 172.21.15.16:0/2907003318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c36ffd640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c38100e20 0x7f9c381984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c2ffff640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c38102570 0x7f9c381989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c377fe640 1 -- 172.21.15.16:0/2907003318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c38100e20 msgr2=0x7f9c381984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c377fe640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c38100e20 0x7f9c381984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c377fe640 1 -- 172.21.15.16:0/2907003318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c38102570 msgr2=0x7f9c381989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c377fe640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c38102570 0x7f9c381989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c377fe640 1 -- 172.21.15.16:0/2907003318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3819d450 con 0x7f9c38102ef0 2024-09-19T09:25:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c2ffff640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c38102570 0x7f9c381989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:25:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.807+0000 7f9c377fe640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c38102ef0 0x7f9c3819cd70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000cce0 tx=0x7f9c3000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.808+0000 7f9c34ff9640 1 -- 172.21.15.16:0/2907003318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000f450 con 0x7f9c38102ef0 2024-09-19T09:25:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.808+0000 7f9c34ff9640 1 -- 172.21.15.16:0/2907003318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c3000fad0 con 0x7f9c38102ef0 2024-09-19T09:25:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.808+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2907003318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c3819d740 con 0x7f9c38102ef0 2024-09-19T09:25:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.808+0000 7f9c34ff9640 1 -- 172.21.15.16:0/2907003318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30018c10 con 0x7f9c38102ef0 2024-09-19T09:25:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.808+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2907003318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c381a5020 con 0x7f9c38102ef0 2024-09-19T09:25:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.809+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2907003318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c38101200 con 0x7f9c38102ef0 2024-09-19T09:25:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.810+0000 7f9c34ff9640 1 -- 172.21.15.16:0/2907003318 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9c3000f5f0 con 0x7f9c38102ef0 2024-09-19T09:25:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.811+0000 7f9c34ff9640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9c240a32c0 0x7f9c240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.811+0000 7f9c36ffd640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9c240a32c0 0x7f9c240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.811+0000 7f9c34ff9640 1 -- 172.21.15.16:0/2907003318 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f9c300cfc00 con 0x7f9c38102ef0 2024-09-19T09:25:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.811+0000 7f9c36ffd640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9c240a32c0 0x7f9c240a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9c28005ab0 tx=0x7f9c28005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:17.815+0000 7f9c34ff9640 1 -- 172.21.15.16:0/2907003318 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9c30014030 con 0x7f9c38102ef0 2024-09-19T09:25:18.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.034+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2907003318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c381a5220 con 0x7f9c38102ef0 2024-09-19T09:25:18.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.035+0000 7f9c34ff9640 1 -- 172.21.15.16:0/2907003318 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9692 (secure 0 0 0) 0x7f9c3009cd20 con 0x7f9c38102ef0 2024-09-19T09:25:18.036 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:18.036 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":12,"btime":"2024-09-19T09:25:17:816054+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T09:25:18.036 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 12 2024-09-19T09:25:18.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.037+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2907003318 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9c240a32c0 msgr2=0x7f9c240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:18.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.037+0000 7f9c3d5c6640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9c240a32c0 0x7f9c240a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9c28005ab0 tx=0x7f9c28005a20 comp rx=0 tx=0).stop 2024-09-19T09:25:18.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.037+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2907003318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c38102ef0 msgr2=0x7f9c3819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:18.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.037+0000 7f9c3d5c6640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c38102ef0 0x7f9c3819cd70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000cce0 tx=0x7f9c3000a620 comp rx=0 tx=0).stop 2024-09-19T09:25:18.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.037+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2907003318 shutdown_connections 2024-09-19T09:25:18.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.037+0000 7f9c3d5c6640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c38102ef0 0x7f9c3819cd70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:18.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.037+0000 7f9c3d5c6640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c38102570 0x7f9c381989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:18.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.037+0000 7f9c3d5c6640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9c240a32c0 0x7f9c240a5780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:18.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.037+0000 7f9c3d5c6640 1 --2- 172.21.15.16:0/2907003318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c38100e20 0x7f9c381984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:18.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.037+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2907003318 >> 172.21.15.16:0/2907003318 conn(0x7f9c380fc7d0 msgr2=0x7f9c380fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:18.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.037+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2907003318 shutdown_connections 2024-09-19T09:25:18.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:18.037+0000 7f9c3d5c6640 1 -- 172.21.15.16:0/2907003318 wait complete. 2024-09-19T09:25:18.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:18.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:18.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:18.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:18.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:18.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:18.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:17 smithi016 ceph-mon[30979]: Deploying daemon mds.b on smithi097 2024-09-19T09:25:18.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:17 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.16:6840/4017062522,v1:172.21.15.16:6841/4017062522] up:boot 2024-09-19T09:25:18.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:17 smithi016 ceph-mon[30979]: fsmap 11 up:standby 2024-09-19T09:25:18.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:17 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:25:18.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:18.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:18.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:18.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:18.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:18.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:18.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:17 smithi142 ceph-mon[35032]: Deploying daemon mds.b on smithi097 2024-09-19T09:25:18.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:17 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.16:6840/4017062522,v1:172.21.15.16:6841/4017062522] up:boot 2024-09-19T09:25:18.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:17 smithi142 ceph-mon[35032]: fsmap 11 up:standby 2024-09-19T09:25:18.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:17 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:25:18.381 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:18.381 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:18.381 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:18.381 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T09:25:18.382 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T09:25:18.382 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:18.383 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:17 smithi097 ceph-mon[34864]: Deploying daemon mds.b on smithi097 2024-09-19T09:25:18.383 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:17 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.16:6840/4017062522,v1:172.21.15.16:6841/4017062522] up:boot 2024-09-19T09:25:18.383 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:17 smithi097 ceph-mon[34864]: fsmap 11 up:standby 2024-09-19T09:25:18.383 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:17 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:25:18.942 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:25:18 smithi097 systemd[1]: Starting Ceph mds.b for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:25:19.205 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:18 smithi097 ceph-mon[34864]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:19.205 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:18 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2907003318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:19.206 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:25:19 smithi097 podman[64773]: 2024-09-19 09:25:19.074089212 +0000 UTC m=+0.109203842 container create 582e7ba753cc229799fca4ce6491f95b427d6adcdcb6ba6a80dd7272cb4976a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T09:25:19.206 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:25:19 smithi097 podman[64773]: 2024-09-19 09:25:18.980519898 +0000 UTC m=+0.015634544 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:25:19.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:18 smithi016 ceph-mon[30979]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:19.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:18 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2907003318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:19.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:18 smithi142 ceph-mon[35032]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:19.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:18 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2907003318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:19.636 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:25:19 smithi097 podman[64773]: 2024-09-19 09:25:19.301693891 +0000 UTC m=+0.336808519 container init 582e7ba753cc229799fca4ce6491f95b427d6adcdcb6ba6a80dd7272cb4976a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T09:25:19.636 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:25:19 smithi097 podman[64773]: 2024-09-19 09:25:19.304802897 +0000 UTC m=+0.339917538 container start 582e7ba753cc229799fca4ce6491f95b427d6adcdcb6ba6a80dd7272cb4976a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:25:19.636 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:25:19 smithi097 ceph-mds[64792]: -- 172.21.15.97:0/4266380360 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5afa19860 con 0x55f5af9f6800 2024-09-19T09:25:19.636 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:25:19 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b[64788]: starting mds.b at 2024-09-19T09:25:19.636 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:25:19 smithi097 bash[64773]: 582e7ba753cc229799fca4ce6491f95b427d6adcdcb6ba6a80dd7272cb4976a5 2024-09-19T09:25:19.636 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:25:19 smithi097 systemd[1]: Started Ceph mds.b for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:25:20.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:20 smithi016 ceph-mon[30979]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:20.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:20 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] up:boot 2024-09-19T09:25:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:20 smithi016 ceph-mon[30979]: fsmap 12 up:standby 2024-09-19T09:25:20.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:20 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:25:20.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:25:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:20.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:20 smithi142 ceph-mon[35032]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:20.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:20.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:20 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] up:boot 2024-09-19T09:25:20.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:20 smithi142 ceph-mon[35032]: fsmap 12 up:standby 2024-09-19T09:25:20.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:20 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:25:20.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:20 smithi097 ceph-mon[34864]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:20.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:20.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:20.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:20 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] up:boot 2024-09-19T09:25:20.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:20 smithi097 ceph-mon[34864]: fsmap 12 up:standby 2024-09-19T09:25:20.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:20 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:25:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.323+0000 7fc194b66640 1 -- 172.21.15.16:0/740962260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc190106f10 msgr2=0x7fc19010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.323+0000 7fc194b66640 1 --2- 172.21.15.16:0/740962260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc190106f10 0x7fc19010ea90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc188009ca0 tx=0x7fc18802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:25:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.323+0000 7fc194b66640 1 -- 172.21.15.16:0/740962260 shutdown_connections 2024-09-19T09:25:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.323+0000 7fc194b66640 1 --2- 172.21.15.16:0/740962260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc190106f10 0x7fc19010ea90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.323+0000 7fc194b66640 1 --2- 172.21.15.16:0/740962260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc190102790 0x7fc1901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.323+0000 7fc194b66640 1 --2- 172.21.15.16:0/740962260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc190101de0 0x7fc1901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.323+0000 7fc194b66640 1 -- 172.21.15.16:0/740962260 >> 172.21.15.16:0/740962260 conn(0x7fc1900fd8c0 msgr2=0x7fc1900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.323+0000 7fc194b66640 1 -- 172.21.15.16:0/740962260 shutdown_connections 2024-09-19T09:25:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.323+0000 7fc194b66640 1 -- 172.21.15.16:0/740962260 wait complete. 2024-09-19T09:25:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.324+0000 7fc194b66640 1 Processor -- start 2024-09-19T09:25:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.324+0000 7fc194b66640 1 -- start start 2024-09-19T09:25:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.324+0000 7fc194b66640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc190101de0 0x7fc1901983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.324+0000 7fc194b66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc190102790 0x7fc190198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.324+0000 7fc194b66640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc190106f10 0x7fc19019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.324+0000 7fc194b66640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc190111f70 con 0x7fc190102790 2024-09-19T09:25:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.324+0000 7fc194b66640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc190111df0 con 0x7fc190106f10 2024-09-19T09:25:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.324+0000 7fc194b66640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901120f0 con 0x7fc190101de0 2024-09-19T09:25:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc185d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc190102790 0x7fc190198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc185d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc190102790 0x7fc190198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37452/0 (socket says 172.21.15.16:37452) 2024-09-19T09:25:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc185d9b640 1 -- 172.21.15.16:0/3533913620 learned_addr learned my addr 172.21.15.16:0/3533913620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc18e59c640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc190101de0 0x7fc1901983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc18ed9d640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc190106f10 0x7fc19019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc185d9b640 1 -- 172.21.15.16:0/3533913620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc190101de0 msgr2=0x7fc1901983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc185d9b640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc190101de0 0x7fc1901983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc185d9b640 1 -- 172.21.15.16:0/3533913620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc190106f10 msgr2=0x7fc19019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc185d9b640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc190106f10 0x7fc19019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc185d9b640 1 -- 172.21.15.16:0/3533913620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19019d380 con 0x7fc190102790 2024-09-19T09:25:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc185d9b640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc190102790 0x7fc190198910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc17800d950 tx=0x7fc17800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc187fff640 1 -- 172.21.15.16:0/3533913620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178017070 con 0x7fc190102790 2024-09-19T09:25:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc194b66640 1 -- 172.21.15.16:0/3533913620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19019d670 con 0x7fc190102790 2024-09-19T09:25:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc187fff640 1 -- 172.21.15.16:0/3533913620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc178009c30 con 0x7fc190102790 2024-09-19T09:25:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc187fff640 1 -- 172.21.15.16:0/3533913620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17800f6b0 con 0x7fc190102790 2024-09-19T09:25:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.325+0000 7fc194b66640 1 -- 172.21.15.16:0/3533913620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1901a4f50 con 0x7fc190102790 2024-09-19T09:25:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.326+0000 7fc194b66640 1 -- 172.21.15.16:0/3533913620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1901021c0 con 0x7fc190102790 2024-09-19T09:25:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.328+0000 7fc187fff640 1 -- 172.21.15.16:0/3533913620 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc1780109d0 con 0x7fc190102790 2024-09-19T09:25:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.329+0000 7fc187fff640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc17c0a3240 0x7fc17c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.329+0000 7fc18e59c640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc17c0a3240 0x7fc17c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.329+0000 7fc187fff640 1 -- 172.21.15.16:0/3533913620 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fc1780cef80 con 0x7fc190102790 2024-09-19T09:25:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.329+0000 7fc18e59c640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc17c0a3240 0x7fc17c0a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc180005fd0 tx=0x7fc18000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.332+0000 7fc187fff640 1 -- 172.21.15.16:0/3533913620 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc178009de0 con 0x7fc190102790 2024-09-19T09:25:21.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.564+0000 7fc194b66640 1 -- 172.21.15.16:0/3533913620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc19010a1b0 con 0x7fc190102790 2024-09-19T09:25:21.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.565+0000 7fc187fff640 1 -- 172.21.15.16:0/3533913620 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10518 (secure 0 0 0) 0x7fc17809baf0 con 0x7fc190102790 2024-09-19T09:25:21.567 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:21.567 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":13,"btime":"2024-09-19T09:25:19:941186+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24500,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T09:25:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 13 2024-09-19T09:25:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.568+0000 7fc194b66640 1 -- 172.21.15.16:0/3533913620 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc17c0a3240 msgr2=0x7fc17c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.568+0000 7fc194b66640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc17c0a3240 0x7fc17c0a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc180005fd0 tx=0x7fc18000b390 comp rx=0 tx=0).stop 2024-09-19T09:25:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.568+0000 7fc194b66640 1 -- 172.21.15.16:0/3533913620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc190102790 msgr2=0x7fc190198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.568+0000 7fc194b66640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc190102790 0x7fc190198910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc17800d950 tx=0x7fc17800de10 comp rx=0 tx=0).stop 2024-09-19T09:25:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.568+0000 7fc194b66640 1 -- 172.21.15.16:0/3533913620 shutdown_connections 2024-09-19T09:25:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.568+0000 7fc194b66640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc190106f10 0x7fc19019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.568+0000 7fc194b66640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc190102790 0x7fc190198910 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.568+0000 7fc194b66640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc17c0a3240 0x7fc17c0a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.568+0000 7fc194b66640 1 --2- 172.21.15.16:0/3533913620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc190101de0 0x7fc1901983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.568+0000 7fc194b66640 1 -- 172.21.15.16:0/3533913620 >> 172.21.15.16:0/3533913620 conn(0x7fc1900fd8c0 msgr2=0x7fc1900feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.568+0000 7fc194b66640 1 -- 172.21.15.16:0/3533913620 shutdown_connections 2024-09-19T09:25:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:21.568+0000 7fc194b66640 1 -- 172.21.15.16:0/3533913620 wait complete. 2024-09-19T09:25:22.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:22.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:22.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:21 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3533913620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:22.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:21 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:22.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T09:25:22.239 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.240 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.240 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.240 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.240 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.240 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.240 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.240 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:22.240 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:22.240 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.240 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:21 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3533913620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:22.240 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:21 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:22.355 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.355 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.356 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.356 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.356 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.356 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.356 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.356 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:22.356 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:22.356 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:22.356 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:21 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3533913620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:22.356 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:21 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:25:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:23.126 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:22 smithi016 ceph-mon[30979]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:22 smithi142 ceph-mon[35032]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:23.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:22 smithi097 ceph-mon[34864]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:23.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.439+0000 7fca252ac640 1 -- 172.21.15.16:0/1299433096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2010c170 msgr2=0x7fca2010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:23.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.439+0000 7fca252ac640 1 --2- 172.21.15.16:0/1299433096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2010c170 0x7fca2010e610 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca18009ca0 tx=0x7fca1802a950 comp rx=0 tx=0).stop 2024-09-19T09:25:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.440+0000 7fca252ac640 1 -- 172.21.15.16:0/1299433096 shutdown_connections 2024-09-19T09:25:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.440+0000 7fca252ac640 1 --2- 172.21.15.16:0/1299433096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2010c170 0x7fca2010e610 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.440+0000 7fca252ac640 1 --2- 172.21.15.16:0/1299433096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca20103660 0x7fca2010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.440+0000 7fca252ac640 1 --2- 172.21.15.16:0/1299433096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca20102d40 0x7fca20103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.440+0000 7fca252ac640 1 -- 172.21.15.16:0/1299433096 >> 172.21.15.16:0/1299433096 conn(0x7fca200fc7a0 msgr2=0x7fca200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.440+0000 7fca252ac640 1 -- 172.21.15.16:0/1299433096 shutdown_connections 2024-09-19T09:25:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.440+0000 7fca252ac640 1 -- 172.21.15.16:0/1299433096 wait complete. 2024-09-19T09:25:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.440+0000 7fca252ac640 1 Processor -- start 2024-09-19T09:25:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca252ac640 1 -- start start 2024-09-19T09:25:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca252ac640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca20102d40 0x7fca20198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca252ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca20103660 0x7fca20198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca252ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2010c170 0x7fca2019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca252ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca20111f70 con 0x7fca2010c170 2024-09-19T09:25:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca252ac640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fca20111df0 con 0x7fca20103660 2024-09-19T09:25:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca252ac640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fca201120f0 con 0x7fca20102d40 2024-09-19T09:25:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca1f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2010c170 0x7fca2019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca1effd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca20102d40 0x7fca20198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca1f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2010c170 0x7fca2019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37482/0 (socket says 172.21.15.16:37482) 2024-09-19T09:25:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca1f7fe640 1 -- 172.21.15.16:0/3045913983 learned_addr learned my addr 172.21.15.16:0/3045913983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca1effd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca20102d40 0x7fca20198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:51808/0 (socket says 172.21.15.16:51808) 2024-09-19T09:25:23.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca1e7fc640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca20103660 0x7fca20198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca1effd640 1 -- 172.21.15.16:0/3045913983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca20103660 msgr2=0x7fca20198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca1effd640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca20103660 0x7fca20198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca1effd640 1 -- 172.21.15.16:0/3045913983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2010c170 msgr2=0x7fca2019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca1effd640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2010c170 0x7fca2019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca1effd640 1 -- 172.21.15.16:0/3045913983 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2019d3c0 con 0x7fca20102d40 2024-09-19T09:25:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca1f7fe640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2010c170 0x7fca2019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.441+0000 7fca1e7fc640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca20103660 0x7fca20198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.442+0000 7fca1effd640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca20102d40 0x7fca20198410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fca1000e9f0 tx=0x7fca1000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:23.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.442+0000 7fca07fff640 1 -- 172.21.15.16:0/3045913983 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1000ccc0 con 0x7fca20102d40 2024-09-19T09:25:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.442+0000 7fca252ac640 1 -- 172.21.15.16:0/3045913983 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2019d6b0 con 0x7fca20102d40 2024-09-19T09:25:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.442+0000 7fca07fff640 1 -- 172.21.15.16:0/3045913983 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca10005020 con 0x7fca20102d40 2024-09-19T09:25:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.442+0000 7fca07fff640 1 -- 172.21.15.16:0/3045913983 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca100049e0 con 0x7fca20102d40 2024-09-19T09:25:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.442+0000 7fca252ac640 1 -- 172.21.15.16:0/3045913983 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca201008d0 con 0x7fca20102d40 2024-09-19T09:25:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.443+0000 7fca252ac640 1 -- 172.21.15.16:0/3045913983 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca2006b070 con 0x7fca20102d40 2024-09-19T09:25:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.444+0000 7fca07fff640 1 -- 172.21.15.16:0/3045913983 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fca10005560 con 0x7fca20102d40 2024-09-19T09:25:23.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.444+0000 7fca07fff640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fca080a3240 0x7fca080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:23.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.445+0000 7fca1e7fc640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fca080a3240 0x7fca080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:23.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.445+0000 7fca07fff640 1 -- 172.21.15.16:0/3045913983 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fca100cf770 con 0x7fca20102d40 2024-09-19T09:25:23.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.445+0000 7fca1e7fc640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fca080a3240 0x7fca080a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fca2010d490 tx=0x7fca14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:23.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.448+0000 7fca07fff640 1 -- 172.21.15.16:0/3045913983 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fca100051d0 con 0x7fca20102d40 2024-09-19T09:25:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.629+0000 7fca252ac640 1 -- 172.21.15.16:0/3045913983 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fca20103120 con 0x7fca20102d40 2024-09-19T09:25:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.630+0000 7fca07fff640 1 -- 172.21.15.16:0/3045913983 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7fca1009c890 con 0x7fca20102d40 2024-09-19T09:25:23.631 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:23.631 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-19T09:25:23.631 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:25:23.631 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-19T09:25:23.631 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-19T09:25:23.631 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:25:23.631 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-19T09:25:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.632+0000 7fca252ac640 1 -- 172.21.15.16:0/3045913983 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fca080a3240 msgr2=0x7fca080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.632+0000 7fca252ac640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fca080a3240 0x7fca080a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fca2010d490 tx=0x7fca14005f90 comp rx=0 tx=0).stop 2024-09-19T09:25:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.632+0000 7fca252ac640 1 -- 172.21.15.16:0/3045913983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca20102d40 msgr2=0x7fca20198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.632+0000 7fca252ac640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca20102d40 0x7fca20198410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fca1000e9f0 tx=0x7fca1000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:25:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.632+0000 7fca252ac640 1 -- 172.21.15.16:0/3045913983 shutdown_connections 2024-09-19T09:25:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.632+0000 7fca252ac640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2010c170 0x7fca2019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.632+0000 7fca252ac640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca20103660 0x7fca20198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.632+0000 7fca252ac640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fca080a3240 0x7fca080a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.632+0000 7fca252ac640 1 --2- 172.21.15.16:0/3045913983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca20102d40 0x7fca20198410 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.632+0000 7fca252ac640 1 -- 172.21.15.16:0/3045913983 >> 172.21.15.16:0/3045913983 conn(0x7fca200fc7a0 msgr2=0x7fca200fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.632+0000 7fca252ac640 1 -- 172.21.15.16:0/3045913983 shutdown_connections 2024-09-19T09:25:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:23.632+0000 7fca252ac640 1 -- 172.21.15.16:0/3045913983 wait complete. 2024-09-19T09:25:24.370 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:24.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:24.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:24.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:24 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:24.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:24 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3045913983' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T09:25:24.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:24.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:24.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:24.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:24 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:24.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:24 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3045913983' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T09:25:24.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:24.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:24.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:24.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:24 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:24.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:24 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3045913983' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T09:25:24.397 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T09:25:24.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T09:25:24.675 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.076+0000 7f333372a640 1 -- 172.21.15.16:0/3658890792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332c106f10 msgr2=0x7f332c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.076+0000 7f333372a640 1 --2- 172.21.15.16:0/3658890792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332c106f10 0x7f332c10ea90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3324009ca0 tx=0x7f332402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:25:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.076+0000 7f333372a640 1 -- 172.21.15.16:0/3658890792 shutdown_connections 2024-09-19T09:25:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.076+0000 7f333372a640 1 --2- 172.21.15.16:0/3658890792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c102790 0x7f332c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.076+0000 7f333372a640 1 --2- 172.21.15.16:0/3658890792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332c106f10 0x7f332c10ea90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.076+0000 7f333372a640 1 --2- 172.21.15.16:0/3658890792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c101de0 0x7f332c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.076+0000 7f333372a640 1 -- 172.21.15.16:0/3658890792 >> 172.21.15.16:0/3658890792 conn(0x7f332c0fd8c0 msgr2=0x7f332c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.076+0000 7f333372a640 1 -- 172.21.15.16:0/3658890792 shutdown_connections 2024-09-19T09:25:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.076+0000 7f333372a640 1 -- 172.21.15.16:0/3658890792 wait complete. 2024-09-19T09:25:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.077+0000 7f333372a640 1 Processor -- start 2024-09-19T09:25:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.077+0000 7f333372a640 1 -- start start 2024-09-19T09:25:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.077+0000 7f333372a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c101de0 0x7f332c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.077+0000 7f333372a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332c102790 0x7f332c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.077+0000 7f333372a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c106f10 0x7f332c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.077+0000 7f333372a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f332c111f70 con 0x7f332c102790 2024-09-19T09:25:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.077+0000 7f333372a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f332c111df0 con 0x7f332c101de0 2024-09-19T09:25:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.077+0000 7f333372a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f332c1120f0 con 0x7f332c106f10 2024-09-19T09:25:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.077+0000 7f3330cc5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332c102790 0x7f332c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.077+0000 7f3330cc5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332c102790 0x7f332c198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37502/0 (socket says 172.21.15.16:37502) 2024-09-19T09:25:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.077+0000 7f3330cc5640 1 -- 172.21.15.16:0/926683154 learned_addr learned my addr 172.21.15.16:0/926683154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.077+0000 7f33314c6640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c101de0 0x7f332c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.077+0000 7f3331cc7640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c106f10 0x7f332c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.078+0000 7f3330cc5640 1 -- 172.21.15.16:0/926683154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c106f10 msgr2=0x7f332c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.078+0000 7f3330cc5640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c106f10 0x7f332c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.078+0000 7f3330cc5640 1 -- 172.21.15.16:0/926683154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c101de0 msgr2=0x7f332c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.078+0000 7f3330cc5640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c101de0 0x7f332c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.078+0000 7f3330cc5640 1 -- 172.21.15.16:0/926683154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332c19d2e0 con 0x7f332c102790 2024-09-19T09:25:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.078+0000 7f3331cc7640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c106f10 0x7f332c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.078+0000 7f3330cc5640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332c102790 0x7f332c198870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f331c00d770 tx=0x7f331c00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.078+0000 7f33227fc640 1 -- 172.21.15.16:0/926683154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c017070 con 0x7f332c102790 2024-09-19T09:25:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.078+0000 7f33227fc640 1 -- 172.21.15.16:0/926683154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f331c009820 con 0x7f332c102790 2024-09-19T09:25:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.078+0000 7f333372a640 1 -- 172.21.15.16:0/926683154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332c19d5d0 con 0x7f332c102790 2024-09-19T09:25:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.078+0000 7f33227fc640 1 -- 172.21.15.16:0/926683154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c004cd0 con 0x7f332c102790 2024-09-19T09:25:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.078+0000 7f333372a640 1 -- 172.21.15.16:0/926683154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f332c1a4eb0 con 0x7f332c102790 2024-09-19T09:25:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.079+0000 7f333372a640 1 -- 172.21.15.16:0/926683154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f332c1021c0 con 0x7f332c102790 2024-09-19T09:25:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.080+0000 7f33227fc640 1 -- 172.21.15.16:0/926683154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f331c0099d0 con 0x7f332c102790 2024-09-19T09:25:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.081+0000 7f33227fc640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f33180a32c0 0x7f33180a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.081+0000 7f33227fc640 1 -- 172.21.15.16:0/926683154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f331c0cedd0 con 0x7f332c102790 2024-09-19T09:25:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.081+0000 7f33314c6640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f33180a32c0 0x7f33180a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.081+0000 7f33314c6640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f33180a32c0 0x7f33180a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f332800abe0 tx=0x7f332800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.085+0000 7f33227fc640 1 -- 172.21.15.16:0/926683154 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f331c09bf90 con 0x7f332c102790 2024-09-19T09:25:25.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:25.263+0000 7f333372a640 1 -- 172.21.15.16:0/926683154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f332c10a150 con 0x7f332c102790 2024-09-19T09:25:25.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:25 smithi016 ceph-mon[30979]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:25.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:25 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:25.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:25 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:25.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:25 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:25.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:25 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:25.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:25 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:25.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:25 smithi142 ceph-mon[35032]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:25.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:25 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:25.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:25 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:25.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:25 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:25.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:25 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:25.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:25 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:25.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:25 smithi097 ceph-mon[34864]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:25.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:25 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:25.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:25 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:25.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:25 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:25:25.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:25 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:25:25.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:25 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:25:26.262 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-19T09:25:26.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:26.260+0000 7f33227fc640 1 -- 172.21.15.16:0/926683154 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7f331c09b890 con 0x7f332c102790 2024-09-19T09:25:26.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:26.263+0000 7f333372a640 1 -- 172.21.15.16:0/926683154 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f33180a32c0 msgr2=0x7f33180a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:26.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:26.263+0000 7f333372a640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f33180a32c0 0x7f33180a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f332800abe0 tx=0x7f332800a2d0 comp rx=0 tx=0).stop 2024-09-19T09:25:26.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:26.263+0000 7f333372a640 1 -- 172.21.15.16:0/926683154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332c102790 msgr2=0x7f332c198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:26.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:26.263+0000 7f333372a640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332c102790 0x7f332c198870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f331c00d770 tx=0x7f331c00db70 comp rx=0 tx=0).stop 2024-09-19T09:25:26.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:26.263+0000 7f333372a640 1 -- 172.21.15.16:0/926683154 shutdown_connections 2024-09-19T09:25:26.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:26.263+0000 7f333372a640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f33180a32c0 0x7f33180a5780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:26.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:26.263+0000 7f333372a640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c106f10 0x7f332c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:26.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:26.263+0000 7f333372a640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f332c102790 0x7f332c198870 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:26.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:26.263+0000 7f333372a640 1 --2- 172.21.15.16:0/926683154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c101de0 0x7f332c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:26.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:26.263+0000 7f333372a640 1 -- 172.21.15.16:0/926683154 >> 172.21.15.16:0/926683154 conn(0x7f332c0fd8c0 msgr2=0x7f332c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:26.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:26.263+0000 7f333372a640 1 -- 172.21.15.16:0/926683154 shutdown_connections 2024-09-19T09:25:26.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:26.264+0000 7f333372a640 1 -- 172.21.15.16:0/926683154 wait complete. 2024-09-19T09:25:26.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:26 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/926683154' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T09:25:26.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:26 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/926683154' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T09:25:26.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:26 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/926683154' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T09:25:26.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T09:25:27.204 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:27.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:27 smithi016 ceph-mon[30979]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:27 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/926683154' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T09:25:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:27 smithi016 ceph-mon[30979]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T09:25:27.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:27 smithi142 ceph-mon[35032]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:27.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:27 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/926683154' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T09:25:27.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:27 smithi142 ceph-mon[35032]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T09:25:27.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:27 smithi097 ceph-mon[34864]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:27.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:27 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/926683154' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T09:25:27.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:27 smithi097 ceph-mon[34864]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T09:25:27.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.645+0000 7f97a30c4640 1 -- 172.21.15.16:0/1708243666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f979c101420 msgr2=0x7f979c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:27.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.645+0000 7f97a30c4640 1 --2- 172.21.15.16:0/1708243666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f979c101420 0x7f979c10d950 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9794009cd0 tx=0x7f979402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:25:27.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.646+0000 7f97a30c4640 1 -- 172.21.15.16:0/1708243666 shutdown_connections 2024-09-19T09:25:27.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.646+0000 7f97a30c4640 1 --2- 172.21.15.16:0/1708243666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f979c100a80 0x7f979c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:27.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.646+0000 7f97a30c4640 1 --2- 172.21.15.16:0/1708243666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f979c101420 0x7f979c10d950 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:27.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.646+0000 7f97a30c4640 1 --2- 172.21.15.16:0/1708243666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c106a80 0x7f979c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:27.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.646+0000 7f97a30c4640 1 -- 172.21.15.16:0/1708243666 >> 172.21.15.16:0/1708243666 conn(0x7f979c0fc7a0 msgr2=0x7f979c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:27.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.646+0000 7f97a30c4640 1 -- 172.21.15.16:0/1708243666 shutdown_connections 2024-09-19T09:25:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.646+0000 7f97a30c4640 1 -- 172.21.15.16:0/1708243666 wait complete. 2024-09-19T09:25:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.646+0000 7f97a30c4640 1 Processor -- start 2024-09-19T09:25:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.646+0000 7f97a30c4640 1 -- start start 2024-09-19T09:25:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.646+0000 7f97a30c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f979c100a80 0x7f979c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.646+0000 7f97a30c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c101420 0x7f979c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a30c4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f979c106a80 0x7f979c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a30c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f979c110e80 con 0x7f979c100a80 2024-09-19T09:25:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a30c4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f979c110d00 con 0x7f979c101420 2024-09-19T09:25:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a30c4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f979c111000 con 0x7f979c106a80 2024-09-19T09:25:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a0e60640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f979c100a80 0x7f979c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a0e60640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f979c100a80 0x7f979c197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43572/0 (socket says 172.21.15.16:43572) 2024-09-19T09:25:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a0e60640 1 -- 172.21.15.16:0/2366361194 learned_addr learned my addr 172.21.15.16:0/2366361194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f979bfff640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c101420 0x7f979c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:27.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a0e60640 1 -- 172.21.15.16:0/2366361194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f979c106a80 msgr2=0x7f979c19bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:27.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a0e60640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f979c106a80 0x7f979c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:27.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a0e60640 1 -- 172.21.15.16:0/2366361194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c101420 msgr2=0x7f979c197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:27.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a0e60640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c101420 0x7f979c197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:27.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a0e60640 1 -- 172.21.15.16:0/2366361194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979c19c2e0 con 0x7f979c100a80 2024-09-19T09:25:27.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f979bfff640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c101420 0x7f979c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:27.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a0e60640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f979c100a80 0x7f979c197330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f978c00eb00 tx=0x7f978c009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:27.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f9799ffb640 1 -- 172.21.15.16:0/2366361194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c01c070 con 0x7f979c100a80 2024-09-19T09:25:27.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f97a30c4640 1 -- 172.21.15.16:0/2366361194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979c19c5d0 con 0x7f979c100a80 2024-09-19T09:25:27.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f9799ffb640 1 -- 172.21.15.16:0/2366361194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f978c00bc30 con 0x7f979c100a80 2024-09-19T09:25:27.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.647+0000 7f9799ffb640 1 -- 172.21.15.16:0/2366361194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c0166a0 con 0x7f979c100a80 2024-09-19T09:25:27.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.648+0000 7f97a30c4640 1 -- 172.21.15.16:0/2366361194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f979c06a790 con 0x7f979c100a80 2024-09-19T09:25:27.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.649+0000 7f97a30c4640 1 -- 172.21.15.16:0/2366361194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f979c100ee0 con 0x7f979c100a80 2024-09-19T09:25:27.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.651+0000 7f9799ffb640 1 -- 172.21.15.16:0/2366361194 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f978c01f070 con 0x7f979c100a80 2024-09-19T09:25:27.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.651+0000 7f9799ffb640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f97840a3240 0x7f97840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:27.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.651+0000 7f979bfff640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f97840a3240 0x7f97840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:27.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.651+0000 7f9799ffb640 1 -- 172.21.15.16:0/2366361194 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7f978c0d10b0 con 0x7f979c100a80 2024-09-19T09:25:27.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.652+0000 7f979bfff640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f97840a3240 0x7f97840a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9790005fd0 tx=0x7f9790005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:27.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.655+0000 7f9799ffb640 1 -- 172.21.15.16:0/2366361194 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f978c00bde0 con 0x7f979c100a80 2024-09-19T09:25:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:27.831+0000 7f97a30c4640 1 -- 172.21.15.16:0/2366361194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f979c198690 con 0x7f979c100a80 2024-09-19T09:25:28.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:28.275+0000 7f9799ffb640 1 -- 172.21.15.16:0/2366361194 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7f978c09e100 con 0x7f979c100a80 2024-09-19T09:25:28.276 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-19T09:25:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:28.278+0000 7f97a30c4640 1 -- 172.21.15.16:0/2366361194 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f97840a3240 msgr2=0x7f97840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:28.278+0000 7f97a30c4640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f97840a3240 0x7f97840a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9790005fd0 tx=0x7f9790005ea0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:28.278+0000 7f97a30c4640 1 -- 172.21.15.16:0/2366361194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f979c100a80 msgr2=0x7f979c197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:28.278+0000 7f97a30c4640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f979c100a80 0x7f979c197330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f978c00eb00 tx=0x7f978c009f20 comp rx=0 tx=0).stop 2024-09-19T09:25:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:28.278+0000 7f97a30c4640 1 -- 172.21.15.16:0/2366361194 shutdown_connections 2024-09-19T09:25:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:28.278+0000 7f97a30c4640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f97840a3240 0x7f97840a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:28.278+0000 7f97a30c4640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f979c106a80 0x7f979c19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:28.278+0000 7f97a30c4640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f979c101420 0x7f979c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:28.278+0000 7f97a30c4640 1 --2- 172.21.15.16:0/2366361194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f979c100a80 0x7f979c197330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:28.278+0000 7f97a30c4640 1 -- 172.21.15.16:0/2366361194 >> 172.21.15.16:0/2366361194 conn(0x7f979c0fc7a0 msgr2=0x7f979c0fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:28.278+0000 7f97a30c4640 1 -- 172.21.15.16:0/2366361194 shutdown_connections 2024-09-19T09:25:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:28.279+0000 7f97a30c4640 1 -- 172.21.15.16:0/2366361194 wait complete. 2024-09-19T09:25:28.585 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:28 smithi016 ceph-mon[30979]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T09:25:28.585 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:28 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2366361194' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T09:25:28.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:28 smithi142 ceph-mon[35032]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T09:25:28.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:28 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2366361194' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T09:25:28.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:28 smithi097 ceph-mon[34864]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T09:25:28.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:28 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2366361194' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T09:25:28.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T09:25:29.161 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:29.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:29 smithi016 ceph-mon[30979]: pgmap v269: 33 pgs: 18 creating+peering, 1 active+clean, 14 unknown; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:29.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:29 smithi016 ceph-mon[30979]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T09:25:29.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:29 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2366361194' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T09:25:29.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:29 smithi016 ceph-mon[30979]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T09:25:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.606+0000 7f3562d6b640 1 -- 172.21.15.16:0/2310466430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f355c100a80 msgr2=0x7f355c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.606+0000 7f3562d6b640 1 --2- 172.21.15.16:0/2310466430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f355c100a80 0x7f355c100ee0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f354c009930 tx=0x7f354c02a420 comp rx=0 tx=0).stop 2024-09-19T09:25:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.606+0000 7f3562d6b640 1 -- 172.21.15.16:0/2310466430 shutdown_connections 2024-09-19T09:25:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.606+0000 7f3562d6b640 1 --2- 172.21.15.16:0/2310466430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355c101420 0x7f355c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.606+0000 7f3562d6b640 1 --2- 172.21.15.16:0/2310466430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f355c100a80 0x7f355c100ee0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.606+0000 7f3562d6b640 1 --2- 172.21.15.16:0/2310466430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f355c106a80 0x7f355c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.606+0000 7f3562d6b640 1 -- 172.21.15.16:0/2310466430 >> 172.21.15.16:0/2310466430 conn(0x7f355c0fc7a0 msgr2=0x7f355c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.606+0000 7f3562d6b640 1 -- 172.21.15.16:0/2310466430 shutdown_connections 2024-09-19T09:25:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.606+0000 7f3562d6b640 1 -- 172.21.15.16:0/2310466430 wait complete. 2024-09-19T09:25:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.606+0000 7f3562d6b640 1 Processor -- start 2024-09-19T09:25:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3562d6b640 1 -- start start 2024-09-19T09:25:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3562d6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f355c100a80 0x7f355c0fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3562d6b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355c101420 0x7f355c0fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3562d6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f355c106a80 0x7f355c0fd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3562d6b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f355c110e80 con 0x7f355c106a80 2024-09-19T09:25:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3562d6b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f355c110d00 con 0x7f355c100a80 2024-09-19T09:25:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3562d6b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f355c111000 con 0x7f355c101420 2024-09-19T09:25:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3561308640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f355c106a80 0x7f355c0fd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3561308640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f355c106a80 0x7f355c0fd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43580/0 (socket says 172.21.15.16:43580) 2024-09-19T09:25:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3561308640 1 -- 172.21.15.16:0/1561921231 learned_addr learned my addr 172.21.15.16:0/1561921231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3560b07640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f355c100a80 0x7f355c0fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3553fff640 1 --2- 172.21.15.16:0/1561921231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355c101420 0x7f355c0fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3561308640 1 -- 172.21.15.16:0/1561921231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355c101420 msgr2=0x7f355c0fd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3561308640 1 --2- 172.21.15.16:0/1561921231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355c101420 0x7f355c0fd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3561308640 1 -- 172.21.15.16:0/1561921231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f355c100a80 msgr2=0x7f355c0fcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3561308640 1 --2- 172.21.15.16:0/1561921231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f355c100a80 0x7f355c0fcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.607+0000 7f3561308640 1 -- 172.21.15.16:0/1561921231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c1a68f0 con 0x7f355c106a80 2024-09-19T09:25:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.608+0000 7f3561308640 1 --2- 172.21.15.16:0/1561921231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f355c106a80 0x7f355c0fd7e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f355400eaa0 tx=0x7f355400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.608+0000 7f3551ffb640 1 -- 172.21.15.16:0/1561921231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355400cdb0 con 0x7f355c106a80 2024-09-19T09:25:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.608+0000 7f3551ffb640 1 -- 172.21.15.16:0/1561921231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3554005020 con 0x7f355c106a80 2024-09-19T09:25:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.608+0000 7f3562d6b640 1 -- 172.21.15.16:0/1561921231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355c1a6be0 con 0x7f355c106a80 2024-09-19T09:25:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.608+0000 7f3551ffb640 1 -- 172.21.15.16:0/1561921231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35540116b0 con 0x7f355c106a80 2024-09-19T09:25:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.608+0000 7f3562d6b640 1 -- 172.21.15.16:0/1561921231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355c1a70f0 con 0x7f355c106a80 2024-09-19T09:25:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.609+0000 7f3562d6b640 1 -- 172.21.15.16:0/1561921231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355c100ee0 con 0x7f355c106a80 2024-09-19T09:25:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.610+0000 7f3551ffb640 1 -- 172.21.15.16:0/1561921231 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3554005560 con 0x7f355c106a80 2024-09-19T09:25:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.611+0000 7f3551ffb640 1 --2- 172.21.15.16:0/1561921231 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f35440a3240 0x7f35440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.611+0000 7f3551ffb640 1 -- 172.21.15.16:0/1561921231 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7f35540cf9a0 con 0x7f355c106a80 2024-09-19T09:25:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.611+0000 7f3560b07640 1 --2- 172.21.15.16:0/1561921231 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f35440a3240 0x7f35440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.612+0000 7f3560b07640 1 --2- 172.21.15.16:0/1561921231 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f35440a3240 0x7f35440a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f355800bb60 tx=0x7f355800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.616+0000 7f3551ffb640 1 -- 172.21.15.16:0/1561921231 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3554016030 con 0x7f355c106a80 2024-09-19T09:25:29.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:29 smithi142 ceph-mon[35032]: pgmap v269: 33 pgs: 18 creating+peering, 1 active+clean, 14 unknown; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:29.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:29 smithi142 ceph-mon[35032]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T09:25:29.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:29 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2366361194' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T09:25:29.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:29 smithi142 ceph-mon[35032]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T09:25:29.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:29 smithi097 ceph-mon[34864]: pgmap v269: 33 pgs: 18 creating+peering, 1 active+clean, 14 unknown; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:29.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:29 smithi097 ceph-mon[34864]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T09:25:29.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:29 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2366361194' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T09:25:29.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:29 smithi097 ceph-mon[34864]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T09:25:29.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:29.827+0000 7f3562d6b640 1 -- 172.21.15.16:0/1561921231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f355c109a90 con 0x7f355c106a80 2024-09-19T09:25:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:30.284+0000 7f3551ffb640 1 -- 172.21.15.16:0/1561921231 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f355409c180 con 0x7f355c106a80 2024-09-19T09:25:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T09:25:30.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:30.287+0000 7f3562d6b640 1 -- 172.21.15.16:0/1561921231 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f35440a3240 msgr2=0x7f35440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:30.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:30.287+0000 7f3562d6b640 1 --2- 172.21.15.16:0/1561921231 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f35440a3240 0x7f35440a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f355800bb60 tx=0x7f355800b390 comp rx=0 tx=0).stop 2024-09-19T09:25:30.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:30.287+0000 7f3562d6b640 1 -- 172.21.15.16:0/1561921231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f355c106a80 msgr2=0x7f355c0fd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:30.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:30.287+0000 7f3562d6b640 1 --2- 172.21.15.16:0/1561921231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f355c106a80 0x7f355c0fd7e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f355400eaa0 tx=0x7f355400ef60 comp rx=0 tx=0).stop 2024-09-19T09:25:30.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:30.287+0000 7f3562d6b640 1 -- 172.21.15.16:0/1561921231 shutdown_connections 2024-09-19T09:25:30.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:30.287+0000 7f3562d6b640 1 --2- 172.21.15.16:0/1561921231 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f35440a3240 0x7f35440a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:30.287+0000 7f3562d6b640 1 --2- 172.21.15.16:0/1561921231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355c101420 0x7f355c0fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:30.287+0000 7f3562d6b640 1 --2- 172.21.15.16:0/1561921231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f355c106a80 0x7f355c0fd7e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:30.287+0000 7f3562d6b640 1 --2- 172.21.15.16:0/1561921231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f355c100a80 0x7f355c0fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:30.287+0000 7f3562d6b640 1 -- 172.21.15.16:0/1561921231 >> 172.21.15.16:0/1561921231 conn(0x7f355c0fc7a0 msgr2=0x7f355c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:30.287+0000 7f3562d6b640 1 -- 172.21.15.16:0/1561921231 shutdown_connections 2024-09-19T09:25:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:30.287+0000 7f3562d6b640 1 -- 172.21.15.16:0/1561921231 wait complete. 2024-09-19T09:25:30.617 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:30 smithi016 ceph-mon[30979]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T09:25:30.617 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:30 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1561921231' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T09:25:30.618 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:30 smithi016 ceph-mon[30979]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T09:25:30.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:30 smithi142 ceph-mon[35032]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T09:25:30.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:30 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1561921231' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T09:25:30.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:30 smithi142 ceph-mon[35032]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T09:25:30.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:30 smithi097 ceph-mon[34864]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T09:25:30.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:30 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1561921231' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T09:25:30.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:30 smithi097 ceph-mon[34864]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T09:25:30.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:25:31.178 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:31.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:31 smithi016 ceph-mon[30979]: pgmap v272: 97 pgs: 64 unknown, 18 creating+peering, 15 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:31.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:31 smithi016 ceph-mon[30979]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T09:25:31.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:31 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1561921231' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T09:25:31.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:31 smithi016 ceph-mon[30979]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T09:25:31.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:31 smithi016 ceph-mon[30979]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T09:25:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.602+0000 7feca4fba640 1 -- 172.21.15.16:0/4141592677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feca0108300 msgr2=0x7feca010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.602+0000 7feca4fba640 1 --2- 172.21.15.16:0/4141592677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feca0108300 0x7feca010ec70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fec98009cd0 tx=0x7fec9802a8d0 comp rx=0 tx=0).stop 2024-09-19T09:25:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.603+0000 7feca4fba640 1 -- 172.21.15.16:0/4141592677 shutdown_connections 2024-09-19T09:25:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.603+0000 7feca4fba640 1 --2- 172.21.15.16:0/4141592677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feca0108300 0x7feca010ec70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.603+0000 7feca4fba640 1 --2- 172.21.15.16:0/4141592677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0103780 0x7feca0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.603+0000 7feca4fba640 1 --2- 172.21.15.16:0/4141592677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca0102dd0 0x7feca01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.603+0000 7feca4fba640 1 -- 172.21.15.16:0/4141592677 >> 172.21.15.16:0/4141592677 conn(0x7feca00fc7d0 msgr2=0x7feca00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.603+0000 7feca4fba640 1 -- 172.21.15.16:0/4141592677 shutdown_connections 2024-09-19T09:25:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.603+0000 7feca4fba640 1 -- 172.21.15.16:0/4141592677 wait complete. 2024-09-19T09:25:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.603+0000 7feca4fba640 1 Processor -- start 2024-09-19T09:25:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.604+0000 7feca4fba640 1 -- start start 2024-09-19T09:25:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.604+0000 7feca4fba640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca0102dd0 0x7feca0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.604+0000 7feca4fba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0103780 0x7feca0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.604+0000 7feca4fba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feca0108300 0x7feca019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.604+0000 7feca4fba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feca01120d0 con 0x7feca0108300 2024-09-19T09:25:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.604+0000 7feca4fba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feca0111f50 con 0x7feca0103780 2024-09-19T09:25:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.604+0000 7feca4fba640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feca0112250 con 0x7feca0102dd0 2024-09-19T09:25:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.604+0000 7fec9ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feca0108300 0x7feca019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.604+0000 7fec9e59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca0102dd0 0x7feca0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.604+0000 7fec9ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feca0108300 0x7feca019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43594/0 (socket says 172.21.15.16:43594) 2024-09-19T09:25:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.604+0000 7fec9ed9d640 1 -- 172.21.15.16:0/2180748235 learned_addr learned my addr 172.21.15.16:0/2180748235 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.604+0000 7fec95d9b640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0103780 0x7feca0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.605+0000 7fec9e59c640 1 -- 172.21.15.16:0/2180748235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0103780 msgr2=0x7feca0198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.605+0000 7fec9e59c640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0103780 0x7feca0198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.605+0000 7fec9e59c640 1 -- 172.21.15.16:0/2180748235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feca0108300 msgr2=0x7feca019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.605+0000 7fec9e59c640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feca0108300 0x7feca019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.605+0000 7fec9e59c640 1 -- 172.21.15.16:0/2180748235 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca019d520 con 0x7feca0102dd0 2024-09-19T09:25:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.605+0000 7fec9ed9d640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feca0108300 0x7feca019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:25:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.605+0000 7fec9e59c640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca0102dd0 0x7feca0198570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec9000d720 tx=0x7fec9000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.605+0000 7fec97fff640 1 -- 172.21.15.16:0/2180748235 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90017070 con 0x7feca0102dd0 2024-09-19T09:25:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.605+0000 7fec97fff640 1 -- 172.21.15.16:0/2180748235 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec9000de30 con 0x7feca0102dd0 2024-09-19T09:25:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.605+0000 7feca4fba640 1 -- 172.21.15.16:0/2180748235 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca019d810 con 0x7feca0102dd0 2024-09-19T09:25:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.605+0000 7fec97fff640 1 -- 172.21.15.16:0/2180748235 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900047a0 con 0x7feca0102dd0 2024-09-19T09:25:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.605+0000 7feca4fba640 1 -- 172.21.15.16:0/2180748235 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca01a50f0 con 0x7feca0102dd0 2024-09-19T09:25:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.606+0000 7feca4fba640 1 -- 172.21.15.16:0/2180748235 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca01031b0 con 0x7feca0102dd0 2024-09-19T09:25:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.608+0000 7fec97fff640 1 -- 172.21.15.16:0/2180748235 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fec900099d0 con 0x7feca0102dd0 2024-09-19T09:25:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.608+0000 7fec97fff640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fec8c0a3240 0x7fec8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.608+0000 7fec95d9b640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fec8c0a3240 0x7fec8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.608+0000 7fec97fff640 1 -- 172.21.15.16:0/2180748235 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fec900cf880 con 0x7feca0102dd0 2024-09-19T09:25:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.609+0000 7fec95d9b640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fec8c0a3240 0x7fec8c0a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feca00fee30 tx=0x7fec8800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.612+0000 7fec97fff640 1 -- 172.21.15.16:0/2180748235 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fec90020020 con 0x7feca0102dd0 2024-09-19T09:25:31.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:31 smithi142 ceph-mon[35032]: pgmap v272: 97 pgs: 64 unknown, 18 creating+peering, 15 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:31.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:31 smithi142 ceph-mon[35032]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T09:25:31.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:31 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1561921231' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T09:25:31.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:31 smithi142 ceph-mon[35032]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T09:25:31.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:31 smithi142 ceph-mon[35032]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T09:25:31.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:31 smithi097 ceph-mon[34864]: pgmap v272: 97 pgs: 64 unknown, 18 creating+peering, 15 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:31.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:31 smithi097 ceph-mon[34864]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T09:25:31.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:31 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1561921231' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T09:25:31.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:31 smithi097 ceph-mon[34864]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T09:25:31.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:31 smithi097 ceph-mon[34864]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T09:25:31.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.791+0000 7feca4fba640 1 -- 172.21.15.16:0/2180748235 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feca010a290 con 0x7feca0102dd0 2024-09-19T09:25:31.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.792+0000 7fec97fff640 1 -- 172.21.15.16:0/2180748235 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22242 (secure 0 0 0) 0x7fec9009c650 con 0x7feca0102dd0 2024-09-19T09:25:31.794 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:31.794 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":79,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","created":"2024-09-19T09:17:48.242238+0000","modified":"2024-09-19T09:25:29.828798+0000","last_up_change":"2024-09-19T09:24:40.190289+0000","last_in_change":"2024-09-19T09:24:17.190380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:20:57.748686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:25:25.694451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:25:28.262997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbf3b28b-2282-4057-9e65-19ceb78ff19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6803","nonce":1578957623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6805","nonce":1578957623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6809","nonce":1578957623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6807","nonce":1578957623}]},"public_addr":"172.21.15.16:6803/1578957623","cluster_addr":"172.21.15.16:6805/1578957623","heartbeat_back_addr":"172.21.15.16:6809/1578957623","heartbeat_front_addr":"172.21.15.16:6807/1578957623","state":["exists","up"]},{"osd":1,"uuid":"c2517aeb-1d79-41ad-a180-369eecb9732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6801","nonce":3804152349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6803","nonce":3804152349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6807","nonce":3804152349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6805","nonce":3804152349}]},"public_addr":"172.21.15.97:6801/3804152349","cluster_addr":"172.21.15.97:6803/3804152349","heartbeat_back_addr":"172.21.15.97:6807/3804152349","heartbeat_front_addr":"172.21.15.97:6805/3804152349","state":["exists","up"]},{"osd":2,"uuid":"3c367518-5085-471a-98cb-ec5bf5facdf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6801","nonce":2692613951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6803","nonce":2692613951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6807","nonce":2692613951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6805","nonce":2692613951}]},"public_addr":"172.21.15.142:6801/2692613951","cluster_addr":"172.21.15.142:6803/2692613951","heartbeat_back_addr":"172.21.15.142:6807/2692613951","heartbeat_front_addr":"172.21.15.142:6805/2692613951","state":["exists","up"]},{"osd":3,"uuid":"e7118925-f0fa-46b3-8552-3e3bb76a6489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6811","nonce":3169290004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6813","nonce":3169290004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6817","nonce":3169290004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6815","nonce":3169290004}]},"public_addr":"172.21.15.16:6811/3169290004","cluster_addr":"172.21.15.16:6813/3169290004","heartbeat_back_addr":"172.21.15.16:6817/3169290004","heartbeat_front_addr":"172.21.15.16:6815/3169290004","state":["exists","up"]},{"osd":4,"uuid":"dc1964f7-af9c-4853-843b-9818f87c7768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6809","nonce":3491353825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6811","nonce":3491353825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6815","nonce":3491353825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6813","nonce":3491353825}]},"public_addr":"172.21.15.97:6809/3491353825","cluster_addr":"172.21.15.97:6811/3491353825","heartbeat_back_addr":"172.21.15.97:6815/3491353825","heartbeat_front_addr":"172.21.15.97:6813/3491353825","state":["exists","up"]},{"osd":5,"uuid":"4d7fbafc-d9ba-4036-a89d-aa295b8a23a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6809","nonce":577099986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6811","nonce":577099986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6815","nonce":577099986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6813","nonce":577099986}]},"public_addr":"172.21.15.142:6809/577099986","cluster_addr":"172.21.15.142:6811/577099986","heartbeat_back_addr":"172.21.15.142:6815/577099986","heartbeat_front_addr":"172.21.15.142:6813/577099986","state":["exists","up"]},{"osd":6,"uuid":"e3f346fc-c027-44f8-bd37-bccf6ec09e3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6819","nonce":4053539536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6821","nonce":4053539536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6825","nonce":4053539536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6823","nonce":4053539536}]},"public_addr":"172.21.15.16:6819/4053539536","cluster_addr":"172.21.15.16:6821/4053539536","heartbeat_back_addr":"172.21.15.16:6825/4053539536","heartbeat_front_addr":"172.21.15.16:6823/4053539536","state":["exists","up"]},{"osd":7,"uuid":"d390b547-8949-4d4b-8481-c49d6ae7e424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6817","nonce":1157122670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6819","nonce":1157122670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6823","nonce":1157122670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6821","nonce":1157122670}]},"public_addr":"172.21.15.97:6817/1157122670","cluster_addr":"172.21.15.97:6819/1157122670","heartbeat_back_addr":"172.21.15.97:6823/1157122670","heartbeat_front_addr":"172.21.15.97:6821/1157122670","state":["exists","up"]},{"osd":8,"uuid":"78a476c4-5deb-4f9e-a266-db4493850732","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6817","nonce":643795004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6819","nonce":643795004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6823","nonce":643795004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6821","nonce":643795004}]},"public_addr":"172.21.15.142:6817/643795004","cluster_addr":"172.21.15.142:6819/643795004","heartbeat_back_addr":"172.21.15.142:6823/643795004","heartbeat_front_addr":"172.21.15.142:6821/643795004","state":["exists","up"]},{"osd":9,"uuid":"0415d187-4a77-42f7-b8bb-2d0b52e9cb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6827","nonce":2651132360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6829","nonce":2651132360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2651132360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6831","nonce":2651132360}]},"public_addr":"172.21.15.16:6827/2651132360","cluster_addr":"172.21.15.16:6829/2651132360","heartbeat_back_addr":"172.21.15.16:6833/2651132360","heartbeat_front_addr":"172.21.15.16:6831/2651132360","state":["exists","up"]},{"osd":10,"uuid":"6fa78f96-cdfe-42a0-8d27-0705a0c7df8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6825","nonce":1848371486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6827","nonce":1848371486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6831","nonce":1848371486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6829","nonce":1848371486}]},"public_addr":"172.21.15.97:6825/1848371486","cluster_addr":"172.21.15.97:6827/1848371486","heartbeat_back_addr":"172.21.15.97:6831/1848371486","heartbeat_front_addr":"172.21.15.97:6829/1848371486","state":["exists","up"]},{"osd":11,"uuid":"d27f20e3-84ce-470c-9889-cd86906b158d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6825","nonce":155083878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6827","nonce":155083878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6831","nonce":155083878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6829","nonce":155083878}]},"public_addr":"172.21.15.142:6825/155083878","cluster_addr":"172.21.15.142:6827/155083878","heartbeat_back_addr":"172.21.15.142:6831/155083878","heartbeat_front_addr":"172.21.15.142:6829/155083878","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:05.088764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:29.928529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:55.236930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:20.379471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:44.984151+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:10.410074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:35.233553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:59.928700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:24.503069+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:49.853073+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:14.006841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:38.546467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3710267992":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3740275561":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/2755421444":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/357390846":"2024-09-20T09:18:17.070034+0000","172.21.15.16:6800/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3104087402":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3546947161":"2024-09-20T09:18:17.070034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:25:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.794+0000 7feca4fba640 1 -- 172.21.15.16:0/2180748235 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fec8c0a3240 msgr2=0x7fec8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.794+0000 7feca4fba640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fec8c0a3240 0x7fec8c0a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feca00fee30 tx=0x7fec8800b390 comp rx=0 tx=0).stop 2024-09-19T09:25:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.794+0000 7feca4fba640 1 -- 172.21.15.16:0/2180748235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca0102dd0 msgr2=0x7feca0198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.794+0000 7feca4fba640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca0102dd0 0x7feca0198570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec9000d720 tx=0x7fec9000dbe0 comp rx=0 tx=0).stop 2024-09-19T09:25:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.795+0000 7feca4fba640 1 -- 172.21.15.16:0/2180748235 shutdown_connections 2024-09-19T09:25:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.795+0000 7feca4fba640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feca0108300 0x7feca019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.795+0000 7feca4fba640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0103780 0x7feca0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.795+0000 7feca4fba640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fec8c0a3240 0x7fec8c0a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.795+0000 7feca4fba640 1 --2- 172.21.15.16:0/2180748235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feca0102dd0 0x7feca0198570 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.795+0000 7feca4fba640 1 -- 172.21.15.16:0/2180748235 >> 172.21.15.16:0/2180748235 conn(0x7feca00fc7d0 msgr2=0x7feca0105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.795+0000 7feca4fba640 1 -- 172.21.15.16:0/2180748235 shutdown_connections 2024-09-19T09:25:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:31.795+0000 7feca4fba640 1 -- 172.21.15.16:0/2180748235 wait complete. 2024-09-19T09:25:32.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:25:32.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:32 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] up:active 2024-09-19T09:25:32.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:32 smithi142 ceph-mon[35032]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:25:32.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:32 smithi142 ceph-mon[35032]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:25:32.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:32 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2180748235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:25:32.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:32 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] up:active 2024-09-19T09:25:32.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:32 smithi097 ceph-mon[34864]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:25:32.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:32 smithi097 ceph-mon[34864]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:25:32.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:32 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2180748235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:25:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:32.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:32 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] up:active 2024-09-19T09:25:32.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:32 smithi016 ceph-mon[30979]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:25:32.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:32 smithi016 ceph-mon[30979]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:25:32.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:32 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2180748235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:25:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.127+0000 7f99914e0640 1 -- 172.21.15.16:0/244895912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f998c106f10 msgr2=0x7f998c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.127+0000 7f99914e0640 1 --2- 172.21.15.16:0/244895912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f998c106f10 0x7f998c10ea90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9984009ca0 tx=0x7f998402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.127+0000 7f99914e0640 1 -- 172.21.15.16:0/244895912 shutdown_connections 2024-09-19T09:25:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.127+0000 7f99914e0640 1 --2- 172.21.15.16:0/244895912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f998c102790 0x7f998c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.127+0000 7f99914e0640 1 --2- 172.21.15.16:0/244895912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f998c106f10 0x7f998c10ea90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.127+0000 7f99914e0640 1 --2- 172.21.15.16:0/244895912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f998c101de0 0x7f998c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.127+0000 7f99914e0640 1 -- 172.21.15.16:0/244895912 >> 172.21.15.16:0/244895912 conn(0x7f998c0fd8c0 msgr2=0x7f998c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.127+0000 7f99914e0640 1 -- 172.21.15.16:0/244895912 shutdown_connections 2024-09-19T09:25:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.127+0000 7f99914e0640 1 -- 172.21.15.16:0/244895912 wait complete. 2024-09-19T09:25:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.127+0000 7f99914e0640 1 Processor -- start 2024-09-19T09:25:33.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f99914e0640 1 -- start start 2024-09-19T09:25:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f99914e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f998c101de0 0x7f998c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f99914e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f998c102790 0x7f998c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f99914e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f998c106f10 0x7f998c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f99914e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f998c112010 con 0x7f998c102790 2024-09-19T09:25:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f99914e0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f998c111e90 con 0x7f998c101de0 2024-09-19T09:25:33.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f99914e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f998c112190 con 0x7f998c106f10 2024-09-19T09:25:33.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f998a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f998c102790 0x7f998c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f998a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f998c102790 0x7f998c198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43604/0 (socket says 172.21.15.16:43604) 2024-09-19T09:25:33.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f998a7fc640 1 -- 172.21.15.16:0/1482963312 learned_addr learned my addr 172.21.15.16:0/1482963312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:33.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f998affd640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f998c101de0 0x7f998c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f998b7fe640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f998c106f10 0x7f998c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f998a7fc640 1 -- 172.21.15.16:0/1482963312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f998c106f10 msgr2=0x7f998c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f998a7fc640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f998c106f10 0x7f998c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f998a7fc640 1 -- 172.21.15.16:0/1482963312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f998c101de0 msgr2=0x7f998c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f998a7fc640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f998c101de0 0x7f998c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f998a7fc640 1 -- 172.21.15.16:0/1482963312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c19d5a0 con 0x7f998c102790 2024-09-19T09:25:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.128+0000 7f998b7fe640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f998c106f10 0x7f998c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.129+0000 7f998a7fc640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f998c102790 0x7f998c198b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f997400b4a0 tx=0x7f997400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.129+0000 7f9983fff640 1 -- 172.21.15.16:0/1482963312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974015020 con 0x7f998c102790 2024-09-19T09:25:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.129+0000 7f9983fff640 1 -- 172.21.15.16:0/1482963312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9974011d10 con 0x7f998c102790 2024-09-19T09:25:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.129+0000 7f99914e0640 1 -- 172.21.15.16:0/1482963312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998c19d890 con 0x7f998c102790 2024-09-19T09:25:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.129+0000 7f9983fff640 1 -- 172.21.15.16:0/1482963312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997400f2e0 con 0x7f998c102790 2024-09-19T09:25:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.129+0000 7f99914e0640 1 -- 172.21.15.16:0/1482963312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f998c1a5170 con 0x7f998c102790 2024-09-19T09:25:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.130+0000 7f9981ffb640 1 -- 172.21.15.16:0/1482963312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9970005290 con 0x7f998c102790 2024-09-19T09:25:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.131+0000 7f9983fff640 1 -- 172.21.15.16:0/1482963312 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9974018020 con 0x7f998c102790 2024-09-19T09:25:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.132+0000 7f9983fff640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f99780a32c0 0x7f99780a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.132+0000 7f998affd640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f99780a32c0 0x7f99780a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.132+0000 7f9983fff640 1 -- 172.21.15.16:0/1482963312 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f99740cf690 con 0x7f998c102790 2024-09-19T09:25:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.132+0000 7f998affd640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f99780a32c0 0x7f99780a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f997c009990 tx=0x7f997c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.135+0000 7f9983fff640 1 -- 172.21.15.16:0/1482963312 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f997401f020 con 0x7f998c102790 2024-09-19T09:25:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.313+0000 7f9981ffb640 1 -- 172.21.15.16:0/1482963312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9970005850 con 0x7f998c102790 2024-09-19T09:25:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.314+0000 7f9983fff640 1 -- 172.21.15.16:0/1482963312 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22242 (secure 0 0 0) 0x7f997409c460 con 0x7f998c102790 2024-09-19T09:25:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:33.316 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":79,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","created":"2024-09-19T09:17:48.242238+0000","modified":"2024-09-19T09:25:29.828798+0000","last_up_change":"2024-09-19T09:24:40.190289+0000","last_in_change":"2024-09-19T09:24:17.190380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:20:57.748686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:25:25.694451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:25:28.262997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbf3b28b-2282-4057-9e65-19ceb78ff19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6803","nonce":1578957623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6805","nonce":1578957623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6809","nonce":1578957623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6807","nonce":1578957623}]},"public_addr":"172.21.15.16:6803/1578957623","cluster_addr":"172.21.15.16:6805/1578957623","heartbeat_back_addr":"172.21.15.16:6809/1578957623","heartbeat_front_addr":"172.21.15.16:6807/1578957623","state":["exists","up"]},{"osd":1,"uuid":"c2517aeb-1d79-41ad-a180-369eecb9732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6801","nonce":3804152349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6803","nonce":3804152349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6807","nonce":3804152349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6805","nonce":3804152349}]},"public_addr":"172.21.15.97:6801/3804152349","cluster_addr":"172.21.15.97:6803/3804152349","heartbeat_back_addr":"172.21.15.97:6807/3804152349","heartbeat_front_addr":"172.21.15.97:6805/3804152349","state":["exists","up"]},{"osd":2,"uuid":"3c367518-5085-471a-98cb-ec5bf5facdf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6801","nonce":2692613951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6803","nonce":2692613951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6807","nonce":2692613951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6805","nonce":2692613951}]},"public_addr":"172.21.15.142:6801/2692613951","cluster_addr":"172.21.15.142:6803/2692613951","heartbeat_back_addr":"172.21.15.142:6807/2692613951","heartbeat_front_addr":"172.21.15.142:6805/2692613951","state":["exists","up"]},{"osd":3,"uuid":"e7118925-f0fa-46b3-8552-3e3bb76a6489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6811","nonce":3169290004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6813","nonce":3169290004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6817","nonce":3169290004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6815","nonce":3169290004}]},"public_addr":"172.21.15.16:6811/3169290004","cluster_addr":"172.21.15.16:6813/3169290004","heartbeat_back_addr":"172.21.15.16:6817/3169290004","heartbeat_front_addr":"172.21.15.16:6815/3169290004","state":["exists","up"]},{"osd":4,"uuid":"dc1964f7-af9c-4853-843b-9818f87c7768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6809","nonce":3491353825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6811","nonce":3491353825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6815","nonce":3491353825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6813","nonce":3491353825}]},"public_addr":"172.21.15.97:6809/3491353825","cluster_addr":"172.21.15.97:6811/3491353825","heartbeat_back_addr":"172.21.15.97:6815/3491353825","heartbeat_front_addr":"172.21.15.97:6813/3491353825","state":["exists","up"]},{"osd":5,"uuid":"4d7fbafc-d9ba-4036-a89d-aa295b8a23a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6809","nonce":577099986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6811","nonce":577099986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6815","nonce":577099986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6813","nonce":577099986}]},"public_addr":"172.21.15.142:6809/577099986","cluster_addr":"172.21.15.142:6811/577099986","heartbeat_back_addr":"172.21.15.142:6815/577099986","heartbeat_front_addr":"172.21.15.142:6813/577099986","state":["exists","up"]},{"osd":6,"uuid":"e3f346fc-c027-44f8-bd37-bccf6ec09e3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6819","nonce":4053539536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6821","nonce":4053539536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6825","nonce":4053539536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6823","nonce":4053539536}]},"public_addr":"172.21.15.16:6819/4053539536","cluster_addr":"172.21.15.16:6821/4053539536","heartbeat_back_addr":"172.21.15.16:6825/4053539536","heartbeat_front_addr":"172.21.15.16:6823/4053539536","state":["exists","up"]},{"osd":7,"uuid":"d390b547-8949-4d4b-8481-c49d6ae7e424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6817","nonce":1157122670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6819","nonce":1157122670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6823","nonce":1157122670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6821","nonce":1157122670}]},"public_addr":"172.21.15.97:6817/1157122670","cluster_addr":"172.21.15.97:6819/1157122670","heartbeat_back_addr":"172.21.15.97:6823/1157122670","heartbeat_front_addr":"172.21.15.97:6821/1157122670","state":["exists","up"]},{"osd":8,"uuid":"78a476c4-5deb-4f9e-a266-db4493850732","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6817","nonce":643795004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6819","nonce":643795004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6823","nonce":643795004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6821","nonce":643795004}]},"public_addr":"172.21.15.142:6817/643795004","cluster_addr":"172.21.15.142:6819/643795004","heartbeat_back_addr":"172.21.15.142:6823/643795004","heartbeat_front_addr":"172.21.15.142:6821/643795004","state":["exists","up"]},{"osd":9,"uuid":"0415d187-4a77-42f7-b8bb-2d0b52e9cb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6827","nonce":2651132360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6829","nonce":2651132360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2651132360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6831","nonce":2651132360}]},"public_addr":"172.21.15.16:6827/2651132360","cluster_addr":"172.21.15.16:6829/2651132360","heartbeat_back_addr":"172.21.15.16:6833/2651132360","heartbeat_front_addr":"172.21.15.16:6831/2651132360","state":["exists","up"]},{"osd":10,"uuid":"6fa78f96-cdfe-42a0-8d27-0705a0c7df8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6825","nonce":1848371486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6827","nonce":1848371486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6831","nonce":1848371486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6829","nonce":1848371486}]},"public_addr":"172.21.15.97:6825/1848371486","cluster_addr":"172.21.15.97:6827/1848371486","heartbeat_back_addr":"172.21.15.97:6831/1848371486","heartbeat_front_addr":"172.21.15.97:6829/1848371486","state":["exists","up"]},{"osd":11,"uuid":"d27f20e3-84ce-470c-9889-cd86906b158d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6825","nonce":155083878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6827","nonce":155083878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6831","nonce":155083878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6829","nonce":155083878}]},"public_addr":"172.21.15.142:6825/155083878","cluster_addr":"172.21.15.142:6827/155083878","heartbeat_back_addr":"172.21.15.142:6831/155083878","heartbeat_front_addr":"172.21.15.142:6829/155083878","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:05.088764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:29.928529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:55.236930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:20.379471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:44.984151+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:10.410074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:35.233553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:59.928700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:24.503069+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:49.853073+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:14.006841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:38.546467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3710267992":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3740275561":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/2755421444":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/357390846":"2024-09-20T09:18:17.070034+0000","172.21.15.16:6800/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3104087402":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3546947161":"2024-09-20T09:18:17.070034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:25:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.316+0000 7f9981ffb640 1 -- 172.21.15.16:0/1482963312 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f99780a32c0 msgr2=0x7f99780a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.316+0000 7f9981ffb640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f99780a32c0 0x7f99780a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f997c009990 tx=0x7f997c009110 comp rx=0 tx=0).stop 2024-09-19T09:25:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.316+0000 7f9981ffb640 1 -- 172.21.15.16:0/1482963312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f998c102790 msgr2=0x7f998c198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.316+0000 7f9981ffb640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f998c102790 0x7f998c198b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f997400b4a0 tx=0x7f997400b8a0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.316+0000 7f9981ffb640 1 -- 172.21.15.16:0/1482963312 shutdown_connections 2024-09-19T09:25:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.316+0000 7f9981ffb640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f99780a32c0 0x7f99780a5780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.316+0000 7f9981ffb640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f998c106f10 0x7f998c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.316+0000 7f9981ffb640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f998c102790 0x7f998c198b30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.316+0000 7f9981ffb640 1 --2- 172.21.15.16:0/1482963312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f998c101de0 0x7f998c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.316+0000 7f9981ffb640 1 -- 172.21.15.16:0/1482963312 >> 172.21.15.16:0/1482963312 conn(0x7f998c0fd8c0 msgr2=0x7f998c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.316+0000 7f9981ffb640 1 -- 172.21.15.16:0/1482963312 shutdown_connections 2024-09-19T09:25:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:33.316+0000 7f9981ffb640 1 -- 172.21.15.16:0/1482963312 wait complete. 2024-09-19T09:25:33.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:33 smithi097 ceph-mon[34864]: pgmap v274: 97 pgs: 57 unknown, 18 creating+peering, 22 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:33.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:33 smithi097 ceph-mon[34864]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T09:25:33.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:33 smithi097 ceph-mon[34864]: Cluster is now healthy 2024-09-19T09:25:33.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:33 smithi016 ceph-mon[30979]: pgmap v274: 97 pgs: 57 unknown, 18 creating+peering, 22 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:33.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:33 smithi016 ceph-mon[30979]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T09:25:33.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:33 smithi016 ceph-mon[30979]: Cluster is now healthy 2024-09-19T09:25:33.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:33 smithi142 ceph-mon[35032]: pgmap v274: 97 pgs: 57 unknown, 18 creating+peering, 22 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:25:33.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:33 smithi142 ceph-mon[35032]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T09:25:33.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:33 smithi142 ceph-mon[35032]: Cluster is now healthy 2024-09-19T09:25:34.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T09:25:34.287 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:34.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:34 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1482963312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:25:34.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:34 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1482963312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:25:34.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:34 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1482963312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:25:34.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.704+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/3824833804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f4101440 msgr2=0x7fa7f410df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.704+0000 7fa7fc1c7640 1 --2- 172.21.15.16:0/3824833804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f4101440 0x7fa7f410df30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009900 tx=0x7fa7f002a410 comp rx=0 tx=0).stop 2024-09-19T09:25:34.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.704+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/3824833804 shutdown_connections 2024-09-19T09:25:34.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.704+0000 7fa7fc1c7640 1 --2- 172.21.15.16:0/3824833804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f410e500 0x7fa7f4110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.704+0000 7fa7fc1c7640 1 --2- 172.21.15.16:0/3824833804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f4101440 0x7fa7f410df30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.704+0000 7fa7fc1c7640 1 --2- 172.21.15.16:0/3824833804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7f4100b20 0x7fa7f4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.704+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/3824833804 >> 172.21.15.16:0/3824833804 conn(0x7fa7f40fc7d0 msgr2=0x7fa7f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:34.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.704+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/3824833804 shutdown_connections 2024-09-19T09:25:34.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.704+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/3824833804 wait complete. 2024-09-19T09:25:34.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.705+0000 7fa7fc1c7640 1 Processor -- start 2024-09-19T09:25:34.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.705+0000 7fa7fc1c7640 1 -- start start 2024-09-19T09:25:34.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.705+0000 7fa7fc1c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f4100b20 0x7fa7f41a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.705+0000 7fa7fc1c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7f4101440 0x7fa7f41a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.705+0000 7fa7fc1c7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f410e500 0x7fa7f419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.705+0000 7fa7fc1c7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f4114270 con 0x7fa7f4100b20 2024-09-19T09:25:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.705+0000 7fa7fc1c7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41140f0 con 0x7fa7f410e500 2024-09-19T09:25:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.705+0000 7fa7fc1c7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41143f0 con 0x7fa7f4101440 2024-09-19T09:25:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.705+0000 7fa7f9f63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f4100b20 0x7fa7f41a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.705+0000 7fa7f9f63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f4100b20 0x7fa7f41a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43620/0 (socket says 172.21.15.16:43620) 2024-09-19T09:25:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7f9f63640 1 -- 172.21.15.16:0/1651483627 learned_addr learned my addr 172.21.15.16:0/1651483627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7f9762640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7f4101440 0x7fa7f41a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7f9f63640 1 -- 172.21.15.16:0/1651483627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7f4101440 msgr2=0x7fa7f41a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7f9f63640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7f4101440 0x7fa7f41a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7fa764640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f410e500 0x7fa7f419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7f9f63640 1 -- 172.21.15.16:0/1651483627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f410e500 msgr2=0x7fa7f419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7f9f63640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f410e500 0x7fa7f419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7f9f63640 1 -- 172.21.15.16:0/1651483627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f419ad30 con 0x7fa7f4100b20 2024-09-19T09:25:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7fa764640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f410e500 0x7fa7f419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:25:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7f9f63640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f4100b20 0x7fa7f41a0440 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800d770 tx=0x7fa7e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7e6ffd640 1 -- 172.21.15.16:0/1651483627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8017070 con 0x7fa7f4100b20 2024-09-19T09:25:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/1651483627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f419b020 con 0x7fa7f4100b20 2024-09-19T09:25:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7e6ffd640 1 -- 172.21.15.16:0/1651483627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e8005500 con 0x7fa7f4100b20 2024-09-19T09:25:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7e6ffd640 1 -- 172.21.15.16:0/1651483627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8004e00 con 0x7fa7f4100b20 2024-09-19T09:25:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.706+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/1651483627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f41a7310 con 0x7fa7f4100b20 2024-09-19T09:25:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.707+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/1651483627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f4100f00 con 0x7fa7f4100b20 2024-09-19T09:25:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.708+0000 7fa7e6ffd640 1 -- 172.21.15.16:0/1651483627 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa7e8004fa0 con 0x7fa7f4100b20 2024-09-19T09:25:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.709+0000 7fa7e6ffd640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa7e00a3240 0x7fa7e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.709+0000 7fa7e6ffd640 1 -- 172.21.15.16:0/1651483627 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fa7e80cf2f0 con 0x7fa7f4100b20 2024-09-19T09:25:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.709+0000 7fa7f9762640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa7e00a3240 0x7fa7e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.710+0000 7fa7f9762640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa7e00a3240 0x7fa7e00a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0006010 tx=0x7fa7f0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.713+0000 7fa7e6ffd640 1 -- 172.21.15.16:0/1651483627 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa7e809c160 con 0x7fa7f4100b20 2024-09-19T09:25:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:34.930+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/1651483627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa7f4109600 con 0x7fa7f4100b20 2024-09-19T09:25:35.298 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:25:34 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b[64788]: 2024-09-19T09:25:34.951+0000 7f59efbe7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T09:25:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:35.301+0000 7fa7e6ffd640 1 -- 172.21.15.16:0/1651483627 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fa7e809ba60 con 0x7fa7f4100b20 2024-09-19T09:25:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:35.304+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/1651483627 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa7e00a3240 msgr2=0x7fa7e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:35.304+0000 7fa7fc1c7640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa7e00a3240 0x7fa7e00a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0006010 tx=0x7fa7f0000bc0 comp rx=0 tx=0).stop 2024-09-19T09:25:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:35.304+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/1651483627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f4100b20 msgr2=0x7fa7f41a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:35.304+0000 7fa7fc1c7640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f4100b20 0x7fa7f41a0440 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800d770 tx=0x7fa7e800dc30 comp rx=0 tx=0).stop 2024-09-19T09:25:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:35.305+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/1651483627 shutdown_connections 2024-09-19T09:25:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:35.305+0000 7fa7fc1c7640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa7e00a3240 0x7fa7e00a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:35.305+0000 7fa7fc1c7640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7f4101440 0x7fa7f41a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:35.305+0000 7fa7fc1c7640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f410e500 0x7fa7f419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:35.305+0000 7fa7fc1c7640 1 --2- 172.21.15.16:0/1651483627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f4100b20 0x7fa7f41a0440 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:35.305+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/1651483627 >> 172.21.15.16:0/1651483627 conn(0x7fa7f40fc7d0 msgr2=0x7fa7f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:35.305+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/1651483627 shutdown_connections 2024-09-19T09:25:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:35.305+0000 7fa7fc1c7640 1 -- 172.21.15.16:0/1651483627 wait complete. 2024-09-19T09:25:35.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:35 smithi142 ceph-mon[35032]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-19T09:25:35.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:35 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1651483627' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T09:25:35.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:35 smithi097 ceph-mon[34864]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-19T09:25:35.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:35 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1651483627' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T09:25:35.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:35 smithi016 ceph-mon[30979]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-19T09:25:35.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:35 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1651483627' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T09:25:35.966 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-19T09:25:35.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-19T09:25:36.232 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:36.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:36 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1651483627' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T09:25:36.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:36 smithi142 ceph-mon[35032]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:25:36.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:36 smithi142 ceph-mon[35032]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T09:25:36.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:36 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1651483627' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T09:25:36.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:36 smithi097 ceph-mon[34864]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:25:36.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:36 smithi097 ceph-mon[34864]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T09:25:36.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:36 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1651483627' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T09:25:36.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:36 smithi016 ceph-mon[30979]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:25:36.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:36 smithi016 ceph-mon[30979]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-19T09:25:36.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.727+0000 7fde2b3e3640 1 -- 172.21.15.16:0/769415974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101440 msgr2=0x7fde24101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:36.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.727+0000 7fde2b3e3640 1 --2- 172.21.15.16:0/769415974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101440 0x7fde24101820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fde20009900 tx=0x7fde2002a410 comp rx=0 tx=0).stop 2024-09-19T09:25:36.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.727+0000 7fde2b3e3640 1 -- 172.21.15.16:0/769415974 shutdown_connections 2024-09-19T09:25:36.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.727+0000 7fde2b3e3640 1 --2- 172.21.15.16:0/769415974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde24106050 0x7fde2410dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.727+0000 7fde2b3e3640 1 --2- 172.21.15.16:0/769415974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde24101df0 0x7fde24105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.727+0000 7fde2b3e3640 1 --2- 172.21.15.16:0/769415974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101440 0x7fde24101820 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.727+0000 7fde2b3e3640 1 -- 172.21.15.16:0/769415974 >> 172.21.15.16:0/769415974 conn(0x7fde24077210 msgr2=0x7fde240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.727+0000 7fde2b3e3640 1 -- 172.21.15.16:0/769415974 shutdown_connections 2024-09-19T09:25:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.727+0000 7fde2b3e3640 1 -- 172.21.15.16:0/769415974 wait complete. 2024-09-19T09:25:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2b3e3640 1 Processor -- start 2024-09-19T09:25:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2b3e3640 1 -- start start 2024-09-19T09:25:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2b3e3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde24101440 0x7fde240fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2b3e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101df0 0x7fde240fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2b3e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde24106050 0x7fde240fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2b3e3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde241110d0 con 0x7fde24101df0 2024-09-19T09:25:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2b3e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde24110f50 con 0x7fde24106050 2024-09-19T09:25:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2b3e3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fde24111250 con 0x7fde24101440 2024-09-19T09:25:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2897e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101df0 0x7fde240fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2897e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101df0 0x7fde240fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35308/0 (socket says 172.21.15.16:35308) 2024-09-19T09:25:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2897e640 1 -- 172.21.15.16:0/139567512 learned_addr learned my addr 172.21.15.16:0/139567512 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2917f640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde24101440 0x7fde240fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde29980640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde24106050 0x7fde240fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2897e640 1 -- 172.21.15.16:0/139567512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde24101440 msgr2=0x7fde240fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2897e640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde24101440 0x7fde240fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2897e640 1 -- 172.21.15.16:0/139567512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde24106050 msgr2=0x7fde240fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2897e640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde24106050 0x7fde240fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.728+0000 7fde2897e640 1 -- 172.21.15.16:0/139567512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde241a6bf0 con 0x7fde24101df0 2024-09-19T09:25:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.729+0000 7fde2897e640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101df0 0x7fde240fcbd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde1400b4a0 tx=0x7fde1400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.729+0000 7fde1a7fc640 1 -- 172.21.15.16:0/139567512 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14014070 con 0x7fde24101df0 2024-09-19T09:25:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.729+0000 7fde1a7fc640 1 -- 172.21.15.16:0/139567512 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde14010af0 con 0x7fde24101df0 2024-09-19T09:25:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.729+0000 7fde2b3e3640 1 -- 172.21.15.16:0/139567512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde241a6ee0 con 0x7fde24101df0 2024-09-19T09:25:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.729+0000 7fde1a7fc640 1 -- 172.21.15.16:0/139567512 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14017d10 con 0x7fde24101df0 2024-09-19T09:25:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.729+0000 7fde2b3e3640 1 -- 172.21.15.16:0/139567512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde241a73e0 con 0x7fde24101df0 2024-09-19T09:25:36.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.730+0000 7fde2b3e3640 1 -- 172.21.15.16:0/139567512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde24101820 con 0x7fde24101df0 2024-09-19T09:25:36.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.731+0000 7fde1a7fc640 1 -- 172.21.15.16:0/139567512 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fde14017070 con 0x7fde24101df0 2024-09-19T09:25:36.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.732+0000 7fde1a7fc640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fde100a32c0 0x7fde100a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:36.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.732+0000 7fde2917f640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fde100a32c0 0x7fde100a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:36.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.732+0000 7fde1a7fc640 1 -- 172.21.15.16:0/139567512 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fde140cef70 con 0x7fde24101df0 2024-09-19T09:25:36.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.732+0000 7fde2917f640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fde100a32c0 0x7fde100a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde200096c0 tx=0x7fde20031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.735+0000 7fde1a7fc640 1 -- 172.21.15.16:0/139567512 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fde1409bd40 con 0x7fde24101df0 2024-09-19T09:25:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:36.948+0000 7fde2b3e3640 1 -- 172.21.15.16:0/139567512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fde241a76d0 con 0x7fde24101df0 2024-09-19T09:25:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:37.312+0000 7fde1a7fc640 1 -- 172.21.15.16:0/139567512 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fde14097070 con 0x7fde24101df0 2024-09-19T09:25:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:37.314+0000 7fde2b3e3640 1 -- 172.21.15.16:0/139567512 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fde100a32c0 msgr2=0x7fde100a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:37.314+0000 7fde2b3e3640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fde100a32c0 0x7fde100a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde200096c0 tx=0x7fde20031490 comp rx=0 tx=0).stop 2024-09-19T09:25:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:37.314+0000 7fde2b3e3640 1 -- 172.21.15.16:0/139567512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101df0 msgr2=0x7fde240fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:37.314+0000 7fde2b3e3640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101df0 0x7fde240fcbd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde1400b4a0 tx=0x7fde1400b960 comp rx=0 tx=0).stop 2024-09-19T09:25:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:37.314+0000 7fde2b3e3640 1 -- 172.21.15.16:0/139567512 shutdown_connections 2024-09-19T09:25:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:37.314+0000 7fde2b3e3640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde24106050 0x7fde240fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:37.314+0000 7fde2b3e3640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101df0 0x7fde240fcbd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:37.314+0000 7fde2b3e3640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fde100a32c0 0x7fde100a5780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:37.314+0000 7fde2b3e3640 1 --2- 172.21.15.16:0/139567512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde24101440 0x7fde240fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:37.314+0000 7fde2b3e3640 1 -- 172.21.15.16:0/139567512 >> 172.21.15.16:0/139567512 conn(0x7fde24077210 msgr2=0x7fde24100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:37.315+0000 7fde2b3e3640 1 -- 172.21.15.16:0/139567512 shutdown_connections 2024-09-19T09:25:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:37.315+0000 7fde2b3e3640 1 -- 172.21.15.16:0/139567512 wait complete. 2024-09-19T09:25:37.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:37 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/139567512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T09:25:37.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:37 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/139567512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T09:25:37.640 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:37 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/139567512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T09:25:37.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T09:25:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:38.627 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:38 smithi016 ceph-mon[30979]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T09:25:38.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:38 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/139567512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T09:25:38.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:38 smithi016 ceph-mon[30979]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T09:25:38.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:38 smithi016 ceph-mon[30979]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:25:38.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:38 smithi016 ceph-mon[30979]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T09:25:38.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:38 smithi016 ceph-mon[30979]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T09:25:38.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:38 smithi016 ceph-mon[30979]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T09:25:38.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:38 smithi142 ceph-mon[35032]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T09:25:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:38 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/139567512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T09:25:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:38 smithi142 ceph-mon[35032]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T09:25:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:38 smithi142 ceph-mon[35032]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:25:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:38 smithi142 ceph-mon[35032]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T09:25:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:38 smithi142 ceph-mon[35032]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T09:25:38.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:38 smithi142 ceph-mon[35032]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T09:25:38.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:38 smithi097 ceph-mon[34864]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T09:25:38.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:38 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/139567512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T09:25:38.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:38 smithi097 ceph-mon[34864]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T09:25:38.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:38 smithi097 ceph-mon[34864]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T09:25:38.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:38 smithi097 ceph-mon[34864]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T09:25:38.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:38 smithi097 ceph-mon[34864]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T09:25:38.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:38 smithi097 ceph-mon[34864]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T09:25:38.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.712+0000 7f0193b04640 1 -- 172.21.15.16:0/1652558891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c102790 msgr2=0x7f018c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.712+0000 7f0193b04640 1 --2- 172.21.15.16:0/1652558891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c102790 0x7f018c1067e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0180009900 tx=0x7f018002a410 comp rx=0 tx=0).stop 2024-09-19T09:25:38.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.712+0000 7f0193b04640 1 -- 172.21.15.16:0/1652558891 shutdown_connections 2024-09-19T09:25:38.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.712+0000 7f0193b04640 1 --2- 172.21.15.16:0/1652558891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f018c106f10 0x7f018c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.712+0000 7f0193b04640 1 --2- 172.21.15.16:0/1652558891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c102790 0x7f018c1067e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.712+0000 7f0193b04640 1 --2- 172.21.15.16:0/1652558891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c101de0 0x7f018c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.712+0000 7f0193b04640 1 -- 172.21.15.16:0/1652558891 >> 172.21.15.16:0/1652558891 conn(0x7f018c0fd8c0 msgr2=0x7f018c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.712+0000 7f0193b04640 1 -- 172.21.15.16:0/1652558891 shutdown_connections 2024-09-19T09:25:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.712+0000 7f0193b04640 1 -- 172.21.15.16:0/1652558891 wait complete. 2024-09-19T09:25:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.712+0000 7f0193b04640 1 Processor -- start 2024-09-19T09:25:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f0193b04640 1 -- start start 2024-09-19T09:25:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f0193b04640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f018c101de0 0x7f018c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f0193b04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c102790 0x7f018c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f0193b04640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c106f10 0x7f018c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f0193b04640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f018c112010 con 0x7f018c102790 2024-09-19T09:25:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f0193b04640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f018c111e90 con 0x7f018c106f10 2024-09-19T09:25:38.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f0193b04640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f018c112190 con 0x7f018c101de0 2024-09-19T09:25:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f019109f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c102790 0x7f018c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f019109f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c102790 0x7f018c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35318/0 (socket says 172.21.15.16:35318) 2024-09-19T09:25:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f019109f640 1 -- 172.21.15.16:0/1555354932 learned_addr learned my addr 172.21.15.16:0/1555354932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f01918a0640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f018c101de0 0x7f018c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f01920a1640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c106f10 0x7f018c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f019109f640 1 -- 172.21.15.16:0/1555354932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f018c101de0 msgr2=0x7f018c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f019109f640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f018c101de0 0x7f018c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f019109f640 1 -- 172.21.15.16:0/1555354932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c106f10 msgr2=0x7f018c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f019109f640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c106f10 0x7f018c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f019109f640 1 -- 172.21.15.16:0/1555354932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018c19d560 con 0x7f018c102790 2024-09-19T09:25:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f01920a1640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c106f10 0x7f018c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.713+0000 7f019109f640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c102790 0x7f018c198af0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0180005ab0 tx=0x7f01800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.714+0000 7f017effd640 1 -- 172.21.15.16:0/1555354932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018003f030 con 0x7f018c102790 2024-09-19T09:25:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.714+0000 7f017effd640 1 -- 172.21.15.16:0/1555354932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0180003ce0 con 0x7f018c102790 2024-09-19T09:25:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.714+0000 7f0193b04640 1 -- 172.21.15.16:0/1555354932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018c19d7f0 con 0x7f018c102790 2024-09-19T09:25:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.714+0000 7f017effd640 1 -- 172.21.15.16:0/1555354932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180035070 con 0x7f018c102790 2024-09-19T09:25:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.714+0000 7f0193b04640 1 -- 172.21.15.16:0/1555354932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f018c1a5130 con 0x7f018c102790 2024-09-19T09:25:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.715+0000 7f0193b04640 1 -- 172.21.15.16:0/1555354932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f018c1032c0 con 0x7f018c102790 2024-09-19T09:25:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.716+0000 7f017effd640 1 -- 172.21.15.16:0/1555354932 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0180042030 con 0x7f018c102790 2024-09-19T09:25:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.716+0000 7f017effd640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f01780a3240 0x7f01780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:38.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.717+0000 7f017effd640 1 -- 172.21.15.16:0/1555354932 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f01800f0380 con 0x7f018c102790 2024-09-19T09:25:38.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.717+0000 7f01918a0640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f01780a3240 0x7f01780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:38.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.717+0000 7f01918a0640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f01780a3240 0x7f01780a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0188009990 tx=0x7f0188009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:38.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.719+0000 7f017effd640 1 -- 172.21.15.16:0/1555354932 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f01800bd1f0 con 0x7f018c102790 2024-09-19T09:25:38.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.931+0000 7f0193b04640 1 -- 172.21.15.16:0/1555354932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f018c10a0f0 con 0x7f018c102790 2024-09-19T09:25:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.952+0000 7f017effd640 1 -- 172.21.15.16:0/1555354932 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f01800bcaf0 con 0x7f018c102790 2024-09-19T09:25:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.954+0000 7f0193b04640 1 -- 172.21.15.16:0/1555354932 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f01780a3240 msgr2=0x7f01780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.954+0000 7f0193b04640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f01780a3240 0x7f01780a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0188009990 tx=0x7f0188009110 comp rx=0 tx=0).stop 2024-09-19T09:25:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.954+0000 7f0193b04640 1 -- 172.21.15.16:0/1555354932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c102790 msgr2=0x7f018c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.954+0000 7f0193b04640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c102790 0x7f018c198af0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0180005ab0 tx=0x7f01800042f0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.954+0000 7f0193b04640 1 -- 172.21.15.16:0/1555354932 shutdown_connections 2024-09-19T09:25:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.954+0000 7f0193b04640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f018c106f10 0x7f018c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.954+0000 7f0193b04640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c102790 0x7f018c198af0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.954+0000 7f0193b04640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f01780a3240 0x7f01780a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.954+0000 7f0193b04640 1 --2- 172.21.15.16:0/1555354932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f018c101de0 0x7f018c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.954+0000 7f0193b04640 1 -- 172.21.15.16:0/1555354932 >> 172.21.15.16:0/1555354932 conn(0x7f018c0fd8c0 msgr2=0x7f018c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.954+0000 7f0193b04640 1 -- 172.21.15.16:0/1555354932 shutdown_connections 2024-09-19T09:25:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:38.954+0000 7f0193b04640 1 -- 172.21.15.16:0/1555354932 wait complete. 2024-09-19T09:25:39.590 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:39 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] up:active 2024-09-19T09:25:39.590 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:39 smithi016 ceph-mon[30979]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T09:25:39.590 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:39 smithi016 ceph-mon[30979]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T09:25:39.590 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:39 smithi016 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-19T09:25:39.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:39 smithi016 ceph-mon[30979]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-19T09:25:39.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:39 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1555354932' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T09:25:39.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:39 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1555354932' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T09:25:39.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:39 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] up:active 2024-09-19T09:25:39.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:39 smithi016 ceph-mon[30979]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T09:25:39.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:39 smithi016 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T09:25:39.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:39 smithi016 ceph-mon[30979]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-09-19T09:25:39.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:39 smithi016 ceph-mon[30979]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-09-19T09:25:39.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T09:25:39.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:39 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] up:active 2024-09-19T09:25:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:39 smithi142 ceph-mon[35032]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T09:25:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:39 smithi142 ceph-mon[35032]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T09:25:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:39 smithi142 ceph-mon[35032]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-19T09:25:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:39 smithi142 ceph-mon[35032]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-19T09:25:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:39 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1555354932' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T09:25:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:39 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1555354932' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T09:25:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:39 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] up:active 2024-09-19T09:25:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:39 smithi142 ceph-mon[35032]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T09:25:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:39 smithi142 ceph-mon[35032]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T09:25:39.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:39 smithi142 ceph-mon[35032]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-09-19T09:25:39.630 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:39 smithi142 ceph-mon[35032]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-09-19T09:25:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] up:active 2024-09-19T09:25:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-mon[34864]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T09:25:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-mon[34864]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T09:25:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-mon[34864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-19T09:25:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-mon[34864]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-19T09:25:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1555354932' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T09:25:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1555354932' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T09:25:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] up:active 2024-09-19T09:25:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-mon[34864]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T09:25:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-mon[34864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T09:25:39.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-mon[34864]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-09-19T09:25:39.637 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-mon[34864]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-09-19T09:25:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.372+0000 7f76e2341640 1 -- 172.21.15.16:0/218293332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc108300 msgr2=0x7f76dc10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.372+0000 7f76e2341640 1 --2- 172.21.15.16:0/218293332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc108300 0x7f76dc10ec70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009cd0 tx=0x7f76d402a8d0 comp rx=0 tx=0).stop 2024-09-19T09:25:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.372+0000 7f76e2341640 1 -- 172.21.15.16:0/218293332 shutdown_connections 2024-09-19T09:25:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.372+0000 7f76e2341640 1 --2- 172.21.15.16:0/218293332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76dc103780 0x7f76dc107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.372+0000 7f76e2341640 1 --2- 172.21.15.16:0/218293332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc108300 0x7f76dc10ec70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.372+0000 7f76e2341640 1 --2- 172.21.15.16:0/218293332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76dc102dd0 0x7f76dc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.372+0000 7f76e2341640 1 -- 172.21.15.16:0/218293332 >> 172.21.15.16:0/218293332 conn(0x7f76dc0fc7d0 msgr2=0x7f76dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.372+0000 7f76e2341640 1 -- 172.21.15.16:0/218293332 shutdown_connections 2024-09-19T09:25:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.372+0000 7f76e2341640 1 -- 172.21.15.16:0/218293332 wait complete. 2024-09-19T09:25:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.373+0000 7f76e2341640 1 Processor -- start 2024-09-19T09:25:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.373+0000 7f76e2341640 1 -- start start 2024-09-19T09:25:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.373+0000 7f76e2341640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc102dd0 0x7f76dc198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.373+0000 7f76e2341640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76dc103780 0x7f76dc198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.373+0000 7f76e2341640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76dc108300 0x7f76dc19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.373+0000 7f76e2341640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc1120d0 con 0x7f76dc102dd0 2024-09-19T09:25:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.373+0000 7f76e2341640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc111f50 con 0x7f76dc103780 2024-09-19T09:25:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.373+0000 7f76e2341640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc112250 con 0x7f76dc108300 2024-09-19T09:25:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.373+0000 7f76dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc102dd0 0x7f76dc198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.373+0000 7f76dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc102dd0 0x7f76dc198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35326/0 (socket says 172.21.15.16:35326) 2024-09-19T09:25:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.373+0000 7f76dbfff640 1 -- 172.21.15.16:0/230779551 learned_addr learned my addr 172.21.15.16:0/230779551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76d3fff640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76dc103780 0x7f76dc198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76e08de640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76dc108300 0x7f76dc19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76dbfff640 1 -- 172.21.15.16:0/230779551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76dc108300 msgr2=0x7f76dc19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76dbfff640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76dc108300 0x7f76dc19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76dbfff640 1 -- 172.21.15.16:0/230779551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76dc103780 msgr2=0x7f76dc198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76dbfff640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76dc103780 0x7f76dc198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76dbfff640 1 -- 172.21.15.16:0/230779551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76dc19d540 con 0x7f76dc102dd0 2024-09-19T09:25:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76d3fff640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76dc103780 0x7f76dc198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76dbfff640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc102dd0 0x7f76dc198590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00d950 tx=0x7f76cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76d9ffb640 1 -- 172.21.15.16:0/230779551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc017070 con 0x7f76dc102dd0 2024-09-19T09:25:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76e2341640 1 -- 172.21.15.16:0/230779551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76dc19d830 con 0x7f76dc102dd0 2024-09-19T09:25:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76d9ffb640 1 -- 172.21.15.16:0/230779551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76cc009c30 con 0x7f76dc102dd0 2024-09-19T09:25:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76d9ffb640 1 -- 172.21.15.16:0/230779551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00f6b0 con 0x7f76dc102dd0 2024-09-19T09:25:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.374+0000 7f76e2341640 1 -- 172.21.15.16:0/230779551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76dc1a5110 con 0x7f76dc102dd0 2024-09-19T09:25:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.375+0000 7f76e2341640 1 -- 172.21.15.16:0/230779551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76dc1031b0 con 0x7f76dc102dd0 2024-09-19T09:25:40.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.377+0000 7f76d9ffb640 1 -- 172.21.15.16:0/230779551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f76cc0109d0 con 0x7f76dc102dd0 2024-09-19T09:25:40.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.377+0000 7f76d9ffb640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f76c80a3240 0x7f76c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:40.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.377+0000 7f76d3fff640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f76c80a3240 0x7f76c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:40.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.377+0000 7f76d9ffb640 1 -- 172.21.15.16:0/230779551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f76cc0cf810 con 0x7f76dc102dd0 2024-09-19T09:25:40.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.378+0000 7f76d3fff640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f76c80a3240 0x7f76c80a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f76c4005d70 tx=0x7f76c4005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:40.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.382+0000 7f76d9ffb640 1 -- 172.21.15.16:0/230779551 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f76cc009de0 con 0x7f76dc102dd0 2024-09-19T09:25:40.386 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b[64788]: 2024-09-19T09:25:39.951+0000 7f59efbe7640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T09:25:40.386 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:25:39 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b[64788]: 2024-09-19T09:25:39.951+0000 7f59efbe7640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T09:25:40.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.596+0000 7f76e2341640 1 -- 172.21.15.16:0/230779551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f76dc10a290 con 0x7f76dc102dd0 2024-09-19T09:25:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.968+0000 7f76d9ffb640 1 -- 172.21.15.16:0/230779551 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f76cc09c680 con 0x7f76dc102dd0 2024-09-19T09:25:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.970+0000 7f76e2341640 1 -- 172.21.15.16:0/230779551 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f76c80a3240 msgr2=0x7f76c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.970+0000 7f76e2341640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f76c80a3240 0x7f76c80a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f76c4005d70 tx=0x7f76c4005d00 comp rx=0 tx=0).stop 2024-09-19T09:25:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.970+0000 7f76e2341640 1 -- 172.21.15.16:0/230779551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc102dd0 msgr2=0x7f76dc198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.970+0000 7f76e2341640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc102dd0 0x7f76dc198590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00d950 tx=0x7f76cc00de10 comp rx=0 tx=0).stop 2024-09-19T09:25:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.970+0000 7f76e2341640 1 -- 172.21.15.16:0/230779551 shutdown_connections 2024-09-19T09:25:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.970+0000 7f76e2341640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f76c80a3240 0x7f76c80a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:40.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.970+0000 7f76e2341640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76dc108300 0x7f76dc19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:40.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.970+0000 7f76e2341640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76dc103780 0x7f76dc198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:40.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.970+0000 7f76e2341640 1 --2- 172.21.15.16:0/230779551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc102dd0 0x7f76dc198590 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:40.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.970+0000 7f76e2341640 1 -- 172.21.15.16:0/230779551 >> 172.21.15.16:0/230779551 conn(0x7f76dc0fc7d0 msgr2=0x7f76dc105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:40.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.970+0000 7f76e2341640 1 -- 172.21.15.16:0/230779551 shutdown_connections 2024-09-19T09:25:40.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:40.970+0000 7f76e2341640 1 -- 172.21.15.16:0/230779551 wait complete. 2024-09-19T09:25:41.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:40 smithi016 ceph-mon[30979]: pgmap v278: 97 pgs: 97 active+clean; 580 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T09:25:41.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:40 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] up:active 2024-09-19T09:25:41.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:40 smithi016 ceph-mon[30979]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T09:25:41.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:40 smithi016 ceph-mon[30979]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T09:25:41.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:40 smithi016 ceph-mon[30979]: Cluster is now healthy 2024-09-19T09:25:41.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:40 smithi016 ceph-mon[30979]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-09-19T09:25:41.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:40 smithi016 ceph-mon[30979]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-19T09:25:41.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:40 smithi016 ceph-mon[30979]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-19T09:25:41.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:40 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/230779551' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T09:25:41.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:40 smithi142 ceph-mon[35032]: pgmap v278: 97 pgs: 97 active+clean; 580 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T09:25:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:40 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] up:active 2024-09-19T09:25:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:40 smithi142 ceph-mon[35032]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T09:25:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:40 smithi142 ceph-mon[35032]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T09:25:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:40 smithi142 ceph-mon[35032]: Cluster is now healthy 2024-09-19T09:25:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:40 smithi142 ceph-mon[35032]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-09-19T09:25:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:40 smithi142 ceph-mon[35032]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-19T09:25:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:40 smithi142 ceph-mon[35032]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-19T09:25:41.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:40 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/230779551' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T09:25:41.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:40 smithi097 ceph-mon[34864]: pgmap v278: 97 pgs: 97 active+clean; 580 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T09:25:41.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:40 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] up:active 2024-09-19T09:25:41.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:40 smithi097 ceph-mon[34864]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T09:25:41.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:40 smithi097 ceph-mon[34864]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T09:25:41.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:40 smithi097 ceph-mon[34864]: Cluster is now healthy 2024-09-19T09:25:41.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:40 smithi097 ceph-mon[34864]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-09-19T09:25:41.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:40 smithi097 ceph-mon[34864]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-19T09:25:41.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:40 smithi097 ceph-mon[34864]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-19T09:25:41.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:40 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/230779551' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T09:25:41.596 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-19T09:25:41.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T09:25:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:42.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:41 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/230779551' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T09:25:42.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:41 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] up:active 2024-09-19T09:25:42.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:41 smithi016 ceph-mon[30979]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-19T09:25:42.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:41 smithi016 ceph-mon[30979]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:25:42.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-19T09:25:42.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-19T09:25:42.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-19T09:25:42.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-09-19T09:25:42.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-19T09:25:42.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-09-19T09:25:42.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-09-19T09:25:42.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-09-19T09:25:42.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:41 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-19T09:25:42.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.307+0000 7f317479c640 1 -- 172.21.15.16:0/1467812368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c101de0 msgr2=0x7f316c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:42.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.307+0000 7f317479c640 1 --2- 172.21.15.16:0/1467812368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c101de0 0x7f316c1021c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3168009900 tx=0x7f316802a410 comp rx=0 tx=0).stop 2024-09-19T09:25:42.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.307+0000 7f317479c640 1 -- 172.21.15.16:0/1467812368 shutdown_connections 2024-09-19T09:25:42.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.307+0000 7f317479c640 1 --2- 172.21.15.16:0/1467812368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f316c102790 0x7f316c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.307+0000 7f317479c640 1 --2- 172.21.15.16:0/1467812368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c106f10 0x7f316c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.307+0000 7f317479c640 1 --2- 172.21.15.16:0/1467812368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c101de0 0x7f316c1021c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.307+0000 7f317479c640 1 -- 172.21.15.16:0/1467812368 >> 172.21.15.16:0/1467812368 conn(0x7f316c0fd8c0 msgr2=0x7f316c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.307+0000 7f317479c640 1 -- 172.21.15.16:0/1467812368 shutdown_connections 2024-09-19T09:25:42.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.307+0000 7f317479c640 1 -- 172.21.15.16:0/1467812368 wait complete. 2024-09-19T09:25:42.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f317479c640 1 Processor -- start 2024-09-19T09:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f317479c640 1 -- start start 2024-09-19T09:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f317479c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f316c101de0 0x7f316c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f317479c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c102790 0x7f316c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f317479c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c106f10 0x7f316c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f317479c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f316c111fe0 con 0x7f316c102790 2024-09-19T09:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f317479c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f316c111e60 con 0x7f316c106f10 2024-09-19T09:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f317479c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f316c112160 con 0x7f316c101de0 2024-09-19T09:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f3171d37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c102790 0x7f316c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f3171d37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c102790 0x7f316c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35340/0 (socket says 172.21.15.16:35340) 2024-09-19T09:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f3171d37640 1 -- 172.21.15.16:0/4247032697 learned_addr learned my addr 172.21.15.16:0/4247032697 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f3172538640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f316c101de0 0x7f316c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:42.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f3172d39640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c106f10 0x7f316c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f3171d37640 1 -- 172.21.15.16:0/4247032697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f316c101de0 msgr2=0x7f316c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f3171d37640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f316c101de0 0x7f316c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f3171d37640 1 -- 172.21.15.16:0/4247032697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c106f10 msgr2=0x7f316c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f3171d37640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c106f10 0x7f316c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.308+0000 7f3171d37640 1 -- 172.21.15.16:0/4247032697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316c19d3f0 con 0x7f316c102790 2024-09-19T09:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.309+0000 7f3171d37640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c102790 0x7f316c198980 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f315c00d950 tx=0x7f315c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.309+0000 7f31637fe640 1 -- 172.21.15.16:0/4247032697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c017070 con 0x7f316c102790 2024-09-19T09:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.309+0000 7f31637fe640 1 -- 172.21.15.16:0/4247032697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f315c009c30 con 0x7f316c102790 2024-09-19T09:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.309+0000 7f317479c640 1 -- 172.21.15.16:0/4247032697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316c19d6e0 con 0x7f316c102790 2024-09-19T09:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.309+0000 7f31637fe640 1 -- 172.21.15.16:0/4247032697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c00f6b0 con 0x7f316c102790 2024-09-19T09:25:42.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.309+0000 7f317479c640 1 -- 172.21.15.16:0/4247032697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316c1a4fc0 con 0x7f316c102790 2024-09-19T09:25:42.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.310+0000 7f317479c640 1 -- 172.21.15.16:0/4247032697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f316c1021c0 con 0x7f316c102790 2024-09-19T09:25:42.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.311+0000 7f31637fe640 1 -- 172.21.15.16:0/4247032697 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f315c0109d0 con 0x7f316c102790 2024-09-19T09:25:42.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.312+0000 7f31637fe640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f31580a32c0 0x7f31580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:42.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.312+0000 7f3172538640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f31580a32c0 0x7f31580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:42.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.312+0000 7f31637fe640 1 -- 172.21.15.16:0/4247032697 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8240+0+0 (secure 0 0 0) 0x7f315c0cf850 con 0x7f316c102790 2024-09-19T09:25:42.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.312+0000 7f3172538640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f31580a32c0 0x7f31580a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3168000c00 tx=0x7f3168005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:42.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.316+0000 7f31637fe640 1 -- 172.21.15.16:0/4247032697 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f315c009de0 con 0x7f316c102790 2024-09-19T09:25:42.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:41 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/230779551' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T09:25:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:41 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] up:active 2024-09-19T09:25:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:41 smithi142 ceph-mon[35032]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-19T09:25:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:41 smithi142 ceph-mon[35032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:25:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-19T09:25:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-19T09:25:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-19T09:25:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-09-19T09:25:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-19T09:25:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-09-19T09:25:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-09-19T09:25:42.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-09-19T09:25:42.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:41 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-19T09:25:42.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:41 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/230779551' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T09:25:42.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:41 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] up:active 2024-09-19T09:25:42.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:41 smithi097 ceph-mon[34864]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-19T09:25:42.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:41 smithi097 ceph-mon[34864]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:25:42.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-19T09:25:42.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-19T09:25:42.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-19T09:25:42.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-09-19T09:25:42.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-19T09:25:42.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-09-19T09:25:42.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-09-19T09:25:42.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-09-19T09:25:42.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:41 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-19T09:25:42.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:42.514+0000 7f317479c640 1 -- 172.21.15.16:0/4247032697 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f316c062630 con 0x7f31580a32c0 2024-09-19T09:25:42.993 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:25:42.519+0000 7f4475dea640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:25:42.994 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:25:42.519+0000 7f4475dea640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:25:42.994 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:25:42.519+0000 7f4475dea640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:25:42.994 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:25:42.519+0000 7f4475dea640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:25:42.994 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:25:42.519+0000 7f4475dea640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T09:25:43.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:43.014+0000 7f31637fe640 1 -- 172.21.15.16:0/4247032697 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f316c062630 con 0x7f31580a32c0 2024-09-19T09:25:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:43.016+0000 7f317479c640 1 -- 172.21.15.16:0/4247032697 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f31580a32c0 msgr2=0x7f31580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:43.016+0000 7f317479c640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f31580a32c0 0x7f31580a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3168000c00 tx=0x7f3168005b50 comp rx=0 tx=0).stop 2024-09-19T09:25:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:43.016+0000 7f317479c640 1 -- 172.21.15.16:0/4247032697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c102790 msgr2=0x7f316c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:43.016+0000 7f317479c640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c102790 0x7f316c198980 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f315c00d950 tx=0x7f315c00de10 comp rx=0 tx=0).stop 2024-09-19T09:25:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:43.016+0000 7f317479c640 1 -- 172.21.15.16:0/4247032697 shutdown_connections 2024-09-19T09:25:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:43.016+0000 7f317479c640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f316c106f10 0x7f316c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:43.016+0000 7f317479c640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c102790 0x7f316c198980 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:43.016+0000 7f317479c640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f31580a32c0 0x7f31580a5780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:43.016+0000 7f317479c640 1 --2- 172.21.15.16:0/4247032697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f316c101de0 0x7f316c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:43.016+0000 7f317479c640 1 -- 172.21.15.16:0/4247032697 >> 172.21.15.16:0/4247032697 conn(0x7f316c0fd8c0 msgr2=0x7f316c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:43.016+0000 7f317479c640 1 -- 172.21.15.16:0/4247032697 shutdown_connections 2024-09-19T09:25:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:43.016+0000 7f317479c640 1 -- 172.21.15.16:0/4247032697 wait complete. 2024-09-19T09:25:43.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-mon[30979]: pgmap v279: 97 pgs: 97 active+clean; 580 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-19T09:25:43.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-19T09:25:43.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-19T09:25:43.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-19T09:25:43.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-09-19T09:25:43.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-19T09:25:43.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-09-19T09:25:43.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-09-19T09:25:43.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-19T09:25:43.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-19T09:25:43.254 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:42 smithi016 ceph-mon[30979]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T09:25:43.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:42 smithi142 ceph-mon[35032]: pgmap v279: 97 pgs: 97 active+clean; 580 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-19T09:25:43.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-19T09:25:43.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-19T09:25:43.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-19T09:25:43.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-09-19T09:25:43.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-19T09:25:43.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-09-19T09:25:43.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-09-19T09:25:43.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-19T09:25:43.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:42 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-19T09:25:43.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:42 smithi142 ceph-mon[35032]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T09:25:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:42 smithi097 ceph-mon[34864]: pgmap v279: 97 pgs: 97 active+clean; 580 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-19T09:25:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-19T09:25:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-19T09:25:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-19T09:25:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-09-19T09:25:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-19T09:25:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-09-19T09:25:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-09-19T09:25:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-19T09:25:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:42 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-19T09:25:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:42 smithi097 ceph-mon[34864]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T09:25:43.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T09:25:43.916 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:44.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:44 smithi016 ceph-mon[30979]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:44.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:44 smithi016 ceph-mon[30979]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T09:25:44.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:44 smithi142 ceph-mon[35032]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:44.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:44 smithi142 ceph-mon[35032]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T09:25:44.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:44 smithi097 ceph-mon[34864]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:44.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:44 smithi097 ceph-mon[34864]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T09:25:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.387+0000 7f18f1a67640 1 -- 172.21.15.16:0/1935120964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec101de0 msgr2=0x7f18ec1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.387+0000 7f18f1a67640 1 --2- 172.21.15.16:0/1935120964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec101de0 0x7f18ec1021c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009900 tx=0x7f18dc02a410 comp rx=0 tx=0).stop 2024-09-19T09:25:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.387+0000 7f18f1a67640 1 -- 172.21.15.16:0/1935120964 shutdown_connections 2024-09-19T09:25:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.387+0000 7f18f1a67640 1 --2- 172.21.15.16:0/1935120964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18ec106f10 0x7f18ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.387+0000 7f18f1a67640 1 --2- 172.21.15.16:0/1935120964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ec102790 0x7f18ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.387+0000 7f18f1a67640 1 --2- 172.21.15.16:0/1935120964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec101de0 0x7f18ec1021c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.387+0000 7f18f1a67640 1 -- 172.21.15.16:0/1935120964 >> 172.21.15.16:0/1935120964 conn(0x7f18ec0fd8c0 msgr2=0x7f18ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.387+0000 7f18f1a67640 1 -- 172.21.15.16:0/1935120964 shutdown_connections 2024-09-19T09:25:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.388+0000 7f18f1a67640 1 -- 172.21.15.16:0/1935120964 wait complete. 2024-09-19T09:25:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.388+0000 7f18f1a67640 1 Processor -- start 2024-09-19T09:25:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.388+0000 7f18f1a67640 1 -- start start 2024-09-19T09:25:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18f1a67640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18ec101de0 0x7f18ec198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18f1a67640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ec102790 0x7f18ec1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18f1a67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec106f10 0x7f18ec19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18f1a67640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec111f90 con 0x7f18ec106f10 2024-09-19T09:25:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18f1a67640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec111e10 con 0x7f18ec102790 2024-09-19T09:25:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18f1a67640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec112110 con 0x7f18ec101de0 2024-09-19T09:25:44.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec106f10 0x7f18ec19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18eb7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18ec101de0 0x7f18ec198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18eb7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18ec101de0 0x7f18ec198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:45758/0 (socket says 172.21.15.16:45758) 2024-09-19T09:25:44.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18eb7fe640 1 -- 172.21.15.16:0/1548295766 learned_addr learned my addr 172.21.15.16:0/1548295766 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:44.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec106f10 0x7f18ec19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35372/0 (socket says 172.21.15.16:35372) 2024-09-19T09:25:44.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18eaffd640 1 --2- 172.21.15.16:0/1548295766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ec102790 0x7f18ec1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18ebfff640 1 -- 172.21.15.16:0/1548295766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18ec101de0 msgr2=0x7f18ec198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18ebfff640 1 --2- 172.21.15.16:0/1548295766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18ec101de0 0x7f18ec198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18ebfff640 1 -- 172.21.15.16:0/1548295766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ec102790 msgr2=0x7f18ec1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18ebfff640 1 --2- 172.21.15.16:0/1548295766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ec102790 0x7f18ec1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18ebfff640 1 -- 172.21.15.16:0/1548295766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec19d440 con 0x7f18ec106f10 2024-09-19T09:25:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.389+0000 7f18ebfff640 1 --2- 172.21.15.16:0/1548295766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec106f10 0x7f18ec19cd60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f18e400be20 tx=0x7f18e400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.390+0000 7f18e8ff9640 1 -- 172.21.15.16:0/1548295766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4019070 con 0x7f18ec106f10 2024-09-19T09:25:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.390+0000 7f18e8ff9640 1 -- 172.21.15.16:0/1548295766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e4014920 con 0x7f18ec106f10 2024-09-19T09:25:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.390+0000 7f18f1a67640 1 -- 172.21.15.16:0/1548295766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec19d730 con 0x7f18ec106f10 2024-09-19T09:25:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.390+0000 7f18e8ff9640 1 -- 172.21.15.16:0/1548295766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4012920 con 0x7f18ec106f10 2024-09-19T09:25:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.390+0000 7f18f1a67640 1 -- 172.21.15.16:0/1548295766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec1a5010 con 0x7f18ec106f10 2024-09-19T09:25:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.391+0000 7f18f1a67640 1 -- 172.21.15.16:0/1548295766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ec1021c0 con 0x7f18ec106f10 2024-09-19T09:25:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.392+0000 7f18e8ff9640 1 -- 172.21.15.16:0/1548295766 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f18e401c070 con 0x7f18ec106f10 2024-09-19T09:25:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.393+0000 7f18e8ff9640 1 --2- 172.21.15.16:0/1548295766 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f18d40a3350 0x7f18d40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.393+0000 7f18eb7fe640 1 --2- 172.21.15.16:0/1548295766 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f18d40a3350 0x7f18d40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.393+0000 7f18e8ff9640 1 -- 172.21.15.16:0/1548295766 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f18e40cf1a0 con 0x7f18ec106f10 2024-09-19T09:25:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.393+0000 7f18eb7fe640 1 --2- 172.21.15.16:0/1548295766 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f18d40a3350 0x7f18d40a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f18dc0096c0 tx=0x7f18dc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.397+0000 7f18e8ff9640 1 -- 172.21.15.16:0/1548295766 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f18e409b820 con 0x7f18ec106f10 2024-09-19T09:25:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.598+0000 7f18f1a67640 1 -- 172.21.15.16:0/1548295766 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f18ec062630 con 0x7f18d40a3350 2024-09-19T09:25:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.624+0000 7f18e8ff9640 1 -- 172.21.15.16:0/1548295766 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f18ec062630 con 0x7f18d40a3350 2024-09-19T09:25:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.626+0000 7f18f1a67640 1 -- 172.21.15.16:0/1548295766 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f18d40a3350 msgr2=0x7f18d40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.626+0000 7f18f1a67640 1 --2- 172.21.15.16:0/1548295766 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f18d40a3350 0x7f18d40a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f18dc0096c0 tx=0x7f18dc005a20 comp rx=0 tx=0).stop 2024-09-19T09:25:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.626+0000 7f18f1a67640 1 -- 172.21.15.16:0/1548295766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec106f10 msgr2=0x7f18ec19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.626+0000 7f18f1a67640 1 --2- 172.21.15.16:0/1548295766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec106f10 0x7f18ec19cd60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f18e400be20 tx=0x7f18e400bf10 comp rx=0 tx=0).stop 2024-09-19T09:25:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.627+0000 7f18f1a67640 1 -- 172.21.15.16:0/1548295766 shutdown_connections 2024-09-19T09:25:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.627+0000 7f18f1a67640 1 --2- 172.21.15.16:0/1548295766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec106f10 0x7f18ec19cd60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.627+0000 7f18f1a67640 1 --2- 172.21.15.16:0/1548295766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18ec102790 0x7f18ec1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.627+0000 7f18f1a67640 1 --2- 172.21.15.16:0/1548295766 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f18d40a3350 0x7f18d40a5810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.627+0000 7f18f1a67640 1 --2- 172.21.15.16:0/1548295766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18ec101de0 0x7f18ec198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.627+0000 7f18f1a67640 1 -- 172.21.15.16:0/1548295766 >> 172.21.15.16:0/1548295766 conn(0x7f18ec0fd8c0 msgr2=0x7f18ec0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.627+0000 7f18f1a67640 1 -- 172.21.15.16:0/1548295766 shutdown_connections 2024-09-19T09:25:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:44.627+0000 7f18f1a67640 1 -- 172.21.15.16:0/1548295766 wait complete. 2024-09-19T09:25:45.198 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:45 smithi016 ceph-mon[30979]: pgmap v282: 97 pgs: 5 peering, 92 active+clean; 585 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.1 KiB/s wr, 27 op/s 2024-09-19T09:25:45.198 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:45 smithi016 ceph-mon[30979]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T09:25:45.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T09:25:45.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:45 smithi142 ceph-mon[35032]: pgmap v282: 97 pgs: 5 peering, 92 active+clean; 585 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.1 KiB/s wr, 27 op/s 2024-09-19T09:25:45.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:45 smithi142 ceph-mon[35032]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T09:25:45.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:45 smithi097 ceph-mon[34864]: pgmap v282: 97 pgs: 5 peering, 92 active+clean; 585 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.1 KiB/s wr, 27 op/s 2024-09-19T09:25:45.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:45 smithi097 ceph-mon[34864]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T09:25:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:45.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.983+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/3438848196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44103660 msgr2=0x7f6c4410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.983+0000 7f6c4b3bf640 1 --2- 172.21.15.16:0/3438848196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44103660 0x7f6c4410bb50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009900 tx=0x7f6c3402a480 comp rx=0 tx=0).stop 2024-09-19T09:25:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.983+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/3438848196 shutdown_connections 2024-09-19T09:25:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.983+0000 7f6c4b3bf640 1 --2- 172.21.15.16:0/3438848196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4410c170 0x7f6c4410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.983+0000 7f6c4b3bf640 1 --2- 172.21.15.16:0/3438848196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44103660 0x7f6c4410bb50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.983+0000 7f6c4b3bf640 1 --2- 172.21.15.16:0/3438848196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c44102d40 0x7f6c44103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.983+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/3438848196 >> 172.21.15.16:0/3438848196 conn(0x7f6c440fc7a0 msgr2=0x7f6c440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.983+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/3438848196 shutdown_connections 2024-09-19T09:25:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.983+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/3438848196 wait complete. 2024-09-19T09:25:46.831 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:46 smithi097 ceph-mon[34864]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:46.832 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:46 smithi142 ceph-mon[35032]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.983+0000 7f6c4b3bf640 1 Processor -- start 2024-09-19T09:25:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4b3bf640 1 -- start start 2024-09-19T09:25:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4b3bf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c44102d40 0x7f6c441009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4b3bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44103660 0x7f6c44100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4b3bf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4410c170 0x7f6c44101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4b3bf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44111f70 con 0x7f6c44103660 2024-09-19T09:25:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4b3bf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44111df0 con 0x7f6c4410c170 2024-09-19T09:25:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4b3bf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441120f0 con 0x7f6c44102d40 2024-09-19T09:25:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4895a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44103660 0x7f6c44100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4895a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44103660 0x7f6c44100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44146/0 (socket says 172.21.15.16:44146) 2024-09-19T09:25:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4895a640 1 -- 172.21.15.16:0/1780589336 learned_addr learned my addr 172.21.15.16:0/1780589336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4915b640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c44102d40 0x7f6c441009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4995c640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4410c170 0x7f6c44101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4895a640 1 -- 172.21.15.16:0/1780589336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c44102d40 msgr2=0x7f6c441009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4895a640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c44102d40 0x7f6c441009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4895a640 1 -- 172.21.15.16:0/1780589336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4410c170 msgr2=0x7f6c44101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4895a640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4410c170 0x7f6c44101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.984+0000 7f6c4895a640 1 -- 172.21.15.16:0/1780589336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c441a6930 con 0x7f6c44103660 2024-09-19T09:25:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.985+0000 7f6c4895a640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44103660 0x7f6c44100ee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c34006010 tx=0x7f6c340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.985+0000 7f6c3a7fc640 1 -- 172.21.15.16:0/1780589336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3403f030 con 0x7f6c44103660 2024-09-19T09:25:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.985+0000 7f6c3a7fc640 1 -- 172.21.15.16:0/1780589336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c34003ce0 con 0x7f6c44103660 2024-09-19T09:25:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.985+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/1780589336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c441a6bc0 con 0x7f6c44103660 2024-09-19T09:25:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.985+0000 7f6c3a7fc640 1 -- 172.21.15.16:0/1780589336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34035070 con 0x7f6c44103660 2024-09-19T09:25:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.985+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/1780589336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c441a70f0 con 0x7f6c44103660 2024-09-19T09:25:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.986+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/1780589336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c44103120 con 0x7f6c44103660 2024-09-19T09:25:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.987+0000 7f6c3a7fc640 1 -- 172.21.15.16:0/1780589336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6c34042030 con 0x7f6c44103660 2024-09-19T09:25:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.988+0000 7f6c3a7fc640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6c300a3350 0x7f6c300a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.988+0000 7f6c4915b640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6c300a3350 0x7f6c300a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.988+0000 7f6c3a7fc640 1 -- 172.21.15.16:0/1780589336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6c340eff30 con 0x7f6c44103660 2024-09-19T09:25:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.988+0000 7f6c4915b640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6c300a3350 0x7f6c300a5810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6c40005fd0 tx=0x7f6c4000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:45.992+0000 7f6c3a7fc640 1 -- 172.21.15.16:0/1780589336 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6c340bc560 con 0x7f6c44103660 2024-09-19T09:25:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.195+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/1780589336 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f6c44107250 con 0x7f6c300a3350 2024-09-19T09:25:46.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:46 smithi016 ceph-mon[30979]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.200+0000 7f6c3a7fc640 1 -- 172.21.15.16:0/1780589336 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f6c40003890 con 0x7f6c300a3350 2024-09-19T09:25:46.839 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_0/d2017c54-eaa5-4013-8d36-71b84849a23b 2024-09-19T09:25:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.202+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/1780589336 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6c300a3350 msgr2=0x7f6c300a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.202+0000 7f6c4b3bf640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6c300a3350 0x7f6c300a5810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6c40005fd0 tx=0x7f6c4000b390 comp rx=0 tx=0).stop 2024-09-19T09:25:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.202+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/1780589336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44103660 msgr2=0x7f6c44100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.202+0000 7f6c4b3bf640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44103660 0x7f6c44100ee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c34006010 tx=0x7f6c340042f0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.202+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/1780589336 shutdown_connections 2024-09-19T09:25:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.202+0000 7f6c4b3bf640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4410c170 0x7f6c44101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.202+0000 7f6c4b3bf640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44103660 0x7f6c44100ee0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.202+0000 7f6c4b3bf640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6c300a3350 0x7f6c300a5810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.202+0000 7f6c4b3bf640 1 --2- 172.21.15.16:0/1780589336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c44102d40 0x7f6c441009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:46.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.202+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/1780589336 >> 172.21.15.16:0/1780589336 conn(0x7f6c440fc7a0 msgr2=0x7f6c440fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.203+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/1780589336 shutdown_connections 2024-09-19T09:25:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:46.203+0000 7f6c4b3bf640 1 -- 172.21.15.16:0/1780589336 wait complete. 2024-09-19T09:25:46.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T09:25:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:47.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:47 smithi016 ceph-mon[30979]: pgmap v283: 97 pgs: 5 peering, 92 active+clean; 585 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.1 KiB/s wr, 27 op/s 2024-09-19T09:25:47.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:47 smithi016 ceph-mon[30979]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:47.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:47 smithi142 ceph-mon[35032]: pgmap v283: 97 pgs: 5 peering, 92 active+clean; 585 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.1 KiB/s wr, 27 op/s 2024-09-19T09:25:47.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:47 smithi142 ceph-mon[35032]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:47.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:47 smithi097 ceph-mon[34864]: pgmap v283: 97 pgs: 5 peering, 92 active+clean; 585 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.1 KiB/s wr, 27 op/s 2024-09-19T09:25:47.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:47 smithi097 ceph-mon[34864]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:47.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.654+0000 7f045754c640 1 -- 172.21.15.16:0/3186923031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0450100e20 msgr2=0x7f0450101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:47.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.654+0000 7f045754c640 1 --2- 172.21.15.16:0/3186923031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0450100e20 0x7f0450101200 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f044c009900 tx=0x7f044c02a480 comp rx=0 tx=0).stop 2024-09-19T09:25:47.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.654+0000 7f045754c640 1 -- 172.21.15.16:0/3186923031 shutdown_connections 2024-09-19T09:25:47.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.654+0000 7f045754c640 1 --2- 172.21.15.16:0/3186923031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0450102ef0 0x7f045010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.654+0000 7f045754c640 1 --2- 172.21.15.16:0/3186923031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0450102570 0x7f04501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.654+0000 7f045754c640 1 --2- 172.21.15.16:0/3186923031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0450100e20 0x7f0450101200 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.654+0000 7f045754c640 1 -- 172.21.15.16:0/3186923031 >> 172.21.15.16:0/3186923031 conn(0x7f04500fc7d0 msgr2=0x7f04500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:47.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.655+0000 7f045754c640 1 -- 172.21.15.16:0/3186923031 shutdown_connections 2024-09-19T09:25:47.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.655+0000 7f045754c640 1 -- 172.21.15.16:0/3186923031 wait complete. 2024-09-19T09:25:47.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.655+0000 7f045754c640 1 Processor -- start 2024-09-19T09:25:47.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.655+0000 7f045754c640 1 -- start start 2024-09-19T09:25:47.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.655+0000 7f045754c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0450100e20 0x7f0450103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f045754c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0450102570 0x7f0450103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f045754c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0450102ef0 0x7f04501041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f045754c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0450110f80 con 0x7f0450102570 2024-09-19T09:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f045754c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0450110e00 con 0x7f0450100e20 2024-09-19T09:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f045754c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0450111100 con 0x7f0450102ef0 2024-09-19T09:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f0454ae7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0450102570 0x7f0450103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f0454ae7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0450102570 0x7f0450103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44172/0 (socket says 172.21.15.16:44172) 2024-09-19T09:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f0454ae7640 1 -- 172.21.15.16:0/773716059 learned_addr learned my addr 172.21.15.16:0/773716059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f04552e8640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0450100e20 0x7f0450103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f0455ae9640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0450102ef0 0x7f04501041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f0454ae7640 1 -- 172.21.15.16:0/773716059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0450102ef0 msgr2=0x7f04501041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f0454ae7640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0450102ef0 0x7f04501041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f0454ae7640 1 -- 172.21.15.16:0/773716059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0450100e20 msgr2=0x7f0450103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f0454ae7640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0450100e20 0x7f0450103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f0454ae7640 1 -- 172.21.15.16:0/773716059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04501a6a10 con 0x7f0450102570 2024-09-19T09:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f0455ae9640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0450102ef0 0x7f04501041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f0454ae7640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0450102570 0x7f0450103c90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f044000d950 tx=0x7f044000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f04467fc640 1 -- 172.21.15.16:0/773716059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440017070 con 0x7f0450102570 2024-09-19T09:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f04467fc640 1 -- 172.21.15.16:0/773716059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0440009c30 con 0x7f0450102570 2024-09-19T09:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f045754c640 1 -- 172.21.15.16:0/773716059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04501a6d00 con 0x7f0450102570 2024-09-19T09:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f04467fc640 1 -- 172.21.15.16:0/773716059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044000f6b0 con 0x7f0450102570 2024-09-19T09:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.656+0000 7f045754c640 1 -- 172.21.15.16:0/773716059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04501a7230 con 0x7f0450102570 2024-09-19T09:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.657+0000 7f045754c640 1 -- 172.21.15.16:0/773716059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0450101200 con 0x7f0450102570 2024-09-19T09:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.659+0000 7f04467fc640 1 -- 172.21.15.16:0/773716059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f04400109d0 con 0x7f0450102570 2024-09-19T09:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.659+0000 7f04467fc640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f043c0a3350 0x7f043c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.660+0000 7f04552e8640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f043c0a3350 0x7f043c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.660+0000 7f04467fc640 1 -- 172.21.15.16:0/773716059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f04400cf630 con 0x7f0450102570 2024-09-19T09:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.660+0000 7f04552e8640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f043c0a3350 0x7f043c0a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f044c009230 tx=0x7f044c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.664+0000 7f04467fc640 1 -- 172.21.15.16:0/773716059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0440009de0 con 0x7f0450102570 2024-09-19T09:25:47.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.862+0000 7f045754c640 1 -- 172.21.15.16:0/773716059 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f04501068a0 con 0x7f043c0a3350 2024-09-19T09:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.889+0000 7f04467fc640 1 -- 172.21.15.16:0/773716059 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f04501068a0 con 0x7f043c0a3350 2024-09-19T09:25:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.891+0000 7f045754c640 1 -- 172.21.15.16:0/773716059 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f043c0a3350 msgr2=0x7f043c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.891+0000 7f045754c640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f043c0a3350 0x7f043c0a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f044c009230 tx=0x7f044c0058d0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.891+0000 7f045754c640 1 -- 172.21.15.16:0/773716059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0450102570 msgr2=0x7f0450103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.891+0000 7f045754c640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0450102570 0x7f0450103c90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f044000d950 tx=0x7f044000de10 comp rx=0 tx=0).stop 2024-09-19T09:25:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.891+0000 7f045754c640 1 -- 172.21.15.16:0/773716059 shutdown_connections 2024-09-19T09:25:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.891+0000 7f045754c640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f043c0a3350 0x7f043c0a5810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.891+0000 7f045754c640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0450102ef0 0x7f04501041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.891+0000 7f045754c640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0450102570 0x7f0450103c90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.891+0000 7f045754c640 1 --2- 172.21.15.16:0/773716059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0450100e20 0x7f0450103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.891+0000 7f045754c640 1 -- 172.21.15.16:0/773716059 >> 172.21.15.16:0/773716059 conn(0x7f04500fc7d0 msgr2=0x7f045006a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.891+0000 7f045754c640 1 -- 172.21.15.16:0/773716059 shutdown_connections 2024-09-19T09:25:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:47.891+0000 7f045754c640 1 -- 172.21.15.16:0/773716059 wait complete. 2024-09-19T09:25:48.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T09:25:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:49.152 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:49 smithi016 ceph-mon[30979]: pgmap v284: 97 pgs: 3 peering, 94 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 32 op/s; 146 B/s, 0 objects/s recovering 2024-09-19T09:25:49.152 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:49 smithi016 ceph-mon[30979]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.239+0000 7f0178e7f640 1 -- 172.21.15.16:0/736087875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174106a80 msgr2=0x7f0174106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.239+0000 7f0178e7f640 1 --2- 172.21.15.16:0/736087875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174106a80 0x7f0174106e60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0164009930 tx=0x7f016402a420 comp rx=0 tx=0).stop 2024-09-19T09:25:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.239+0000 7f0178e7f640 1 -- 172.21.15.16:0/736087875 shutdown_connections 2024-09-19T09:25:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.239+0000 7f0178e7f640 1 --2- 172.21.15.16:0/736087875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0174101420 0x7f017410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.239+0000 7f0178e7f640 1 --2- 172.21.15.16:0/736087875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0174100a80 0x7f0174100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.239+0000 7f0178e7f640 1 --2- 172.21.15.16:0/736087875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174106a80 0x7f0174106e60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.239+0000 7f0178e7f640 1 -- 172.21.15.16:0/736087875 >> 172.21.15.16:0/736087875 conn(0x7f01740fc7a0 msgr2=0x7f01740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.239+0000 7f0178e7f640 1 -- 172.21.15.16:0/736087875 shutdown_connections 2024-09-19T09:25:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.239+0000 7f0178e7f640 1 -- 172.21.15.16:0/736087875 wait complete. 2024-09-19T09:25:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.240+0000 7f0178e7f640 1 Processor -- start 2024-09-19T09:25:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.240+0000 7f0178e7f640 1 -- start start 2024-09-19T09:25:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.240+0000 7f0178e7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0174100a80 0x7f0174198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.240+0000 7f0178e7f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0174101420 0x7f0174198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.240+0000 7f0178e7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174106a80 0x7f017419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.240+0000 7f0178e7f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0174110ea0 con 0x7f0174106a80 2024-09-19T09:25:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.240+0000 7f0178e7f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0174110d20 con 0x7f0174100a80 2024-09-19T09:25:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.240+0000 7f0178e7f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0174111020 con 0x7f0174101420 2024-09-19T09:25:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f0172d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174106a80 0x7f017419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f0172d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174106a80 0x7f017419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44196/0 (socket says 172.21.15.16:44196) 2024-09-19T09:25:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f0172d9d640 1 -- 172.21.15.16:0/1425918600 learned_addr learned my addr 172.21.15.16:0/1425918600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f0171d9b640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0174101420 0x7f0174198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f017259c640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0174100a80 0x7f0174198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f0172d9d640 1 -- 172.21.15.16:0/1425918600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0174101420 msgr2=0x7f0174198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f0172d9d640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0174101420 0x7f0174198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f0172d9d640 1 -- 172.21.15.16:0/1425918600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0174100a80 msgr2=0x7f0174198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f0172d9d640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0174100a80 0x7f0174198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f0172d9d640 1 -- 172.21.15.16:0/1425918600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017419d3e0 con 0x7f0174106a80 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f0172d9d640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174106a80 0x7f017419cd00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f016c00b970 tx=0x7f016c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f016b7fe640 1 -- 172.21.15.16:0/1425918600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c019070 con 0x7f0174106a80 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f016b7fe640 1 -- 172.21.15.16:0/1425918600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f016c014440 con 0x7f0174106a80 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f0178e7f640 1 -- 172.21.15.16:0/1425918600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017419d6d0 con 0x7f0174106a80 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f016b7fe640 1 -- 172.21.15.16:0/1425918600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c0075d0 con 0x7f0174106a80 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.241+0000 7f0178e7f640 1 -- 172.21.15.16:0/1425918600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01741a4fb0 con 0x7f0174106a80 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.242+0000 7f0178e7f640 1 -- 172.21.15.16:0/1425918600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0174100ee0 con 0x7f0174106a80 2024-09-19T09:25:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.243+0000 7f016b7fe640 1 -- 172.21.15.16:0/1425918600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f016c01c070 con 0x7f0174106a80 2024-09-19T09:25:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.244+0000 7f016b7fe640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f01600a32d0 0x7f01600a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.244+0000 7f017259c640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f01600a32d0 0x7f01600a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.245+0000 7f016b7fe640 1 -- 172.21.15.16:0/1425918600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f016c0cef50 con 0x7f0174106a80 2024-09-19T09:25:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.245+0000 7f017259c640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f01600a32d0 0x7f01600a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0164009900 tx=0x7f01640058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.249+0000 7f016b7fe640 1 -- 172.21.15.16:0/1425918600 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f016c09b590 con 0x7f0174106a80 2024-09-19T09:25:49.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:49 smithi142 ceph-mon[35032]: pgmap v284: 97 pgs: 3 peering, 94 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 32 op/s; 146 B/s, 0 objects/s recovering 2024-09-19T09:25:49.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:49 smithi142 ceph-mon[35032]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:49.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:49 smithi097 ceph-mon[34864]: pgmap v284: 97 pgs: 3 peering, 94 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 32 op/s; 146 B/s, 0 objects/s recovering 2024-09-19T09:25:49.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:49 smithi097 ceph-mon[34864]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:49.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.455+0000 7f0178e7f640 1 -- 172.21.15.16:0/1425918600 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f0174062630 con 0x7f01600a32d0 2024-09-19T09:25:49.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.461+0000 7f016b7fe640 1 -- 172.21.15.16:0/1425918600 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0174062630 con 0x7f01600a32d0 2024-09-19T09:25:49.462 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_1/fb0e7bdc-edce-464b-b5ca-12d056277b39 2024-09-19T09:25:49.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.463+0000 7f0178e7f640 1 -- 172.21.15.16:0/1425918600 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f01600a32d0 msgr2=0x7f01600a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:49.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.463+0000 7f0178e7f640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f01600a32d0 0x7f01600a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0164009900 tx=0x7f01640058d0 comp rx=0 tx=0).stop 2024-09-19T09:25:49.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.463+0000 7f0178e7f640 1 -- 172.21.15.16:0/1425918600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174106a80 msgr2=0x7f017419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:49.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.463+0000 7f0178e7f640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174106a80 0x7f017419cd00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f016c00b970 tx=0x7f016c00bd70 comp rx=0 tx=0).stop 2024-09-19T09:25:49.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.463+0000 7f0178e7f640 1 -- 172.21.15.16:0/1425918600 shutdown_connections 2024-09-19T09:25:49.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.463+0000 7f0178e7f640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f01600a32d0 0x7f01600a5790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:49.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.463+0000 7f0178e7f640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0174101420 0x7f0174198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:49.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.463+0000 7f0178e7f640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174106a80 0x7f017419cd00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:49.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.463+0000 7f0178e7f640 1 --2- 172.21.15.16:0/1425918600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0174100a80 0x7f0174198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:49.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.463+0000 7f0178e7f640 1 -- 172.21.15.16:0/1425918600 >> 172.21.15.16:0/1425918600 conn(0x7f01740fc7a0 msgr2=0x7f017406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:49.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.463+0000 7f0178e7f640 1 -- 172.21.15.16:0/1425918600 shutdown_connections 2024-09-19T09:25:49.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:49.463+0000 7f0178e7f640 1 -- 172.21.15.16:0/1425918600 wait complete. 2024-09-19T09:25:50.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:25:50.335 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:50.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.773+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/934154371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb8101df0 msgr2=0x7f0cb81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:50.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.773+0000 7f0cbd8e6640 1 --2- 172.21.15.16:0/934154371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb8101df0 0x7f0cb81021d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009ca0 tx=0x7f0ca802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:25:50.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.773+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/934154371 shutdown_connections 2024-09-19T09:25:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.773+0000 7f0cbd8e6640 1 --2- 172.21.15.16:0/934154371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cb8106f20 0x7f0cb810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.773+0000 7f0cbd8e6640 1 --2- 172.21.15.16:0/934154371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb81027a0 0x7f0cb81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.773+0000 7f0cbd8e6640 1 --2- 172.21.15.16:0/934154371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb8101df0 0x7f0cb81021d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.773+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/934154371 >> 172.21.15.16:0/934154371 conn(0x7f0cb80fd8b0 msgr2=0x7f0cb80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.773+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/934154371 shutdown_connections 2024-09-19T09:25:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.773+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/934154371 wait complete. 2024-09-19T09:25:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.774+0000 7f0cbd8e6640 1 Processor -- start 2024-09-19T09:25:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.774+0000 7f0cbd8e6640 1 -- start start 2024-09-19T09:25:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.774+0000 7f0cbd8e6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb8101df0 0x7f0cb80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.774+0000 7f0cbd8e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81027a0 0x7f0cb8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.774+0000 7f0cbd8e6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cb8106f20 0x7f0cb8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cbd8e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb8111fd0 con 0x7f0cb81027a0 2024-09-19T09:25:50.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cbd8e6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb8111e50 con 0x7f0cb8101df0 2024-09-19T09:25:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cbd8e6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb8112150 con 0x7f0cb8106f20 2024-09-19T09:25:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cb67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81027a0 0x7f0cb8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cb67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81027a0 0x7f0cb8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44206/0 (socket says 172.21.15.16:44206) 2024-09-19T09:25:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cb67fc640 1 -- 172.21.15.16:0/4226554827 learned_addr learned my addr 172.21.15.16:0/4226554827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cb6ffd640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb8101df0 0x7f0cb80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cb77fe640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cb8106f20 0x7f0cb8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cb67fc640 1 -- 172.21.15.16:0/4226554827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cb8106f20 msgr2=0x7f0cb8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cb67fc640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cb8106f20 0x7f0cb8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cb67fc640 1 -- 172.21.15.16:0/4226554827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb8101df0 msgr2=0x7f0cb80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cb67fc640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb8101df0 0x7f0cb80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cb67fc640 1 -- 172.21.15.16:0/4226554827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb81a68b0 con 0x7f0cb81027a0 2024-09-19T09:25:50.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cb67fc640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81027a0 0x7f0cb8074a30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00d950 tx=0x7f0cac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cbc8e4640 1 -- 172.21.15.16:0/4226554827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac017070 con 0x7f0cb81027a0 2024-09-19T09:25:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cbc8e4640 1 -- 172.21.15.16:0/4226554827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cac009c30 con 0x7f0cb81027a0 2024-09-19T09:25:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/4226554827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb81a6ba0 con 0x7f0cb81027a0 2024-09-19T09:25:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cbc8e4640 1 -- 172.21.15.16:0/4226554827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00f6b0 con 0x7f0cb81027a0 2024-09-19T09:25:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.775+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/4226554827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb81a7130 con 0x7f0cb81027a0 2024-09-19T09:25:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.776+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/4226554827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb81021d0 con 0x7f0cb81027a0 2024-09-19T09:25:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.778+0000 7f0cbc8e4640 1 -- 172.21.15.16:0/4226554827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0cac0109d0 con 0x7f0cb81027a0 2024-09-19T09:25:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.778+0000 7f0cbc8e4640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0ca00a3350 0x7f0ca00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.778+0000 7f0cb6ffd640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0ca00a3350 0x7f0ca00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.778+0000 7f0cbc8e4640 1 -- 172.21.15.16:0/4226554827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0cac0cf560 con 0x7f0cb81027a0 2024-09-19T09:25:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.779+0000 7f0cb6ffd640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0ca00a3350 0x7f0ca00a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8006010 tx=0x7f0ca8009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:50.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.781+0000 7f0cbc8e4640 1 -- 172.21.15.16:0/4226554827 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0cac009de0 con 0x7f0cb81027a0 2024-09-19T09:25:50.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.995+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/4226554827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cb810a100 con 0x7f0cb81027a0 2024-09-19T09:25:50.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.996+0000 7f0cbc8e4640 1 -- 172.21.15.16:0/4226554827 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11967 (secure 0 0 0) 0x7f0cac09bb50 con 0x7f0cb81027a0 2024-09-19T09:25:50.998 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:50.998 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"btime":"2024-09-19T09:25:40:972080+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:25:40.972058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24484,24470,24428,24424]},"id":1}]} 2024-09-19T09:25:50.998 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-19T09:25:50.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.998+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/4226554827 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0ca00a3350 msgr2=0x7f0ca00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:50.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.998+0000 7f0cbd8e6640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0ca00a3350 0x7f0ca00a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8006010 tx=0x7f0ca8009820 comp rx=0 tx=0).stop 2024-09-19T09:25:50.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.998+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/4226554827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81027a0 msgr2=0x7f0cb8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:50.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.998+0000 7f0cbd8e6640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81027a0 0x7f0cb8074a30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00d950 tx=0x7f0cac00de10 comp rx=0 tx=0).stop 2024-09-19T09:25:50.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.998+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/4226554827 shutdown_connections 2024-09-19T09:25:50.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.998+0000 7f0cbd8e6640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0ca00a3350 0x7f0ca00a5810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:50.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.998+0000 7f0cbd8e6640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cb8106f20 0x7f0cb8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:50.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.998+0000 7f0cbd8e6640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81027a0 0x7f0cb8074a30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.998+0000 7f0cbd8e6640 1 --2- 172.21.15.16:0/4226554827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb8101df0 0x7f0cb80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.998+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/4226554827 >> 172.21.15.16:0/4226554827 conn(0x7f0cb80fd8b0 msgr2=0x7f0cb80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.998+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/4226554827 shutdown_connections 2024-09-19T09:25:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:50.998+0000 7f0cbd8e6640 1 -- 172.21.15.16:0/4226554827 wait complete. 2024-09-19T09:25:51.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:51 smithi016 ceph-mon[30979]: pgmap v285: 97 pgs: 97 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 34 op/s; 230 B/s, 1 objects/s recovering 2024-09-19T09:25:51.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:51 smithi016 ceph-mon[30979]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:51.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:51 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/4226554827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:51.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:51 smithi142 ceph-mon[35032]: pgmap v285: 97 pgs: 97 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 34 op/s; 230 B/s, 1 objects/s recovering 2024-09-19T09:25:51.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:51 smithi142 ceph-mon[35032]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:51.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:51 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/4226554827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:51.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:51 smithi097 ceph-mon[34864]: pgmap v285: 97 pgs: 97 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 34 op/s; 230 B/s, 1 objects/s recovering 2024-09-19T09:25:51.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:51 smithi097 ceph-mon[34864]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:25:51.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:51 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/4226554827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:25:51.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:25:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:52.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.424+0000 7fdd5be85640 1 -- 172.21.15.16:0/3919169308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd54100b20 msgr2=0x7fdd54100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:52.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.424+0000 7fdd5be85640 1 --2- 172.21.15.16:0/3919169308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd54100b20 0x7fdd54100f00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009900 tx=0x7fdd4c02a410 comp rx=0 tx=0).stop 2024-09-19T09:25:52.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.424+0000 7fdd5be85640 1 -- 172.21.15.16:0/3919169308 shutdown_connections 2024-09-19T09:25:52.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.424+0000 7fdd5be85640 1 --2- 172.21.15.16:0/3919169308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd5410e500 0x7fdd54110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:52.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.424+0000 7fdd5be85640 1 --2- 172.21.15.16:0/3919169308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd54101440 0x7fdd5410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:52.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.424+0000 7fdd5be85640 1 --2- 172.21.15.16:0/3919169308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd54100b20 0x7fdd54100f00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:52.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.424+0000 7fdd5be85640 1 -- 172.21.15.16:0/3919169308 >> 172.21.15.16:0/3919169308 conn(0x7fdd540fc7d0 msgr2=0x7fdd540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:52.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.424+0000 7fdd5be85640 1 -- 172.21.15.16:0/3919169308 shutdown_connections 2024-09-19T09:25:52.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.424+0000 7fdd5be85640 1 -- 172.21.15.16:0/3919169308 wait complete. 2024-09-19T09:25:52.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd5be85640 1 Processor -- start 2024-09-19T09:25:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd5be85640 1 -- start start 2024-09-19T09:25:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd5be85640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54100b20 0x7fdd541a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd5be85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd54101440 0x7fdd541a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd5be85640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5410e500 0x7fdd5419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd5be85640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd54114280 con 0x7fdd54101440 2024-09-19T09:25:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd5be85640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd54114100 con 0x7fdd5410e500 2024-09-19T09:25:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd5be85640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd54114400 con 0x7fdd54100b20 2024-09-19T09:25:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd59420640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd54101440 0x7fdd541a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd59420640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd54101440 0x7fdd541a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44226/0 (socket says 172.21.15.16:44226) 2024-09-19T09:25:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd59420640 1 -- 172.21.15.16:0/3101722940 learned_addr learned my addr 172.21.15.16:0/3101722940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd59c21640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54100b20 0x7fdd541a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd5a422640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5410e500 0x7fdd5419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd59420640 1 -- 172.21.15.16:0/3101722940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54100b20 msgr2=0x7fdd541a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd59420640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54100b20 0x7fdd541a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd59420640 1 -- 172.21.15.16:0/3101722940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5410e500 msgr2=0x7fdd5419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd59420640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5410e500 0x7fdd5419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd59420640 1 -- 172.21.15.16:0/3101722940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5419ace0 con 0x7fdd54101440 2024-09-19T09:25:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.425+0000 7fdd5a422640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5410e500 0x7fdd5419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.426+0000 7fdd59420640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd54101440 0x7fdd541a0930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800c970 tx=0x7fdd4800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.426+0000 7fdd52ffd640 1 -- 172.21.15.16:0/3101722940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48004220 con 0x7fdd54101440 2024-09-19T09:25:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.426+0000 7fdd52ffd640 1 -- 172.21.15.16:0/3101722940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd480043c0 con 0x7fdd54101440 2024-09-19T09:25:52.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.426+0000 7fdd5be85640 1 -- 172.21.15.16:0/3101722940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd5419afd0 con 0x7fdd54101440 2024-09-19T09:25:52.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.426+0000 7fdd52ffd640 1 -- 172.21.15.16:0/3101722940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800f450 con 0x7fdd54101440 2024-09-19T09:25:52.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.426+0000 7fdd5be85640 1 -- 172.21.15.16:0/3101722940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd541a72c0 con 0x7fdd54101440 2024-09-19T09:25:52.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.427+0000 7fdd5be85640 1 -- 172.21.15.16:0/3101722940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd5406a850 con 0x7fdd54101440 2024-09-19T09:25:52.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.428+0000 7fdd52ffd640 1 -- 172.21.15.16:0/3101722940 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdd480079b0 con 0x7fdd54101440 2024-09-19T09:25:52.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.429+0000 7fdd52ffd640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdd3c0a32d0 0x7fdd3c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:52.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.429+0000 7fdd59c21640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdd3c0a32d0 0x7fdd3c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:52.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.429+0000 7fdd52ffd640 1 -- 172.21.15.16:0/3101722940 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fdd480cfc70 con 0x7fdd54101440 2024-09-19T09:25:52.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.430+0000 7fdd59c21640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdd3c0a32d0 0x7fdd3c0a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009230 tx=0x7fdd4c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:52.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.433+0000 7fdd52ffd640 1 -- 172.21.15.16:0/3101722940 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdd48014030 con 0x7fdd54101440 2024-09-19T09:25:52.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.608+0000 7fdd5be85640 1 -- 172.21.15.16:0/3101722940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd54100f00 con 0x7fdd54101440 2024-09-19T09:25:52.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.609+0000 7fdd52ffd640 1 -- 172.21.15.16:0/3101722940 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22665 (secure 0 0 0) 0x7fdd54100f00 con 0x7fdd54101440 2024-09-19T09:25:52.611 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:52.611 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","created":"2024-09-19T09:17:48.242238+0000","modified":"2024-09-19T09:25:42.982671+0000","last_up_change":"2024-09-19T09:24:40.190289+0000","last_in_change":"2024-09-19T09:24:17.190380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:20:57.748686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:25:25.694451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:25:28.262997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbf3b28b-2282-4057-9e65-19ceb78ff19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6803","nonce":1578957623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6805","nonce":1578957623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6809","nonce":1578957623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6807","nonce":1578957623}]},"public_addr":"172.21.15.16:6803/1578957623","cluster_addr":"172.21.15.16:6805/1578957623","heartbeat_back_addr":"172.21.15.16:6809/1578957623","heartbeat_front_addr":"172.21.15.16:6807/1578957623","state":["exists","up"]},{"osd":1,"uuid":"c2517aeb-1d79-41ad-a180-369eecb9732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6801","nonce":3804152349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6803","nonce":3804152349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6807","nonce":3804152349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6805","nonce":3804152349}]},"public_addr":"172.21.15.97:6801/3804152349","cluster_addr":"172.21.15.97:6803/3804152349","heartbeat_back_addr":"172.21.15.97:6807/3804152349","heartbeat_front_addr":"172.21.15.97:6805/3804152349","state":["exists","up"]},{"osd":2,"uuid":"3c367518-5085-471a-98cb-ec5bf5facdf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6801","nonce":2692613951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6803","nonce":2692613951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6807","nonce":2692613951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6805","nonce":2692613951}]},"public_addr":"172.21.15.142:6801/2692613951","cluster_addr":"172.21.15.142:6803/2692613951","heartbeat_back_addr":"172.21.15.142:6807/2692613951","heartbeat_front_addr":"172.21.15.142:6805/2692613951","state":["exists","up"]},{"osd":3,"uuid":"e7118925-f0fa-46b3-8552-3e3bb76a6489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6811","nonce":3169290004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6813","nonce":3169290004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6817","nonce":3169290004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6815","nonce":3169290004}]},"public_addr":"172.21.15.16:6811/3169290004","cluster_addr":"172.21.15.16:6813/3169290004","heartbeat_back_addr":"172.21.15.16:6817/3169290004","heartbeat_front_addr":"172.21.15.16:6815/3169290004","state":["exists","up"]},{"osd":4,"uuid":"dc1964f7-af9c-4853-843b-9818f87c7768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6809","nonce":3491353825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6811","nonce":3491353825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6815","nonce":3491353825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6813","nonce":3491353825}]},"public_addr":"172.21.15.97:6809/3491353825","cluster_addr":"172.21.15.97:6811/3491353825","heartbeat_back_addr":"172.21.15.97:6815/3491353825","heartbeat_front_addr":"172.21.15.97:6813/3491353825","state":["exists","up"]},{"osd":5,"uuid":"4d7fbafc-d9ba-4036-a89d-aa295b8a23a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6809","nonce":577099986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6811","nonce":577099986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6815","nonce":577099986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6813","nonce":577099986}]},"public_addr":"172.21.15.142:6809/577099986","cluster_addr":"172.21.15.142:6811/577099986","heartbeat_back_addr":"172.21.15.142:6815/577099986","heartbeat_front_addr":"172.21.15.142:6813/577099986","state":["exists","up"]},{"osd":6,"uuid":"e3f346fc-c027-44f8-bd37-bccf6ec09e3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6819","nonce":4053539536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6821","nonce":4053539536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6825","nonce":4053539536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6823","nonce":4053539536}]},"public_addr":"172.21.15.16:6819/4053539536","cluster_addr":"172.21.15.16:6821/4053539536","heartbeat_back_addr":"172.21.15.16:6825/4053539536","heartbeat_front_addr":"172.21.15.16:6823/4053539536","state":["exists","up"]},{"osd":7,"uuid":"d390b547-8949-4d4b-8481-c49d6ae7e424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6817","nonce":1157122670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6819","nonce":1157122670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6823","nonce":1157122670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6821","nonce":1157122670}]},"public_addr":"172.21.15.97:6817/1157122670","cluster_addr":"172.21.15.97:6819/1157122670","heartbeat_back_addr":"172.21.15.97:6823/1157122670","heartbeat_front_addr":"172.21.15.97:6821/1157122670","state":["exists","up"]},{"osd":8,"uuid":"78a476c4-5deb-4f9e-a266-db4493850732","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6817","nonce":643795004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6819","nonce":643795004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6823","nonce":643795004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6821","nonce":643795004}]},"public_addr":"172.21.15.142:6817/643795004","cluster_addr":"172.21.15.142:6819/643795004","heartbeat_back_addr":"172.21.15.142:6823/643795004","heartbeat_front_addr":"172.21.15.142:6821/643795004","state":["exists","up"]},{"osd":9,"uuid":"0415d187-4a77-42f7-b8bb-2d0b52e9cb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6827","nonce":2651132360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6829","nonce":2651132360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2651132360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6831","nonce":2651132360}]},"public_addr":"172.21.15.16:6827/2651132360","cluster_addr":"172.21.15.16:6829/2651132360","heartbeat_back_addr":"172.21.15.16:6833/2651132360","heartbeat_front_addr":"172.21.15.16:6831/2651132360","state":["exists","up"]},{"osd":10,"uuid":"6fa78f96-cdfe-42a0-8d27-0705a0c7df8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6825","nonce":1848371486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6827","nonce":1848371486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6831","nonce":1848371486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6829","nonce":1848371486}]},"public_addr":"172.21.15.97:6825/1848371486","cluster_addr":"172.21.15.97:6827/1848371486","heartbeat_back_addr":"172.21.15.97:6831/1848371486","heartbeat_front_addr":"172.21.15.97:6829/1848371486","state":["exists","up"]},{"osd":11,"uuid":"d27f20e3-84ce-470c-9889-cd86906b158d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6825","nonce":155083878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6827","nonce":155083878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6831","nonce":155083878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6829","nonce":155083878}]},"public_addr":"172.21.15.142:6825/155083878","cluster_addr":"172.21.15.142:6827/155083878","heartbeat_back_addr":"172.21.15.142:6831/155083878","heartbeat_front_addr":"172.21.15.142:6829/155083878","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:05.088764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:29.928529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:55.236930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:20.379471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:44.984151+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:10.410074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:35.233553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:59.928700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:24.503069+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:49.853073+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:14.006841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:38.546467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3710267992":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3740275561":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/2755421444":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/357390846":"2024-09-20T09:18:17.070034+0000","172.21.15.16:6800/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3104087402":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3546947161":"2024-09-20T09:18:17.070034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:25:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.611+0000 7fdd5be85640 1 -- 172.21.15.16:0/3101722940 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdd3c0a32d0 msgr2=0x7fdd3c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.611+0000 7fdd5be85640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdd3c0a32d0 0x7fdd3c0a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009230 tx=0x7fdd4c0058d0 comp rx=0 tx=0).stop 2024-09-19T09:25:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.611+0000 7fdd5be85640 1 -- 172.21.15.16:0/3101722940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd54101440 msgr2=0x7fdd541a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.611+0000 7fdd5be85640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd54101440 0x7fdd541a0930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800c970 tx=0x7fdd4800ce30 comp rx=0 tx=0).stop 2024-09-19T09:25:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.611+0000 7fdd5be85640 1 -- 172.21.15.16:0/3101722940 shutdown_connections 2024-09-19T09:25:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.611+0000 7fdd5be85640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd5410e500 0x7fdd5419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.611+0000 7fdd5be85640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd54101440 0x7fdd541a0930 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.611+0000 7fdd5be85640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdd3c0a32d0 0x7fdd3c0a5790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:52.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.611+0000 7fdd5be85640 1 --2- 172.21.15.16:0/3101722940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54100b20 0x7fdd541a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:52.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.611+0000 7fdd5be85640 1 -- 172.21.15.16:0/3101722940 >> 172.21.15.16:0/3101722940 conn(0x7fdd540fc7d0 msgr2=0x7fdd540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:52.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.612+0000 7fdd5be85640 1 -- 172.21.15.16:0/3101722940 shutdown_connections 2024-09-19T09:25:52.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:52.612+0000 7fdd5be85640 1 -- 172.21.15.16:0/3101722940 wait complete. 2024-09-19T09:25:53.315 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:53 smithi016 ceph-mon[30979]: pgmap v286: 97 pgs: 97 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.5 KiB/s wr, 22 op/s; 196 B/s, 1 objects/s recovering 2024-09-19T09:25:53.315 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:53 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3101722940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:25:53.348 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T09:25:53.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:25:53.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:53 smithi142 ceph-mon[35032]: pgmap v286: 97 pgs: 97 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.5 KiB/s wr, 22 op/s; 196 B/s, 1 objects/s recovering 2024-09-19T09:25:53.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:53 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3101722940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:25:53.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:53 smithi097 ceph-mon[34864]: pgmap v286: 97 pgs: 97 active+clean; 606 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.5 KiB/s wr, 22 op/s; 196 B/s, 1 objects/s recovering 2024-09-19T09:25:53.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:53 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3101722940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:25:53.609 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.041+0000 7f8447e5c640 1 -- 172.21.15.16:0/2994516160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440101de0 msgr2=0x7f84401021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.041+0000 7f8447e5c640 1 --2- 172.21.15.16:0/2994516160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440101de0 0x7f84401021c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f843c009900 tx=0x7f843c02a410 comp rx=0 tx=0).stop 2024-09-19T09:25:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.041+0000 7f8447e5c640 1 -- 172.21.15.16:0/2994516160 shutdown_connections 2024-09-19T09:25:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.041+0000 7f8447e5c640 1 --2- 172.21.15.16:0/2994516160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8440102790 0x7f84401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.041+0000 7f8447e5c640 1 --2- 172.21.15.16:0/2994516160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8440106f10 0x7f844010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.041+0000 7f8447e5c640 1 --2- 172.21.15.16:0/2994516160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440101de0 0x7f84401021c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.041+0000 7f8447e5c640 1 -- 172.21.15.16:0/2994516160 >> 172.21.15.16:0/2994516160 conn(0x7f84400fd880 msgr2=0x7f84400ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.041+0000 7f8447e5c640 1 -- 172.21.15.16:0/2994516160 shutdown_connections 2024-09-19T09:25:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.041+0000 7f8447e5c640 1 -- 172.21.15.16:0/2994516160 wait complete. 2024-09-19T09:25:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f8447e5c640 1 Processor -- start 2024-09-19T09:25:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f8447e5c640 1 -- start start 2024-09-19T09:25:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f8447e5c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8440101de0 0x7f8440198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f8447e5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440102790 0x7f8440198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f8447e5c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8440106f10 0x7f844019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f8447e5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8440111f70 con 0x7f8440102790 2024-09-19T09:25:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f8447e5c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8440111df0 con 0x7f8440106f10 2024-09-19T09:25:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f8447e5c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84401120f0 con 0x7f8440101de0 2024-09-19T09:25:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f84453f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440102790 0x7f8440198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f84453f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440102790 0x7f8440198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44252/0 (socket says 172.21.15.16:44252) 2024-09-19T09:25:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f84453f7640 1 -- 172.21.15.16:0/1777622129 learned_addr learned my addr 172.21.15.16:0/1777622129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f8445bf8640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8440101de0 0x7f8440198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f84453f7640 1 -- 172.21.15.16:0/1777622129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8440101de0 msgr2=0x7f8440198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f84453f7640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8440101de0 0x7f8440198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f84453f7640 1 -- 172.21.15.16:0/1777622129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8440106f10 msgr2=0x7f844019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f84453f7640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8440106f10 0x7f844019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f84453f7640 1 -- 172.21.15.16:0/1777622129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844019d3b0 con 0x7f8440102790 2024-09-19T09:25:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.042+0000 7f8445bf8640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8440101de0 0x7f8440198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.043+0000 7f84453f7640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440102790 0x7f8440198940 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f843000e9f0 tx=0x7f843000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.043+0000 7f8436ffd640 1 -- 172.21.15.16:0/1777622129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843000cd80 con 0x7f8440102790 2024-09-19T09:25:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.043+0000 7f8436ffd640 1 -- 172.21.15.16:0/1777622129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8430005500 con 0x7f8440102790 2024-09-19T09:25:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.043+0000 7f8447e5c640 1 -- 172.21.15.16:0/1777622129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844019d6a0 con 0x7f8440102790 2024-09-19T09:25:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.043+0000 7f8436ffd640 1 -- 172.21.15.16:0/1777622129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430010690 con 0x7f8440102790 2024-09-19T09:25:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.043+0000 7f8447e5c640 1 -- 172.21.15.16:0/1777622129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84401a4f80 con 0x7f8440102790 2024-09-19T09:25:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.044+0000 7f8447e5c640 1 -- 172.21.15.16:0/1777622129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84401021c0 con 0x7f8440102790 2024-09-19T09:25:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.045+0000 7f8436ffd640 1 -- 172.21.15.16:0/1777622129 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f84300046c0 con 0x7f8440102790 2024-09-19T09:25:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.046+0000 7f8436ffd640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f842c0a3350 0x7f842c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.046+0000 7f8445bf8640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f842c0a3350 0x7f842c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.046+0000 7f8436ffd640 1 -- 172.21.15.16:0/1777622129 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f84300cff70 con 0x7f8440102790 2024-09-19T09:25:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.046+0000 7f8445bf8640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f842c0a3350 0x7f842c0a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f843c005ab0 tx=0x7f843c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.050+0000 7f8436ffd640 1 -- 172.21.15.16:0/1777622129 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f843009cb30 con 0x7f8440102790 2024-09-19T09:25:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.227+0000 7f8447e5c640 1 -- 172.21.15.16:0/1777622129 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8440062630 con 0x7f842c0a3350 2024-09-19T09:25:54.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.234+0000 7f8436ffd640 1 -- 172.21.15.16:0/1777622129 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258775 (secure 0 0 0) 0x7f8440062630 con 0x7f842c0a3350 2024-09-19T09:25:54.237 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T09:25:54.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.236+0000 7f8447e5c640 1 -- 172.21.15.16:0/1777622129 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f842c0a3350 msgr2=0x7f842c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:54.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.236+0000 7f8447e5c640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f842c0a3350 0x7f842c0a5810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f843c005ab0 tx=0x7f843c005a20 comp rx=0 tx=0).stop 2024-09-19T09:25:54.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.236+0000 7f8447e5c640 1 -- 172.21.15.16:0/1777622129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440102790 msgr2=0x7f8440198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:54.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.236+0000 7f8447e5c640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440102790 0x7f8440198940 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f843000e9f0 tx=0x7f843000edf0 comp rx=0 tx=0).stop 2024-09-19T09:25:54.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.236+0000 7f8447e5c640 1 -- 172.21.15.16:0/1777622129 shutdown_connections 2024-09-19T09:25:54.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.236+0000 7f8447e5c640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8440106f10 0x7f844019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:54.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.236+0000 7f8447e5c640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440102790 0x7f8440198940 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:54.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.236+0000 7f8447e5c640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f842c0a3350 0x7f842c0a5810 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:54.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.236+0000 7f8447e5c640 1 --2- 172.21.15.16:0/1777622129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8440101de0 0x7f8440198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:54.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.237+0000 7f8447e5c640 1 -- 172.21.15.16:0/1777622129 >> 172.21.15.16:0/1777622129 conn(0x7f84400fd880 msgr2=0x7f84400feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:54.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.237+0000 7f8447e5c640 1 -- 172.21.15.16:0/1777622129 shutdown_connections 2024-09-19T09:25:54.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:54.237+0000 7f8447e5c640 1 -- 172.21.15.16:0/1777622129 wait complete. 2024-09-19T09:25:55.013 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-19T09:25:53.337746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645398,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":291,"num_read_kb":252,"num_write":418,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1182578,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386828,"kb_used_data":12600,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554996,"statfs":{"total":1151940427776,"available":1151544315904,"internally_reserved":0,"allocated":12902400,"data_stored":7081794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47780,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":-3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":126,"num_read_kb":113,"num_write":52,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1842,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.353312"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988927+0000","last_change":"2024-09-19T09:25:29.294965+0000","last_active":"2024-09-19T09:25:42.988927+0000","last_peered":"2024-09-19T09:25:42.988927+0000","last_clean":"2024-09-19T09:25:42.988927+0000","last_became_active":"2024-09-19T09:25:29.294157+0000","last_became_peered":"2024-09-19T09:25:29.294157+0000","last_unstale":"2024-09-19T09:25:42.988927+0000","last_undegraded":"2024-09-19T09:25:42.988927+0000","last_fullsized":"2024-09-19T09:25:42.988927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:24.643835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992123+0000","last_change":"2024-09-19T09:25:29.298888+0000","last_active":"2024-09-19T09:25:42.992123+0000","last_peered":"2024-09-19T09:25:42.992123+0000","last_clean":"2024-09-19T09:25:42.992123+0000","last_became_active":"2024-09-19T09:25:29.298245+0000","last_became_peered":"2024-09-19T09:25:29.298245+0000","last_unstale":"2024-09-19T09:25:42.992123+0000","last_undegraded":"2024-09-19T09:25:42.992123+0000","last_fullsized":"2024-09-19T09:25:42.992123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:03.162342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992284+0000","last_change":"2024-09-19T09:25:29.292300+0000","last_active":"2024-09-19T09:25:42.992284+0000","last_peered":"2024-09-19T09:25:42.992284+0000","last_clean":"2024-09-19T09:25:42.992284+0000","last_became_active":"2024-09-19T09:25:29.291973+0000","last_became_peered":"2024-09-19T09:25:29.291973+0000","last_unstale":"2024-09-19T09:25:42.992284+0000","last_undegraded":"2024-09-19T09:25:42.992284+0000","last_fullsized":"2024-09-19T09:25:42.992284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:19.672778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990140+0000","last_change":"2024-09-19T09:25:29.305695+0000","last_active":"2024-09-19T09:25:42.990140+0000","last_peered":"2024-09-19T09:25:42.990140+0000","last_clean":"2024-09-19T09:25:42.990140+0000","last_became_active":"2024-09-19T09:25:29.305244+0000","last_became_peered":"2024-09-19T09:25:29.305244+0000","last_unstale":"2024-09-19T09:25:42.990140+0000","last_undegraded":"2024-09-19T09:25:42.990140+0000","last_fullsized":"2024-09-19T09:25:42.990140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:38.054624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991017+0000","last_change":"2024-09-19T09:25:29.295259+0000","last_active":"2024-09-19T09:25:42.991017+0000","last_peered":"2024-09-19T09:25:42.991017+0000","last_clean":"2024-09-19T09:25:42.991017+0000","last_became_active":"2024-09-19T09:25:29.294936+0000","last_became_peered":"2024-09-19T09:25:29.294936+0000","last_unstale":"2024-09-19T09:25:42.991017+0000","last_undegraded":"2024-09-19T09:25:42.991017+0000","last_fullsized":"2024-09-19T09:25:42.991017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:59.193504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986511+0000","last_change":"2024-09-19T09:25:29.299650+0000","last_active":"2024-09-19T09:25:42.986511+0000","last_peered":"2024-09-19T09:25:42.986511+0000","last_clean":"2024-09-19T09:25:42.986511+0000","last_became_active":"2024-09-19T09:25:29.299182+0000","last_became_peered":"2024-09-19T09:25:29.299182+0000","last_unstale":"2024-09-19T09:25:42.986511+0000","last_undegraded":"2024-09-19T09:25:42.986511+0000","last_fullsized":"2024-09-19T09:25:42.986511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:44.637771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989644+0000","last_change":"2024-09-19T09:25:29.302501+0000","last_active":"2024-09-19T09:25:42.989644+0000","last_peered":"2024-09-19T09:25:42.989644+0000","last_clean":"2024-09-19T09:25:42.989644+0000","last_became_active":"2024-09-19T09:25:29.301419+0000","last_became_peered":"2024-09-19T09:25:29.301419+0000","last_unstale":"2024-09-19T09:25:42.989644+0000","last_undegraded":"2024-09-19T09:25:42.989644+0000","last_fullsized":"2024-09-19T09:25:42.989644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:50.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990840+0000","last_change":"2024-09-19T09:25:29.300212+0000","last_active":"2024-09-19T09:25:42.990840+0000","last_peered":"2024-09-19T09:25:42.990840+0000","last_clean":"2024-09-19T09:25:42.990840+0000","last_became_active":"2024-09-19T09:25:29.299873+0000","last_became_peered":"2024-09-19T09:25:29.299873+0000","last_unstale":"2024-09-19T09:25:42.990840+0000","last_undegraded":"2024-09-19T09:25:42.990840+0000","last_fullsized":"2024-09-19T09:25:42.990840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:34.620314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.004117+0000","last_change":"2024-09-19T09:25:29.290214+0000","last_active":"2024-09-19T09:25:43.004117+0000","last_peered":"2024-09-19T09:25:43.004117+0000","last_clean":"2024-09-19T09:25:43.004117+0000","last_became_active":"2024-09-19T09:25:29.289707+0000","last_became_peered":"2024-09-19T09:25:29.289707+0000","last_unstale":"2024-09-19T09:25:43.004117+0000","last_undegraded":"2024-09-19T09:25:43.004117+0000","last_fullsized":"2024-09-19T09:25:43.004117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:16.341340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.987005+0000","last_change":"2024-09-19T09:25:29.301774+0000","last_active":"2024-09-19T09:25:42.987005+0000","last_peered":"2024-09-19T09:25:42.987005+0000","last_clean":"2024-09-19T09:25:42.987005+0000","last_became_active":"2024-09-19T09:25:29.301398+0000","last_became_peered":"2024-09-19T09:25:29.301398+0000","last_unstale":"2024-09-19T09:25:42.987005+0000","last_undegraded":"2024-09-19T09:25:42.987005+0000","last_fullsized":"2024-09-19T09:25:42.987005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:36.058038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989844+0000","last_change":"2024-09-19T09:25:29.301475+0000","last_active":"2024-09-19T09:25:42.989844+0000","last_peered":"2024-09-19T09:25:42.989844+0000","last_clean":"2024-09-19T09:25:42.989844+0000","last_became_active":"2024-09-19T09:25:29.300908+0000","last_became_peered":"2024-09-19T09:25:29.300908+0000","last_unstale":"2024-09-19T09:25:42.989844+0000","last_undegraded":"2024-09-19T09:25:42.989844+0000","last_fullsized":"2024-09-19T09:25:42.989844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:44.774127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994395+0000","last_change":"2024-09-19T09:25:42.994395+0000","last_active":"2024-09-19T09:25:42.994395+0000","last_peered":"2024-09-19T09:25:42.994395+0000","last_clean":"2024-09-19T09:25:42.994395+0000","last_became_active":"2024-09-19T09:25:42.993992+0000","last_became_peered":"2024-09-19T09:25:42.993992+0000","last_unstale":"2024-09-19T09:25:42.994395+0000","last_undegraded":"2024-09-19T09:25:42.994395+0000","last_fullsized":"2024-09-19T09:25:42.994395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.486069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989979+0000","last_change":"2024-09-19T09:25:42.989979+0000","last_active":"2024-09-19T09:25:42.989979+0000","last_peered":"2024-09-19T09:25:42.989979+0000","last_clean":"2024-09-19T09:25:42.989979+0000","last_became_active":"2024-09-19T09:25:42.989578+0000","last_became_peered":"2024-09-19T09:25:42.989578+0000","last_unstale":"2024-09-19T09:25:42.989979+0000","last_undegraded":"2024-09-19T09:25:42.989979+0000","last_fullsized":"2024-09-19T09:25:42.989979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:04.583023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992064+0000","last_change":"2024-09-19T09:25:29.299917+0000","last_active":"2024-09-19T09:25:42.992064+0000","last_peered":"2024-09-19T09:25:42.992064+0000","last_clean":"2024-09-19T09:25:42.992064+0000","last_became_active":"2024-09-19T09:25:29.299571+0000","last_became_peered":"2024-09-19T09:25:29.299571+0000","last_unstale":"2024-09-19T09:25:42.992064+0000","last_undegraded":"2024-09-19T09:25:42.992064+0000","last_fullsized":"2024-09-19T09:25:42.992064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:25.142252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991982+0000","last_change":"2024-09-19T09:25:29.299713+0000","last_active":"2024-09-19T09:25:42.991982+0000","last_peered":"2024-09-19T09:25:42.991982+0000","last_clean":"2024-09-19T09:25:42.991982+0000","last_became_active":"2024-09-19T09:25:29.299234+0000","last_became_peered":"2024-09-19T09:25:29.299234+0000","last_unstale":"2024-09-19T09:25:42.991982+0000","last_undegraded":"2024-09-19T09:25:42.991982+0000","last_fullsized":"2024-09-19T09:25:42.991982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:34.714714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003535+0000","last_change":"2024-09-19T09:25:29.293010+0000","last_active":"2024-09-19T09:25:43.003535+0000","last_peered":"2024-09-19T09:25:43.003535+0000","last_clean":"2024-09-19T09:25:43.003535+0000","last_became_active":"2024-09-19T09:25:29.292645+0000","last_became_peered":"2024-09-19T09:25:29.292645+0000","last_unstale":"2024-09-19T09:25:43.003535+0000","last_undegraded":"2024-09-19T09:25:43.003535+0000","last_fullsized":"2024-09-19T09:25:43.003535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:24.749075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989031+0000","last_change":"2024-09-19T09:25:29.294530+0000","last_active":"2024-09-19T09:25:42.989031+0000","last_peered":"2024-09-19T09:25:42.989031+0000","last_clean":"2024-09-19T09:25:42.989031+0000","last_became_active":"2024-09-19T09:25:29.293699+0000","last_became_peered":"2024-09-19T09:25:29.293699+0000","last_unstale":"2024-09-19T09:25:42.989031+0000","last_undegraded":"2024-09-19T09:25:42.989031+0000","last_fullsized":"2024-09-19T09:25:42.989031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:53.491258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988781+0000","last_change":"2024-09-19T09:25:29.290600+0000","last_active":"2024-09-19T09:25:42.988781+0000","last_peered":"2024-09-19T09:25:42.988781+0000","last_clean":"2024-09-19T09:25:42.988781+0000","last_became_active":"2024-09-19T09:25:29.290305+0000","last_became_peered":"2024-09-19T09:25:29.290305+0000","last_unstale":"2024-09-19T09:25:42.988781+0000","last_undegraded":"2024-09-19T09:25:42.988781+0000","last_fullsized":"2024-09-19T09:25:42.988781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:42.809659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.993154+0000","last_change":"2024-09-19T09:25:29.300591+0000","last_active":"2024-09-19T09:25:42.993154+0000","last_peered":"2024-09-19T09:25:42.993154+0000","last_clean":"2024-09-19T09:25:42.993154+0000","last_became_active":"2024-09-19T09:25:29.300251+0000","last_became_peered":"2024-09-19T09:25:29.300251+0000","last_unstale":"2024-09-19T09:25:42.993154+0000","last_undegraded":"2024-09-19T09:25:42.993154+0000","last_fullsized":"2024-09-19T09:25:42.993154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:08.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988507+0000","last_change":"2024-09-19T09:25:29.298452+0000","last_active":"2024-09-19T09:25:42.988507+0000","last_peered":"2024-09-19T09:25:42.988507+0000","last_clean":"2024-09-19T09:25:42.988507+0000","last_became_active":"2024-09-19T09:25:29.298137+0000","last_became_peered":"2024-09-19T09:25:29.298137+0000","last_unstale":"2024-09-19T09:25:42.988507+0000","last_undegraded":"2024-09-19T09:25:42.988507+0000","last_fullsized":"2024-09-19T09:25:42.988507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:11.716228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985825+0000","last_change":"2024-09-19T09:25:29.294278+0000","last_active":"2024-09-19T09:25:42.985825+0000","last_peered":"2024-09-19T09:25:42.985825+0000","last_clean":"2024-09-19T09:25:42.985825+0000","last_became_active":"2024-09-19T09:25:29.293972+0000","last_became_peered":"2024-09-19T09:25:29.293972+0000","last_unstale":"2024-09-19T09:25:42.985825+0000","last_undegraded":"2024-09-19T09:25:42.985825+0000","last_fullsized":"2024-09-19T09:25:42.985825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:27.923839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003737+0000","last_change":"2024-09-19T09:25:29.294000+0000","last_active":"2024-09-19T09:25:43.003737+0000","last_peered":"2024-09-19T09:25:43.003737+0000","last_clean":"2024-09-19T09:25:43.003737+0000","last_became_active":"2024-09-19T09:25:29.293486+0000","last_became_peered":"2024-09-19T09:25:29.293486+0000","last_unstale":"2024-09-19T09:25:43.003737+0000","last_undegraded":"2024-09-19T09:25:43.003737+0000","last_fullsized":"2024-09-19T09:25:43.003737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:44.053620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994951+0000","last_change":"2024-09-19T09:25:42.994951+0000","last_active":"2024-09-19T09:25:42.994951+0000","last_peered":"2024-09-19T09:25:42.994951+0000","last_clean":"2024-09-19T09:25:42.994951+0000","last_became_active":"2024-09-19T09:25:42.994588+0000","last_became_peered":"2024-09-19T09:25:42.994588+0000","last_unstale":"2024-09-19T09:25:42.994951+0000","last_undegraded":"2024-09-19T09:25:42.994951+0000","last_fullsized":"2024-09-19T09:25:42.994951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:10.141169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995128+0000","last_change":"2024-09-19T09:25:29.286395+0000","last_active":"2024-09-19T09:25:42.995128+0000","last_peered":"2024-09-19T09:25:42.995128+0000","last_clean":"2024-09-19T09:25:42.995128+0000","last_became_active":"2024-09-19T09:25:29.286048+0000","last_became_peered":"2024-09-19T09:25:29.286048+0000","last_unstale":"2024-09-19T09:25:42.995128+0000","last_undegraded":"2024-09-19T09:25:42.995128+0000","last_fullsized":"2024-09-19T09:25:42.995128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:43.121166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991295+0000","last_change":"2024-09-19T09:25:29.298458+0000","last_active":"2024-09-19T09:25:42.991295+0000","last_peered":"2024-09-19T09:25:42.991295+0000","last_clean":"2024-09-19T09:25:42.991295+0000","last_became_active":"2024-09-19T09:25:29.297993+0000","last_became_peered":"2024-09-19T09:25:29.297993+0000","last_unstale":"2024-09-19T09:25:42.991295+0000","last_undegraded":"2024-09-19T09:25:42.991295+0000","last_fullsized":"2024-09-19T09:25:42.991295+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:44.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992762+0000","last_change":"2024-09-19T09:25:29.297167+0000","last_active":"2024-09-19T09:25:42.992762+0000","last_peered":"2024-09-19T09:25:42.992762+0000","last_clean":"2024-09-19T09:25:42.992762+0000","last_became_active":"2024-09-19T09:25:29.296777+0000","last_became_peered":"2024-09-19T09:25:29.296777+0000","last_unstale":"2024-09-19T09:25:42.992762+0000","last_undegraded":"2024-09-19T09:25:42.992762+0000","last_fullsized":"2024-09-19T09:25:42.992762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:40.581643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988833+0000","last_change":"2024-09-19T09:25:29.296169+0000","last_active":"2024-09-19T09:25:42.988833+0000","last_peered":"2024-09-19T09:25:42.988833+0000","last_clean":"2024-09-19T09:25:42.988833+0000","last_became_active":"2024-09-19T09:25:29.295806+0000","last_became_peered":"2024-09-19T09:25:29.295806+0000","last_unstale":"2024-09-19T09:25:42.988833+0000","last_undegraded":"2024-09-19T09:25:42.988833+0000","last_fullsized":"2024-09-19T09:25:42.988833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:39.639884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985866+0000","last_change":"2024-09-19T09:25:29.297960+0000","last_active":"2024-09-19T09:25:42.985866+0000","last_peered":"2024-09-19T09:25:42.985866+0000","last_clean":"2024-09-19T09:25:42.985866+0000","last_became_active":"2024-09-19T09:25:29.297577+0000","last_became_peered":"2024-09-19T09:25:29.297577+0000","last_unstale":"2024-09-19T09:25:42.985866+0000","last_undegraded":"2024-09-19T09:25:42.985866+0000","last_fullsized":"2024-09-19T09:25:42.985866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:27.035504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992420+0000","last_change":"2024-09-19T09:25:29.297896+0000","last_active":"2024-09-19T09:25:42.992420+0000","last_peered":"2024-09-19T09:25:42.992420+0000","last_clean":"2024-09-19T09:25:42.992420+0000","last_became_active":"2024-09-19T09:25:29.297594+0000","last_became_peered":"2024-09-19T09:25:29.297594+0000","last_unstale":"2024-09-19T09:25:42.992420+0000","last_undegraded":"2024-09-19T09:25:42.992420+0000","last_fullsized":"2024-09-19T09:25:42.992420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.366325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992205+0000","last_change":"2024-09-19T09:25:29.301851+0000","last_active":"2024-09-19T09:25:42.992205+0000","last_peered":"2024-09-19T09:25:42.992205+0000","last_clean":"2024-09-19T09:25:42.992205+0000","last_became_active":"2024-09-19T09:25:29.299282+0000","last_became_peered":"2024-09-19T09:25:29.299282+0000","last_unstale":"2024-09-19T09:25:42.992205+0000","last_undegraded":"2024-09-19T09:25:42.992205+0000","last_fullsized":"2024-09-19T09:25:42.992205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:00.382114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992658+0000","last_change":"2024-09-19T09:25:29.303316+0000","last_active":"2024-09-19T09:25:42.992658+0000","last_peered":"2024-09-19T09:25:42.992658+0000","last_clean":"2024-09-19T09:25:42.992658+0000","last_became_active":"2024-09-19T09:25:29.302994+0000","last_became_peered":"2024-09-19T09:25:29.302994+0000","last_unstale":"2024-09-19T09:25:42.992658+0000","last_undegraded":"2024-09-19T09:25:42.992658+0000","last_fullsized":"2024-09-19T09:25:42.992658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:46.233991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988501+0000","last_change":"2024-09-19T09:25:29.293004+0000","last_active":"2024-09-19T09:25:42.988501+0000","last_peered":"2024-09-19T09:25:42.988501+0000","last_clean":"2024-09-19T09:25:42.988501+0000","last_became_active":"2024-09-19T09:25:29.292657+0000","last_became_peered":"2024-09-19T09:25:29.292657+0000","last_unstale":"2024-09-19T09:25:42.988501+0000","last_undegraded":"2024-09-19T09:25:42.988501+0000","last_fullsized":"2024-09-19T09:25:42.988501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:50.966477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991936+0000","last_change":"2024-09-19T09:25:29.300434+0000","last_active":"2024-09-19T09:25:42.991936+0000","last_peered":"2024-09-19T09:25:42.991936+0000","last_clean":"2024-09-19T09:25:42.991936+0000","last_became_active":"2024-09-19T09:25:29.300053+0000","last_became_peered":"2024-09-19T09:25:29.300053+0000","last_unstale":"2024-09-19T09:25:42.991936+0000","last_undegraded":"2024-09-19T09:25:42.991936+0000","last_fullsized":"2024-09-19T09:25:42.991936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:13.283298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":46,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:51.982309+0000","last_change":"2024-09-19T09:25:27.280792+0000","last_active":"2024-09-19T09:25:51.982309+0000","last_peered":"2024-09-19T09:25:51.982309+0000","last_clean":"2024-09-19T09:25:51.982309+0000","last_became_active":"2024-09-19T09:25:27.280443+0000","last_became_peered":"2024-09-19T09:25:27.280443+0000","last_unstale":"2024-09-19T09:25:51.982309+0000","last_undegraded":"2024-09-19T09:25:51.982309+0000","last_fullsized":"2024-09-19T09:25:51.982309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.599512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.997114+0000","last_change":"2024-09-19T09:25:29.298975+0000","last_active":"2024-09-19T09:25:42.997114+0000","last_peered":"2024-09-19T09:25:42.997114+0000","last_clean":"2024-09-19T09:25:42.997114+0000","last_became_active":"2024-09-19T09:25:29.298476+0000","last_became_peered":"2024-09-19T09:25:29.298476+0000","last_unstale":"2024-09-19T09:25:42.997114+0000","last_undegraded":"2024-09-19T09:25:42.997114+0000","last_fullsized":"2024-09-19T09:25:42.997114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:09.658894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986623+0000","last_change":"2024-09-19T09:25:27.278257+0000","last_active":"2024-09-19T09:25:42.986623+0000","last_peered":"2024-09-19T09:25:42.986623+0000","last_clean":"2024-09-19T09:25:42.986623+0000","last_became_active":"2024-09-19T09:25:27.277897+0000","last_became_peered":"2024-09-19T09:25:27.277897+0000","last_unstale":"2024-09-19T09:25:42.986623+0000","last_undegraded":"2024-09-19T09:25:42.986623+0000","last_fullsized":"2024-09-19T09:25:42.986623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:47.550314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985617+0000","last_change":"2024-09-19T09:25:27.272215+0000","last_active":"2024-09-19T09:25:42.985617+0000","last_peered":"2024-09-19T09:25:42.985617+0000","last_clean":"2024-09-19T09:25:42.985617+0000","last_became_active":"2024-09-19T09:25:27.271821+0000","last_became_peered":"2024-09-19T09:25:27.271821+0000","last_unstale":"2024-09-19T09:25:42.985617+0000","last_undegraded":"2024-09-19T09:25:42.985617+0000","last_fullsized":"2024-09-19T09:25:42.985617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:30.615730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990922+0000","last_change":"2024-09-19T09:25:29.289347+0000","last_active":"2024-09-19T09:25:42.990922+0000","last_peered":"2024-09-19T09:25:42.990922+0000","last_clean":"2024-09-19T09:25:42.990922+0000","last_became_active":"2024-09-19T09:25:29.289015+0000","last_became_peered":"2024-09-19T09:25:29.289015+0000","last_unstale":"2024-09-19T09:25:42.990922+0000","last_undegraded":"2024-09-19T09:25:42.990922+0000","last_fullsized":"2024-09-19T09:25:42.990922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:18.419334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995432+0000","last_change":"2024-09-19T09:25:27.277342+0000","last_active":"2024-09-19T09:25:42.995432+0000","last_peered":"2024-09-19T09:25:42.995432+0000","last_clean":"2024-09-19T09:25:42.995432+0000","last_became_active":"2024-09-19T09:25:27.277016+0000","last_became_peered":"2024-09-19T09:25:27.277016+0000","last_unstale":"2024-09-19T09:25:42.995432+0000","last_undegraded":"2024-09-19T09:25:42.995432+0000","last_fullsized":"2024-09-19T09:25:42.995432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:56.736156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988740+0000","last_change":"2024-09-19T09:25:29.290924+0000","last_active":"2024-09-19T09:25:42.988740+0000","last_peered":"2024-09-19T09:25:42.988740+0000","last_clean":"2024-09-19T09:25:42.988740+0000","last_became_active":"2024-09-19T09:25:29.290482+0000","last_became_peered":"2024-09-19T09:25:29.290482+0000","last_unstale":"2024-09-19T09:25:42.988740+0000","last_undegraded":"2024-09-19T09:25:42.988740+0000","last_fullsized":"2024-09-19T09:25:42.988740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:16.265040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:50.980337+0000","last_change":"2024-09-19T09:25:27.277604+0000","last_active":"2024-09-19T09:25:50.980337+0000","last_peered":"2024-09-19T09:25:50.980337+0000","last_clean":"2024-09-19T09:25:50.980337+0000","last_became_active":"2024-09-19T09:25:27.277271+0000","last_became_peered":"2024-09-19T09:25:27.277271+0000","last_unstale":"2024-09-19T09:25:50.980337+0000","last_undegraded":"2024-09-19T09:25:50.980337+0000","last_fullsized":"2024-09-19T09:25:50.980337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:32.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.004044+0000","last_change":"2024-09-19T09:25:29.290537+0000","last_active":"2024-09-19T09:25:43.004044+0000","last_peered":"2024-09-19T09:25:43.004044+0000","last_clean":"2024-09-19T09:25:43.004044+0000","last_became_active":"2024-09-19T09:25:29.287984+0000","last_became_peered":"2024-09-19T09:25:29.287984+0000","last_unstale":"2024-09-19T09:25:43.004044+0000","last_undegraded":"2024-09-19T09:25:43.004044+0000","last_fullsized":"2024-09-19T09:25:43.004044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:26.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990018+0000","last_change":"2024-09-19T09:25:27.278560+0000","last_active":"2024-09-19T09:25:42.990018+0000","last_peered":"2024-09-19T09:25:42.990018+0000","last_clean":"2024-09-19T09:25:42.990018+0000","last_became_active":"2024-09-19T09:25:27.277950+0000","last_became_peered":"2024-09-19T09:25:27.277950+0000","last_unstale":"2024-09-19T09:25:42.990018+0000","last_undegraded":"2024-09-19T09:25:42.990018+0000","last_fullsized":"2024-09-19T09:25:42.990018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:28.387725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992413+0000","last_change":"2024-09-19T09:25:29.298897+0000","last_active":"2024-09-19T09:25:42.992413+0000","last_peered":"2024-09-19T09:25:42.992413+0000","last_clean":"2024-09-19T09:25:42.992413+0000","last_became_active":"2024-09-19T09:25:29.298617+0000","last_became_peered":"2024-09-19T09:25:29.298617+0000","last_unstale":"2024-09-19T09:25:42.992413+0000","last_undegraded":"2024-09-19T09:25:42.992413+0000","last_fullsized":"2024-09-19T09:25:42.992413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:53.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994870+0000","last_change":"2024-09-19T09:25:27.324871+0000","last_active":"2024-09-19T09:25:42.994870+0000","last_peered":"2024-09-19T09:25:42.994870+0000","last_clean":"2024-09-19T09:25:42.994870+0000","last_became_active":"2024-09-19T09:25:27.324471+0000","last_became_peered":"2024-09-19T09:25:27.324471+0000","last_unstale":"2024-09-19T09:25:42.994870+0000","last_undegraded":"2024-09-19T09:25:42.994870+0000","last_fullsized":"2024-09-19T09:25:42.994870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:35.936413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.004242+0000","last_change":"2024-09-19T09:25:29.298332+0000","last_active":"2024-09-19T09:25:43.004242+0000","last_peered":"2024-09-19T09:25:43.004242+0000","last_clean":"2024-09-19T09:25:43.004242+0000","last_became_active":"2024-09-19T09:25:29.298006+0000","last_became_peered":"2024-09-19T09:25:29.298006+0000","last_unstale":"2024-09-19T09:25:43.004242+0000","last_undegraded":"2024-09-19T09:25:43.004242+0000","last_fullsized":"2024-09-19T09:25:43.004242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:02.621821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'24","reported_seq":78,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:50.987443+0000","last_change":"2024-09-19T09:25:27.279528+0000","last_active":"2024-09-19T09:25:50.987443+0000","last_peered":"2024-09-19T09:25:50.987443+0000","last_clean":"2024-09-19T09:25:50.987443+0000","last_became_active":"2024-09-19T09:25:27.279149+0000","last_became_peered":"2024-09-19T09:25:27.279149+0000","last_unstale":"2024-09-19T09:25:50.987443+0000","last_undegraded":"2024-09-19T09:25:50.987443+0000","last_fullsized":"2024-09-19T09:25:50.987443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:47.030736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":61,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991057+0000","last_change":"2024-09-19T09:25:29.297538+0000","last_active":"2024-09-19T09:25:42.991057+0000","last_peered":"2024-09-19T09:25:42.991057+0000","last_clean":"2024-09-19T09:25:42.991057+0000","last_became_active":"2024-09-19T09:25:29.297181+0000","last_became_peered":"2024-09-19T09:25:29.297181+0000","last_unstale":"2024-09-19T09:25:42.991057+0000","last_undegraded":"2024-09-19T09:25:42.991057+0000","last_fullsized":"2024-09-19T09:25:42.991057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:53.135438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992860+0000","last_change":"2024-09-19T09:25:27.277090+0000","last_active":"2024-09-19T09:25:42.992860+0000","last_peered":"2024-09-19T09:25:42.992860+0000","last_clean":"2024-09-19T09:25:42.992860+0000","last_became_active":"2024-09-19T09:25:27.276718+0000","last_became_peered":"2024-09-19T09:25:27.276718+0000","last_unstale":"2024-09-19T09:25:42.992860+0000","last_undegraded":"2024-09-19T09:25:42.992860+0000","last_fullsized":"2024-09-19T09:25:42.992860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:25.664261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985642+0000","last_change":"2024-09-19T09:25:29.294393+0000","last_active":"2024-09-19T09:25:42.985642+0000","last_peered":"2024-09-19T09:25:42.985642+0000","last_clean":"2024-09-19T09:25:42.985642+0000","last_became_active":"2024-09-19T09:25:29.294099+0000","last_became_peered":"2024-09-19T09:25:29.294099+0000","last_unstale":"2024-09-19T09:25:42.985642+0000","last_undegraded":"2024-09-19T09:25:42.985642+0000","last_fullsized":"2024-09-19T09:25:42.985642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:17.333736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995398+0000","last_change":"2024-09-19T09:25:27.278593+0000","last_active":"2024-09-19T09:25:42.995398+0000","last_peered":"2024-09-19T09:25:42.995398+0000","last_clean":"2024-09-19T09:25:42.995398+0000","last_became_active":"2024-09-19T09:25:27.278232+0000","last_became_peered":"2024-09-19T09:25:27.278232+0000","last_unstale":"2024-09-19T09:25:42.995398+0000","last_undegraded":"2024-09-19T09:25:42.995398+0000","last_fullsized":"2024-09-19T09:25:42.995398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:05.852671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989855+0000","last_change":"2024-09-19T09:25:29.300024+0000","last_active":"2024-09-19T09:25:42.989855+0000","last_peered":"2024-09-19T09:25:42.989855+0000","last_clean":"2024-09-19T09:25:42.989855+0000","last_became_active":"2024-09-19T09:25:29.299683+0000","last_became_peered":"2024-09-19T09:25:29.299683+0000","last_unstale":"2024-09-19T09:25:42.989855+0000","last_undegraded":"2024-09-19T09:25:42.989855+0000","last_fullsized":"2024-09-19T09:25:42.989855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:42.249399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:50.981839+0000","last_change":"2024-09-19T09:25:27.272739+0000","last_active":"2024-09-19T09:25:50.981839+0000","last_peered":"2024-09-19T09:25:50.981839+0000","last_clean":"2024-09-19T09:25:50.981839+0000","last_became_active":"2024-09-19T09:25:27.272354+0000","last_became_peered":"2024-09-19T09:25:27.272354+0000","last_unstale":"2024-09-19T09:25:50.981839+0000","last_undegraded":"2024-09-19T09:25:50.981839+0000","last_fullsized":"2024-09-19T09:25:50.981839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:27.863840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989616+0000","last_change":"2024-09-19T09:25:29.303460+0000","last_active":"2024-09-19T09:25:42.989616+0000","last_peered":"2024-09-19T09:25:42.989616+0000","last_clean":"2024-09-19T09:25:42.989616+0000","last_became_active":"2024-09-19T09:25:29.303127+0000","last_became_peered":"2024-09-19T09:25:29.303127+0000","last_unstale":"2024-09-19T09:25:42.989616+0000","last_undegraded":"2024-09-19T09:25:42.989616+0000","last_fullsized":"2024-09-19T09:25:42.989616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:35.313924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990876+0000","last_change":"2024-09-19T09:25:27.323238+0000","last_active":"2024-09-19T09:25:42.990876+0000","last_peered":"2024-09-19T09:25:42.990876+0000","last_clean":"2024-09-19T09:25:42.990876+0000","last_became_active":"2024-09-19T09:25:27.322868+0000","last_became_peered":"2024-09-19T09:25:27.322868+0000","last_unstale":"2024-09-19T09:25:42.990876+0000","last_undegraded":"2024-09-19T09:25:42.990876+0000","last_fullsized":"2024-09-19T09:25:42.990876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:22.890280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.005569+0000","last_change":"2024-09-19T09:25:43.005569+0000","last_active":"2024-09-19T09:25:43.005569+0000","last_peered":"2024-09-19T09:25:43.005569+0000","last_clean":"2024-09-19T09:25:43.005569+0000","last_became_active":"2024-09-19T09:25:43.005143+0000","last_became_peered":"2024-09-19T09:25:43.005143+0000","last_unstale":"2024-09-19T09:25:43.005569+0000","last_undegraded":"2024-09-19T09:25:43.005569+0000","last_fullsized":"2024-09-19T09:25:43.005569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:26.297013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991438+0000","last_change":"2024-09-19T09:25:27.282737+0000","last_active":"2024-09-19T09:25:42.991438+0000","last_peered":"2024-09-19T09:25:42.991438+0000","last_clean":"2024-09-19T09:25:42.991438+0000","last_became_active":"2024-09-19T09:25:27.281785+0000","last_became_peered":"2024-09-19T09:25:27.281785+0000","last_unstale":"2024-09-19T09:25:42.991438+0000","last_undegraded":"2024-09-19T09:25:42.991438+0000","last_fullsized":"2024-09-19T09:25:42.991438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:41.552206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988791+0000","last_change":"2024-09-19T09:25:29.300168+0000","last_active":"2024-09-19T09:25:42.988791+0000","last_peered":"2024-09-19T09:25:42.988791+0000","last_clean":"2024-09-19T09:25:42.988791+0000","last_became_active":"2024-09-19T09:25:29.299811+0000","last_became_peered":"2024-09-19T09:25:29.299811+0000","last_unstale":"2024-09-19T09:25:42.988791+0000","last_undegraded":"2024-09-19T09:25:42.988791+0000","last_fullsized":"2024-09-19T09:25:42.988791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:06.691259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985783+0000","last_change":"2024-09-19T09:25:27.278433+0000","last_active":"2024-09-19T09:25:42.985783+0000","last_peered":"2024-09-19T09:25:42.985783+0000","last_clean":"2024-09-19T09:25:42.985783+0000","last_became_active":"2024-09-19T09:25:27.278076+0000","last_became_peered":"2024-09-19T09:25:27.278076+0000","last_unstale":"2024-09-19T09:25:42.985783+0000","last_undegraded":"2024-09-19T09:25:42.985783+0000","last_fullsized":"2024-09-19T09:25:42.985783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:53.799265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990338+0000","last_change":"2024-09-19T09:25:29.295650+0000","last_active":"2024-09-19T09:25:42.990338+0000","last_peered":"2024-09-19T09:25:42.990338+0000","last_clean":"2024-09-19T09:25:42.990338+0000","last_became_active":"2024-09-19T09:25:29.295277+0000","last_became_peered":"2024-09-19T09:25:29.295277+0000","last_unstale":"2024-09-19T09:25:42.990338+0000","last_undegraded":"2024-09-19T09:25:42.990338+0000","last_fullsized":"2024-09-19T09:25:42.990338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:57.690699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996123+0000","last_change":"2024-09-19T09:25:27.279239+0000","last_active":"2024-09-19T09:25:42.996123+0000","last_peered":"2024-09-19T09:25:42.996123+0000","last_clean":"2024-09-19T09:25:42.996123+0000","last_became_active":"2024-09-19T09:25:27.278868+0000","last_became_peered":"2024-09-19T09:25:27.278868+0000","last_unstale":"2024-09-19T09:25:42.996123+0000","last_undegraded":"2024-09-19T09:25:42.996123+0000","last_fullsized":"2024-09-19T09:25:42.996123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:41.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990429+0000","last_change":"2024-09-19T09:25:29.300833+0000","last_active":"2024-09-19T09:25:42.990429+0000","last_peered":"2024-09-19T09:25:42.990429+0000","last_clean":"2024-09-19T09:25:42.990429+0000","last_became_active":"2024-09-19T09:25:29.300472+0000","last_became_peered":"2024-09-19T09:25:29.300472+0000","last_unstale":"2024-09-19T09:25:42.990429+0000","last_undegraded":"2024-09-19T09:25:42.990429+0000","last_fullsized":"2024-09-19T09:25:42.990429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:50.591276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:51.008075+0000","last_change":"2024-09-19T09:25:43.001722+0000","last_active":"2024-09-19T09:25:51.008075+0000","last_peered":"2024-09-19T09:25:51.008075+0000","last_clean":"2024-09-19T09:25:51.008075+0000","last_became_active":"2024-09-19T09:25:42.992796+0000","last_became_peered":"2024-09-19T09:25:42.992796+0000","last_unstale":"2024-09-19T09:25:51.008075+0000","last_undegraded":"2024-09-19T09:25:51.008075+0000","last_fullsized":"2024-09-19T09:25:51.008075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:59.639010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990231+0000","last_change":"2024-09-19T09:25:29.295906+0000","last_active":"2024-09-19T09:25:42.990231+0000","last_peered":"2024-09-19T09:25:42.990231+0000","last_clean":"2024-09-19T09:25:42.990231+0000","last_became_active":"2024-09-19T09:25:29.295565+0000","last_became_peered":"2024-09-19T09:25:29.295565+0000","last_unstale":"2024-09-19T09:25:42.990231+0000","last_undegraded":"2024-09-19T09:25:42.990231+0000","last_fullsized":"2024-09-19T09:25:42.990231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:24.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":334,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989806+0000","last_change":"2024-09-19T09:24:19.256300+0000","last_active":"2024-09-19T09:25:42.989806+0000","last_peered":"2024-09-19T09:25:42.989806+0000","last_clean":"2024-09-19T09:25:42.989806+0000","last_became_active":"2024-09-19T09:24:19.255833+0000","last_became_peered":"2024-09-19T09:24:19.255833+0000","last_unstale":"2024-09-19T09:25:42.989806+0000","last_undegraded":"2024-09-19T09:25:42.989806+0000","last_fullsized":"2024-09-19T09:25:42.989806+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_clean_scrub_stamp":"2024-09-19T09:20:57.798838+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:14:33.343748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989637+0000","last_change":"2024-09-19T09:25:27.272090+0000","last_active":"2024-09-19T09:25:42.989637+0000","last_peered":"2024-09-19T09:25:42.989637+0000","last_clean":"2024-09-19T09:25:42.989637+0000","last_became_active":"2024-09-19T09:25:27.271651+0000","last_became_peered":"2024-09-19T09:25:27.271651+0000","last_unstale":"2024-09-19T09:25:42.989637+0000","last_undegraded":"2024-09-19T09:25:42.989637+0000","last_fullsized":"2024-09-19T09:25:42.989637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:45.847082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:29.300718+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:29.299460+0000","last_became_peered":"2024-09-19T09:25:29.299460+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:43.063909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003299+0000","last_change":"2024-09-19T09:25:43.003299+0000","last_active":"2024-09-19T09:25:43.003299+0000","last_peered":"2024-09-19T09:25:43.003299+0000","last_clean":"2024-09-19T09:25:43.003299+0000","last_became_active":"2024-09-19T09:25:42.999686+0000","last_became_peered":"2024-09-19T09:25:42.999686+0000","last_unstale":"2024-09-19T09:25:43.003299+0000","last_undegraded":"2024-09-19T09:25:43.003299+0000","last_fullsized":"2024-09-19T09:25:43.003299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:05.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985606+0000","last_change":"2024-09-19T09:25:29.297833+0000","last_active":"2024-09-19T09:25:42.985606+0000","last_peered":"2024-09-19T09:25:42.985606+0000","last_clean":"2024-09-19T09:25:42.985606+0000","last_became_active":"2024-09-19T09:25:29.297534+0000","last_became_peered":"2024-09-19T09:25:29.297534+0000","last_unstale":"2024-09-19T09:25:42.985606+0000","last_undegraded":"2024-09-19T09:25:42.985606+0000","last_fullsized":"2024-09-19T09:25:42.985606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:47.156253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:51.982811+0000","last_change":"2024-09-19T09:25:27.322759+0000","last_active":"2024-09-19T09:25:51.982811+0000","last_peered":"2024-09-19T09:25:51.982811+0000","last_clean":"2024-09-19T09:25:51.982811+0000","last_became_active":"2024-09-19T09:25:27.322402+0000","last_became_peered":"2024-09-19T09:25:27.322402+0000","last_unstale":"2024-09-19T09:25:51.982811+0000","last_undegraded":"2024-09-19T09:25:51.982811+0000","last_fullsized":"2024-09-19T09:25:51.982811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:06.276957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.999834+0000","last_change":"2024-09-19T09:25:42.999834+0000","last_active":"2024-09-19T09:25:42.999834+0000","last_peered":"2024-09-19T09:25:42.999834+0000","last_clean":"2024-09-19T09:25:42.999834+0000","last_became_active":"2024-09-19T09:25:42.999450+0000","last_became_peered":"2024-09-19T09:25:42.999450+0000","last_unstale":"2024-09-19T09:25:42.999834+0000","last_undegraded":"2024-09-19T09:25:42.999834+0000","last_fullsized":"2024-09-19T09:25:42.999834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:00.509844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:51.006904+0000","last_change":"2024-09-19T09:25:27.275212+0000","last_active":"2024-09-19T09:25:51.006904+0000","last_peered":"2024-09-19T09:25:51.006904+0000","last_clean":"2024-09-19T09:25:51.006904+0000","last_became_active":"2024-09-19T09:25:27.274802+0000","last_became_peered":"2024-09-19T09:25:27.274802+0000","last_unstale":"2024-09-19T09:25:51.006904+0000","last_undegraded":"2024-09-19T09:25:51.006904+0000","last_fullsized":"2024-09-19T09:25:51.006904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:48.392026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986730+0000","last_change":"2024-09-19T09:25:29.304747+0000","last_active":"2024-09-19T09:25:42.986730+0000","last_peered":"2024-09-19T09:25:42.986730+0000","last_clean":"2024-09-19T09:25:42.986730+0000","last_became_active":"2024-09-19T09:25:29.304386+0000","last_became_peered":"2024-09-19T09:25:29.304386+0000","last_unstale":"2024-09-19T09:25:42.986730+0000","last_undegraded":"2024-09-19T09:25:42.986730+0000","last_fullsized":"2024-09-19T09:25:42.986730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.788697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988854+0000","last_change":"2024-09-19T09:25:27.323083+0000","last_active":"2024-09-19T09:25:42.988854+0000","last_peered":"2024-09-19T09:25:42.988854+0000","last_clean":"2024-09-19T09:25:42.988854+0000","last_became_active":"2024-09-19T09:25:27.322707+0000","last_became_peered":"2024-09-19T09:25:27.322707+0000","last_unstale":"2024-09-19T09:25:42.988854+0000","last_undegraded":"2024-09-19T09:25:42.988854+0000","last_fullsized":"2024-09-19T09:25:42.988854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:24.415116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986687+0000","last_change":"2024-09-19T09:25:29.292472+0000","last_active":"2024-09-19T09:25:42.986687+0000","last_peered":"2024-09-19T09:25:42.986687+0000","last_clean":"2024-09-19T09:25:42.986687+0000","last_became_active":"2024-09-19T09:25:29.292079+0000","last_became_peered":"2024-09-19T09:25:29.292079+0000","last_unstale":"2024-09-19T09:25:42.986687+0000","last_undegraded":"2024-09-19T09:25:42.986687+0000","last_fullsized":"2024-09-19T09:25:42.986687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:35.801594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991224+0000","last_change":"2024-09-19T09:25:27.324839+0000","last_active":"2024-09-19T09:25:42.991224+0000","last_peered":"2024-09-19T09:25:42.991224+0000","last_clean":"2024-09-19T09:25:42.991224+0000","last_became_active":"2024-09-19T09:25:27.324475+0000","last_became_peered":"2024-09-19T09:25:27.324475+0000","last_unstale":"2024-09-19T09:25:42.991224+0000","last_undegraded":"2024-09-19T09:25:42.991224+0000","last_fullsized":"2024-09-19T09:25:42.991224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:32.251753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985998+0000","last_change":"2024-09-19T09:25:29.293025+0000","last_active":"2024-09-19T09:25:42.985998+0000","last_peered":"2024-09-19T09:25:42.985998+0000","last_clean":"2024-09-19T09:25:42.985998+0000","last_became_active":"2024-09-19T09:25:29.292570+0000","last_became_peered":"2024-09-19T09:25:29.292570+0000","last_unstale":"2024-09-19T09:25:42.985998+0000","last_undegraded":"2024-09-19T09:25:42.985998+0000","last_fullsized":"2024-09-19T09:25:42.985998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:16.394524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'13","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:51.295097+0000","last_change":"2024-09-19T09:25:27.278100+0000","last_active":"2024-09-19T09:25:51.295097+0000","last_peered":"2024-09-19T09:25:51.295097+0000","last_clean":"2024-09-19T09:25:51.295097+0000","last_became_active":"2024-09-19T09:25:27.277769+0000","last_became_peered":"2024-09-19T09:25:27.277769+0000","last_unstale":"2024-09-19T09:25:51.295097+0000","last_undegraded":"2024-09-19T09:25:51.295097+0000","last_fullsized":"2024-09-19T09:25:51.295097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:31.885142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991325+0000","last_change":"2024-09-19T09:25:29.293651+0000","last_active":"2024-09-19T09:25:42.991325+0000","last_peered":"2024-09-19T09:25:42.991325+0000","last_clean":"2024-09-19T09:25:42.991325+0000","last_became_active":"2024-09-19T09:25:29.293317+0000","last_became_peered":"2024-09-19T09:25:29.293317+0000","last_unstale":"2024-09-19T09:25:42.991325+0000","last_undegraded":"2024-09-19T09:25:42.991325+0000","last_fullsized":"2024-09-19T09:25:42.991325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:40.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990030+0000","last_change":"2024-09-19T09:25:27.269367+0000","last_active":"2024-09-19T09:25:42.990030+0000","last_peered":"2024-09-19T09:25:42.990030+0000","last_clean":"2024-09-19T09:25:42.990030+0000","last_became_active":"2024-09-19T09:25:27.269011+0000","last_became_peered":"2024-09-19T09:25:27.269011+0000","last_unstale":"2024-09-19T09:25:42.990030+0000","last_undegraded":"2024-09-19T09:25:42.990030+0000","last_fullsized":"2024-09-19T09:25:42.990030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:00.930752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992257+0000","last_change":"2024-09-19T09:25:29.299134+0000","last_active":"2024-09-19T09:25:42.992257+0000","last_peered":"2024-09-19T09:25:42.992257+0000","last_clean":"2024-09-19T09:25:42.992257+0000","last_became_active":"2024-09-19T09:25:29.298648+0000","last_became_peered":"2024-09-19T09:25:29.298648+0000","last_unstale":"2024-09-19T09:25:42.992257+0000","last_undegraded":"2024-09-19T09:25:42.992257+0000","last_fullsized":"2024-09-19T09:25:42.992257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:39.627852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003686+0000","last_change":"2024-09-19T09:25:27.322861+0000","last_active":"2024-09-19T09:25:43.003686+0000","last_peered":"2024-09-19T09:25:43.003686+0000","last_clean":"2024-09-19T09:25:43.003686+0000","last_became_active":"2024-09-19T09:25:27.322560+0000","last_became_peered":"2024-09-19T09:25:27.322560+0000","last_unstale":"2024-09-19T09:25:43.003686+0000","last_undegraded":"2024-09-19T09:25:43.003686+0000","last_fullsized":"2024-09-19T09:25:43.003686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:35.993093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990868+0000","last_change":"2024-09-19T09:25:29.301680+0000","last_active":"2024-09-19T09:25:42.990868+0000","last_peered":"2024-09-19T09:25:42.990868+0000","last_clean":"2024-09-19T09:25:42.990868+0000","last_became_active":"2024-09-19T09:25:29.301331+0000","last_became_peered":"2024-09-19T09:25:29.301331+0000","last_unstale":"2024-09-19T09:25:42.990868+0000","last_undegraded":"2024-09-19T09:25:42.990868+0000","last_fullsized":"2024-09-19T09:25:42.990868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:57.364245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:27.272609+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:27.272169+0000","last_became_peered":"2024-09-19T09:25:27.272169+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.879474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988584+0000","last_change":"2024-09-19T09:25:29.296847+0000","last_active":"2024-09-19T09:25:42.988584+0000","last_peered":"2024-09-19T09:25:42.988584+0000","last_clean":"2024-09-19T09:25:42.988584+0000","last_became_active":"2024-09-19T09:25:29.296465+0000","last_became_peered":"2024-09-19T09:25:29.296465+0000","last_unstale":"2024-09-19T09:25:42.988584+0000","last_undegraded":"2024-09-19T09:25:42.988584+0000","last_fullsized":"2024-09-19T09:25:42.988584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:05:48.553252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.012954+0000","last_change":"2024-09-19T09:25:43.012532+0000","last_active":"2024-09-19T09:25:43.012954+0000","last_peered":"2024-09-19T09:25:43.012954+0000","last_clean":"2024-09-19T09:25:43.012954+0000","last_became_active":"2024-09-19T09:25:42.994996+0000","last_became_peered":"2024-09-19T09:25:42.994996+0000","last_unstale":"2024-09-19T09:25:43.012954+0000","last_undegraded":"2024-09-19T09:25:43.012954+0000","last_fullsized":"2024-09-19T09:25:43.012954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:25.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991072+0000","last_change":"2024-09-19T09:25:29.297633+0000","last_active":"2024-09-19T09:25:42.991072+0000","last_peered":"2024-09-19T09:25:42.991072+0000","last_clean":"2024-09-19T09:25:42.991072+0000","last_became_active":"2024-09-19T09:25:29.297284+0000","last_became_peered":"2024-09-19T09:25:29.297284+0000","last_unstale":"2024-09-19T09:25:42.991072+0000","last_undegraded":"2024-09-19T09:25:42.991072+0000","last_fullsized":"2024-09-19T09:25:42.991072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:50.582546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991172+0000","last_change":"2024-09-19T09:25:27.281778+0000","last_active":"2024-09-19T09:25:42.991172+0000","last_peered":"2024-09-19T09:25:42.991172+0000","last_clean":"2024-09-19T09:25:42.991172+0000","last_became_active":"2024-09-19T09:25:27.281280+0000","last_became_peered":"2024-09-19T09:25:27.281280+0000","last_unstale":"2024-09-19T09:25:42.991172+0000","last_undegraded":"2024-09-19T09:25:42.991172+0000","last_fullsized":"2024-09-19T09:25:42.991172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:46.811968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989808+0000","last_change":"2024-09-19T09:25:29.295663+0000","last_active":"2024-09-19T09:25:42.989808+0000","last_peered":"2024-09-19T09:25:42.989808+0000","last_clean":"2024-09-19T09:25:42.989808+0000","last_became_active":"2024-09-19T09:25:29.295146+0000","last_became_peered":"2024-09-19T09:25:29.295146+0000","last_unstale":"2024-09-19T09:25:42.989808+0000","last_undegraded":"2024-09-19T09:25:42.989808+0000","last_fullsized":"2024-09-19T09:25:42.989808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:36.749339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996010+0000","last_change":"2024-09-19T09:25:29.298004+0000","last_active":"2024-09-19T09:25:42.996010+0000","last_peered":"2024-09-19T09:25:42.996010+0000","last_clean":"2024-09-19T09:25:42.996010+0000","last_became_active":"2024-09-19T09:25:29.297600+0000","last_became_peered":"2024-09-19T09:25:29.297600+0000","last_unstale":"2024-09-19T09:25:42.996010+0000","last_undegraded":"2024-09-19T09:25:42.996010+0000","last_fullsized":"2024-09-19T09:25:42.996010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:09.738615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992323+0000","last_change":"2024-09-19T09:25:27.282876+0000","last_active":"2024-09-19T09:25:42.992323+0000","last_peered":"2024-09-19T09:25:42.992323+0000","last_clean":"2024-09-19T09:25:42.992323+0000","last_became_active":"2024-09-19T09:25:27.282201+0000","last_became_peered":"2024-09-19T09:25:27.282201+0000","last_unstale":"2024-09-19T09:25:42.992323+0000","last_undegraded":"2024-09-19T09:25:42.992323+0000","last_fullsized":"2024-09-19T09:25:42.992323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:08.508662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003650+0000","last_change":"2024-09-19T09:25:29.288379+0000","last_active":"2024-09-19T09:25:43.003650+0000","last_peered":"2024-09-19T09:25:43.003650+0000","last_clean":"2024-09-19T09:25:43.003650+0000","last_became_active":"2024-09-19T09:25:29.287917+0000","last_became_peered":"2024-09-19T09:25:29.287917+0000","last_unstale":"2024-09-19T09:25:43.003650+0000","last_undegraded":"2024-09-19T09:25:43.003650+0000","last_fullsized":"2024-09-19T09:25:43.003650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:03.628233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991076+0000","last_change":"2024-09-19T09:25:27.279551+0000","last_active":"2024-09-19T09:25:42.991076+0000","last_peered":"2024-09-19T09:25:42.991076+0000","last_clean":"2024-09-19T09:25:42.991076+0000","last_became_active":"2024-09-19T09:25:27.278956+0000","last_became_peered":"2024-09-19T09:25:27.278956+0000","last_unstale":"2024-09-19T09:25:42.991076+0000","last_undegraded":"2024-09-19T09:25:42.991076+0000","last_fullsized":"2024-09-19T09:25:42.991076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:17.186453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992345+0000","last_change":"2024-09-19T09:25:29.288657+0000","last_active":"2024-09-19T09:25:42.992345+0000","last_peered":"2024-09-19T09:25:42.992345+0000","last_clean":"2024-09-19T09:25:42.992345+0000","last_became_active":"2024-09-19T09:25:29.288355+0000","last_became_peered":"2024-09-19T09:25:29.288355+0000","last_unstale":"2024-09-19T09:25:42.992345+0000","last_undegraded":"2024-09-19T09:25:42.992345+0000","last_fullsized":"2024-09-19T09:25:42.992345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:46.237558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991315+0000","last_change":"2024-09-19T09:25:27.323312+0000","last_active":"2024-09-19T09:25:42.991315+0000","last_peered":"2024-09-19T09:25:42.991315+0000","last_clean":"2024-09-19T09:25:42.991315+0000","last_became_active":"2024-09-19T09:25:27.322984+0000","last_became_peered":"2024-09-19T09:25:27.322984+0000","last_unstale":"2024-09-19T09:25:42.991315+0000","last_undegraded":"2024-09-19T09:25:42.991315+0000","last_fullsized":"2024-09-19T09:25:42.991315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:48.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991159+0000","last_change":"2024-09-19T09:25:29.301832+0000","last_active":"2024-09-19T09:25:42.991159+0000","last_peered":"2024-09-19T09:25:42.991159+0000","last_clean":"2024-09-19T09:25:42.991159+0000","last_became_active":"2024-09-19T09:25:29.301539+0000","last_became_peered":"2024-09-19T09:25:29.301539+0000","last_unstale":"2024-09-19T09:25:42.991159+0000","last_undegraded":"2024-09-19T09:25:42.991159+0000","last_fullsized":"2024-09-19T09:25:42.991159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:00.244948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:51.006233+0000","last_change":"2024-09-19T09:25:43.009668+0000","last_active":"2024-09-19T09:25:51.006233+0000","last_peered":"2024-09-19T09:25:51.006233+0000","last_clean":"2024-09-19T09:25:51.006233+0000","last_became_active":"2024-09-19T09:25:42.994966+0000","last_became_peered":"2024-09-19T09:25:42.994966+0000","last_unstale":"2024-09-19T09:25:51.006233+0000","last_undegraded":"2024-09-19T09:25:51.006233+0000","last_fullsized":"2024-09-19T09:25:51.006233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:04:05.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54796,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":165,"num_read_kb":143,"num_write":235,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1842,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":164388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612625,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":431057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1068216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005082,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234015,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":430300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1022934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593770,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":901120,"data_stored":430032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822703,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1022498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986227,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":909312,"data_stored":431456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215160,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":476797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":477747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":921600,"data_stored":431371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":47997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:25:55.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:25:55.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:55 smithi016 ceph-mon[30979]: pgmap v287: 97 pgs: 97 active+clean; 630 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 17 op/s; 177 B/s, 1 objects/s recovering 2024-09-19T09:25:55.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:55 smithi016 ceph-mon[30979]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:25:55.283 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:55.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:55 smithi142 ceph-mon[35032]: pgmap v287: 97 pgs: 97 active+clean; 630 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 17 op/s; 177 B/s, 1 objects/s recovering 2024-09-19T09:25:55.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:55 smithi142 ceph-mon[35032]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:25:55.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:55 smithi097 ceph-mon[34864]: pgmap v287: 97 pgs: 97 active+clean; 630 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 17 op/s; 177 B/s, 1 objects/s recovering 2024-09-19T09:25:55.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:55 smithi097 ceph-mon[34864]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:25:55.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.776+0000 7f04333d9640 1 -- 172.21.15.16:0/2886208214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f042c10c170 msgr2=0x7f042c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:55.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.776+0000 7f04333d9640 1 --2- 172.21.15.16:0/2886208214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f042c10c170 0x7f042c10e610 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0424009ca0 tx=0x7f042402a950 comp rx=0 tx=0).stop 2024-09-19T09:25:55.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.776+0000 7f04333d9640 1 -- 172.21.15.16:0/2886208214 shutdown_connections 2024-09-19T09:25:55.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.776+0000 7f04333d9640 1 --2- 172.21.15.16:0/2886208214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042c103660 0x7f042c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:55.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.776+0000 7f04333d9640 1 --2- 172.21.15.16:0/2886208214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f042c10c170 0x7f042c10e610 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.776+0000 7f04333d9640 1 --2- 172.21.15.16:0/2886208214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f042c102d40 0x7f042c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.776+0000 7f04333d9640 1 -- 172.21.15.16:0/2886208214 >> 172.21.15.16:0/2886208214 conn(0x7f042c0fc7a0 msgr2=0x7f042c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.776+0000 7f04333d9640 1 -- 172.21.15.16:0/2886208214 shutdown_connections 2024-09-19T09:25:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.776+0000 7f04333d9640 1 -- 172.21.15.16:0/2886208214 wait complete. 2024-09-19T09:25:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f04333d9640 1 Processor -- start 2024-09-19T09:25:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f04333d9640 1 -- start start 2024-09-19T09:25:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f04333d9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042c102d40 0x7f042c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f04333d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f042c103660 0x7f042c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f04333d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f042c10c170 0x7f042c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f04333d9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f042c111f50 con 0x7f042c103660 2024-09-19T09:25:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f04333d9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f042c111dd0 con 0x7f042c10c170 2024-09-19T09:25:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f04333d9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f042c1120d0 con 0x7f042c102d40 2024-09-19T09:25:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f0430974640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f042c103660 0x7f042c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f0430974640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f042c103660 0x7f042c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49598/0 (socket says 172.21.15.16:49598) 2024-09-19T09:25:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f0430974640 1 -- 172.21.15.16:0/4188045736 learned_addr learned my addr 172.21.15.16:0/4188045736 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f0431175640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042c102d40 0x7f042c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f0431976640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f042c10c170 0x7f042c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f0430974640 1 -- 172.21.15.16:0/4188045736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042c102d40 msgr2=0x7f042c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f0430974640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042c102d40 0x7f042c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f0430974640 1 -- 172.21.15.16:0/4188045736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f042c10c170 msgr2=0x7f042c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f0430974640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f042c10c170 0x7f042c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f0430974640 1 -- 172.21.15.16:0/4188045736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c1a6920 con 0x7f042c103660 2024-09-19T09:25:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f0431175640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042c102d40 0x7f042c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:25:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.777+0000 7f0430974640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f042c103660 0x7f042c100ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f041c00c970 tx=0x7f041c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.778+0000 7f04227fc640 1 -- 172.21.15.16:0/4188045736 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c007bb0 con 0x7f042c103660 2024-09-19T09:25:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.778+0000 7f04227fc640 1 -- 172.21.15.16:0/4188045736 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f041c007d50 con 0x7f042c103660 2024-09-19T09:25:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.778+0000 7f04227fc640 1 -- 172.21.15.16:0/4188045736 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c00f450 con 0x7f042c103660 2024-09-19T09:25:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.778+0000 7f04333d9640 1 -- 172.21.15.16:0/4188045736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042c1a6c10 con 0x7f042c103660 2024-09-19T09:25:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.778+0000 7f04333d9640 1 -- 172.21.15.16:0/4188045736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042c1a7100 con 0x7f042c103660 2024-09-19T09:25:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.779+0000 7f04333d9640 1 -- 172.21.15.16:0/4188045736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f042c103120 con 0x7f042c103660 2024-09-19T09:25:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.780+0000 7f04227fc640 1 -- 172.21.15.16:0/4188045736 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f041c004f40 con 0x7f042c103660 2024-09-19T09:25:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.781+0000 7f04227fc640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f04180a32d0 0x7f04180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.781+0000 7f0431175640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f04180a32d0 0x7f04180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:55.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.781+0000 7f04227fc640 1 -- 172.21.15.16:0/4188045736 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f041c0cfc00 con 0x7f042c103660 2024-09-19T09:25:55.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.781+0000 7f0431175640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f04180a32d0 0x7f04180a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0428005fd0 tx=0x7f042800b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:55.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.784+0000 7f04227fc640 1 -- 172.21.15.16:0/4188045736 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f041c014030 con 0x7f042c103660 2024-09-19T09:25:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.955+0000 7f04333d9640 1 -- 172.21.15.16:0/4188045736 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f042c062630 con 0x7f04180a32d0 2024-09-19T09:25:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.961+0000 7f04227fc640 1 -- 172.21.15.16:0/4188045736 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258772 (secure 0 0 0) 0x7f042c062630 con 0x7f04180a32d0 2024-09-19T09:25:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T09:25:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.966+0000 7f04333d9640 1 -- 172.21.15.16:0/4188045736 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f04180a32d0 msgr2=0x7f04180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.966+0000 7f04333d9640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f04180a32d0 0x7f04180a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0428005fd0 tx=0x7f042800b5f0 comp rx=0 tx=0).stop 2024-09-19T09:25:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.966+0000 7f04333d9640 1 -- 172.21.15.16:0/4188045736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f042c103660 msgr2=0x7f042c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.966+0000 7f04333d9640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f042c103660 0x7f042c100ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f041c00c970 tx=0x7f041c00ce30 comp rx=0 tx=0).stop 2024-09-19T09:25:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.966+0000 7f04333d9640 1 -- 172.21.15.16:0/4188045736 shutdown_connections 2024-09-19T09:25:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.966+0000 7f04333d9640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f042c10c170 0x7f042c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.966+0000 7f04333d9640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f042c103660 0x7f042c100ee0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.966+0000 7f04333d9640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f04180a32d0 0x7f04180a5790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.966+0000 7f04333d9640 1 --2- 172.21.15.16:0/4188045736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042c102d40 0x7f042c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.966+0000 7f04333d9640 1 -- 172.21.15.16:0/4188045736 >> 172.21.15.16:0/4188045736 conn(0x7f042c0fc7a0 msgr2=0x7f042c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.966+0000 7f04333d9640 1 -- 172.21.15.16:0/4188045736 shutdown_connections 2024-09-19T09:25:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:55.966+0000 7f04333d9640 1 -- 172.21.15.16:0/4188045736 wait complete. 2024-09-19T09:25:56.670 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-19T09:25:55.338603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645398,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":291,"num_read_kb":252,"num_write":418,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1182578,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386828,"kb_used_data":12600,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554996,"statfs":{"total":1151940427776,"available":1151544315904,"internally_reserved":0,"allocated":12902400,"data_stored":7081794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46603,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":-3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":107,"num_read_kb":99,"num_write":33,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002851"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988927+0000","last_change":"2024-09-19T09:25:29.294965+0000","last_active":"2024-09-19T09:25:42.988927+0000","last_peered":"2024-09-19T09:25:42.988927+0000","last_clean":"2024-09-19T09:25:42.988927+0000","last_became_active":"2024-09-19T09:25:29.294157+0000","last_became_peered":"2024-09-19T09:25:29.294157+0000","last_unstale":"2024-09-19T09:25:42.988927+0000","last_undegraded":"2024-09-19T09:25:42.988927+0000","last_fullsized":"2024-09-19T09:25:42.988927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:24.643835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992123+0000","last_change":"2024-09-19T09:25:29.298888+0000","last_active":"2024-09-19T09:25:42.992123+0000","last_peered":"2024-09-19T09:25:42.992123+0000","last_clean":"2024-09-19T09:25:42.992123+0000","last_became_active":"2024-09-19T09:25:29.298245+0000","last_became_peered":"2024-09-19T09:25:29.298245+0000","last_unstale":"2024-09-19T09:25:42.992123+0000","last_undegraded":"2024-09-19T09:25:42.992123+0000","last_fullsized":"2024-09-19T09:25:42.992123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:03.162342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992284+0000","last_change":"2024-09-19T09:25:29.292300+0000","last_active":"2024-09-19T09:25:42.992284+0000","last_peered":"2024-09-19T09:25:42.992284+0000","last_clean":"2024-09-19T09:25:42.992284+0000","last_became_active":"2024-09-19T09:25:29.291973+0000","last_became_peered":"2024-09-19T09:25:29.291973+0000","last_unstale":"2024-09-19T09:25:42.992284+0000","last_undegraded":"2024-09-19T09:25:42.992284+0000","last_fullsized":"2024-09-19T09:25:42.992284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:19.672778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990140+0000","last_change":"2024-09-19T09:25:29.305695+0000","last_active":"2024-09-19T09:25:42.990140+0000","last_peered":"2024-09-19T09:25:42.990140+0000","last_clean":"2024-09-19T09:25:42.990140+0000","last_became_active":"2024-09-19T09:25:29.305244+0000","last_became_peered":"2024-09-19T09:25:29.305244+0000","last_unstale":"2024-09-19T09:25:42.990140+0000","last_undegraded":"2024-09-19T09:25:42.990140+0000","last_fullsized":"2024-09-19T09:25:42.990140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:38.054624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991017+0000","last_change":"2024-09-19T09:25:29.295259+0000","last_active":"2024-09-19T09:25:42.991017+0000","last_peered":"2024-09-19T09:25:42.991017+0000","last_clean":"2024-09-19T09:25:42.991017+0000","last_became_active":"2024-09-19T09:25:29.294936+0000","last_became_peered":"2024-09-19T09:25:29.294936+0000","last_unstale":"2024-09-19T09:25:42.991017+0000","last_undegraded":"2024-09-19T09:25:42.991017+0000","last_fullsized":"2024-09-19T09:25:42.991017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:59.193504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986511+0000","last_change":"2024-09-19T09:25:29.299650+0000","last_active":"2024-09-19T09:25:42.986511+0000","last_peered":"2024-09-19T09:25:42.986511+0000","last_clean":"2024-09-19T09:25:42.986511+0000","last_became_active":"2024-09-19T09:25:29.299182+0000","last_became_peered":"2024-09-19T09:25:29.299182+0000","last_unstale":"2024-09-19T09:25:42.986511+0000","last_undegraded":"2024-09-19T09:25:42.986511+0000","last_fullsized":"2024-09-19T09:25:42.986511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:44.637771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989644+0000","last_change":"2024-09-19T09:25:29.302501+0000","last_active":"2024-09-19T09:25:42.989644+0000","last_peered":"2024-09-19T09:25:42.989644+0000","last_clean":"2024-09-19T09:25:42.989644+0000","last_became_active":"2024-09-19T09:25:29.301419+0000","last_became_peered":"2024-09-19T09:25:29.301419+0000","last_unstale":"2024-09-19T09:25:42.989644+0000","last_undegraded":"2024-09-19T09:25:42.989644+0000","last_fullsized":"2024-09-19T09:25:42.989644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:50.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990840+0000","last_change":"2024-09-19T09:25:29.300212+0000","last_active":"2024-09-19T09:25:42.990840+0000","last_peered":"2024-09-19T09:25:42.990840+0000","last_clean":"2024-09-19T09:25:42.990840+0000","last_became_active":"2024-09-19T09:25:29.299873+0000","last_became_peered":"2024-09-19T09:25:29.299873+0000","last_unstale":"2024-09-19T09:25:42.990840+0000","last_undegraded":"2024-09-19T09:25:42.990840+0000","last_fullsized":"2024-09-19T09:25:42.990840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:34.620314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.004117+0000","last_change":"2024-09-19T09:25:29.290214+0000","last_active":"2024-09-19T09:25:43.004117+0000","last_peered":"2024-09-19T09:25:43.004117+0000","last_clean":"2024-09-19T09:25:43.004117+0000","last_became_active":"2024-09-19T09:25:29.289707+0000","last_became_peered":"2024-09-19T09:25:29.289707+0000","last_unstale":"2024-09-19T09:25:43.004117+0000","last_undegraded":"2024-09-19T09:25:43.004117+0000","last_fullsized":"2024-09-19T09:25:43.004117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:16.341340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.987005+0000","last_change":"2024-09-19T09:25:29.301774+0000","last_active":"2024-09-19T09:25:42.987005+0000","last_peered":"2024-09-19T09:25:42.987005+0000","last_clean":"2024-09-19T09:25:42.987005+0000","last_became_active":"2024-09-19T09:25:29.301398+0000","last_became_peered":"2024-09-19T09:25:29.301398+0000","last_unstale":"2024-09-19T09:25:42.987005+0000","last_undegraded":"2024-09-19T09:25:42.987005+0000","last_fullsized":"2024-09-19T09:25:42.987005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:36.058038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989844+0000","last_change":"2024-09-19T09:25:29.301475+0000","last_active":"2024-09-19T09:25:42.989844+0000","last_peered":"2024-09-19T09:25:42.989844+0000","last_clean":"2024-09-19T09:25:42.989844+0000","last_became_active":"2024-09-19T09:25:29.300908+0000","last_became_peered":"2024-09-19T09:25:29.300908+0000","last_unstale":"2024-09-19T09:25:42.989844+0000","last_undegraded":"2024-09-19T09:25:42.989844+0000","last_fullsized":"2024-09-19T09:25:42.989844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:44.774127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994395+0000","last_change":"2024-09-19T09:25:42.994395+0000","last_active":"2024-09-19T09:25:42.994395+0000","last_peered":"2024-09-19T09:25:42.994395+0000","last_clean":"2024-09-19T09:25:42.994395+0000","last_became_active":"2024-09-19T09:25:42.993992+0000","last_became_peered":"2024-09-19T09:25:42.993992+0000","last_unstale":"2024-09-19T09:25:42.994395+0000","last_undegraded":"2024-09-19T09:25:42.994395+0000","last_fullsized":"2024-09-19T09:25:42.994395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.486069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989979+0000","last_change":"2024-09-19T09:25:42.989979+0000","last_active":"2024-09-19T09:25:42.989979+0000","last_peered":"2024-09-19T09:25:42.989979+0000","last_clean":"2024-09-19T09:25:42.989979+0000","last_became_active":"2024-09-19T09:25:42.989578+0000","last_became_peered":"2024-09-19T09:25:42.989578+0000","last_unstale":"2024-09-19T09:25:42.989979+0000","last_undegraded":"2024-09-19T09:25:42.989979+0000","last_fullsized":"2024-09-19T09:25:42.989979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:04.583023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992064+0000","last_change":"2024-09-19T09:25:29.299917+0000","last_active":"2024-09-19T09:25:42.992064+0000","last_peered":"2024-09-19T09:25:42.992064+0000","last_clean":"2024-09-19T09:25:42.992064+0000","last_became_active":"2024-09-19T09:25:29.299571+0000","last_became_peered":"2024-09-19T09:25:29.299571+0000","last_unstale":"2024-09-19T09:25:42.992064+0000","last_undegraded":"2024-09-19T09:25:42.992064+0000","last_fullsized":"2024-09-19T09:25:42.992064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:25.142252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991982+0000","last_change":"2024-09-19T09:25:29.299713+0000","last_active":"2024-09-19T09:25:42.991982+0000","last_peered":"2024-09-19T09:25:42.991982+0000","last_clean":"2024-09-19T09:25:42.991982+0000","last_became_active":"2024-09-19T09:25:29.299234+0000","last_became_peered":"2024-09-19T09:25:29.299234+0000","last_unstale":"2024-09-19T09:25:42.991982+0000","last_undegraded":"2024-09-19T09:25:42.991982+0000","last_fullsized":"2024-09-19T09:25:42.991982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:34.714714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003535+0000","last_change":"2024-09-19T09:25:29.293010+0000","last_active":"2024-09-19T09:25:43.003535+0000","last_peered":"2024-09-19T09:25:43.003535+0000","last_clean":"2024-09-19T09:25:43.003535+0000","last_became_active":"2024-09-19T09:25:29.292645+0000","last_became_peered":"2024-09-19T09:25:29.292645+0000","last_unstale":"2024-09-19T09:25:43.003535+0000","last_undegraded":"2024-09-19T09:25:43.003535+0000","last_fullsized":"2024-09-19T09:25:43.003535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:24.749075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989031+0000","last_change":"2024-09-19T09:25:29.294530+0000","last_active":"2024-09-19T09:25:42.989031+0000","last_peered":"2024-09-19T09:25:42.989031+0000","last_clean":"2024-09-19T09:25:42.989031+0000","last_became_active":"2024-09-19T09:25:29.293699+0000","last_became_peered":"2024-09-19T09:25:29.293699+0000","last_unstale":"2024-09-19T09:25:42.989031+0000","last_undegraded":"2024-09-19T09:25:42.989031+0000","last_fullsized":"2024-09-19T09:25:42.989031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:53.491258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988781+0000","last_change":"2024-09-19T09:25:29.290600+0000","last_active":"2024-09-19T09:25:42.988781+0000","last_peered":"2024-09-19T09:25:42.988781+0000","last_clean":"2024-09-19T09:25:42.988781+0000","last_became_active":"2024-09-19T09:25:29.290305+0000","last_became_peered":"2024-09-19T09:25:29.290305+0000","last_unstale":"2024-09-19T09:25:42.988781+0000","last_undegraded":"2024-09-19T09:25:42.988781+0000","last_fullsized":"2024-09-19T09:25:42.988781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:42.809659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.993154+0000","last_change":"2024-09-19T09:25:29.300591+0000","last_active":"2024-09-19T09:25:42.993154+0000","last_peered":"2024-09-19T09:25:42.993154+0000","last_clean":"2024-09-19T09:25:42.993154+0000","last_became_active":"2024-09-19T09:25:29.300251+0000","last_became_peered":"2024-09-19T09:25:29.300251+0000","last_unstale":"2024-09-19T09:25:42.993154+0000","last_undegraded":"2024-09-19T09:25:42.993154+0000","last_fullsized":"2024-09-19T09:25:42.993154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:08.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988507+0000","last_change":"2024-09-19T09:25:29.298452+0000","last_active":"2024-09-19T09:25:42.988507+0000","last_peered":"2024-09-19T09:25:42.988507+0000","last_clean":"2024-09-19T09:25:42.988507+0000","last_became_active":"2024-09-19T09:25:29.298137+0000","last_became_peered":"2024-09-19T09:25:29.298137+0000","last_unstale":"2024-09-19T09:25:42.988507+0000","last_undegraded":"2024-09-19T09:25:42.988507+0000","last_fullsized":"2024-09-19T09:25:42.988507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:11.716228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985825+0000","last_change":"2024-09-19T09:25:29.294278+0000","last_active":"2024-09-19T09:25:42.985825+0000","last_peered":"2024-09-19T09:25:42.985825+0000","last_clean":"2024-09-19T09:25:42.985825+0000","last_became_active":"2024-09-19T09:25:29.293972+0000","last_became_peered":"2024-09-19T09:25:29.293972+0000","last_unstale":"2024-09-19T09:25:42.985825+0000","last_undegraded":"2024-09-19T09:25:42.985825+0000","last_fullsized":"2024-09-19T09:25:42.985825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:27.923839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003737+0000","last_change":"2024-09-19T09:25:29.294000+0000","last_active":"2024-09-19T09:25:43.003737+0000","last_peered":"2024-09-19T09:25:43.003737+0000","last_clean":"2024-09-19T09:25:43.003737+0000","last_became_active":"2024-09-19T09:25:29.293486+0000","last_became_peered":"2024-09-19T09:25:29.293486+0000","last_unstale":"2024-09-19T09:25:43.003737+0000","last_undegraded":"2024-09-19T09:25:43.003737+0000","last_fullsized":"2024-09-19T09:25:43.003737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:44.053620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994951+0000","last_change":"2024-09-19T09:25:42.994951+0000","last_active":"2024-09-19T09:25:42.994951+0000","last_peered":"2024-09-19T09:25:42.994951+0000","last_clean":"2024-09-19T09:25:42.994951+0000","last_became_active":"2024-09-19T09:25:42.994588+0000","last_became_peered":"2024-09-19T09:25:42.994588+0000","last_unstale":"2024-09-19T09:25:42.994951+0000","last_undegraded":"2024-09-19T09:25:42.994951+0000","last_fullsized":"2024-09-19T09:25:42.994951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:10.141169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995128+0000","last_change":"2024-09-19T09:25:29.286395+0000","last_active":"2024-09-19T09:25:42.995128+0000","last_peered":"2024-09-19T09:25:42.995128+0000","last_clean":"2024-09-19T09:25:42.995128+0000","last_became_active":"2024-09-19T09:25:29.286048+0000","last_became_peered":"2024-09-19T09:25:29.286048+0000","last_unstale":"2024-09-19T09:25:42.995128+0000","last_undegraded":"2024-09-19T09:25:42.995128+0000","last_fullsized":"2024-09-19T09:25:42.995128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:43.121166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991295+0000","last_change":"2024-09-19T09:25:29.298458+0000","last_active":"2024-09-19T09:25:42.991295+0000","last_peered":"2024-09-19T09:25:42.991295+0000","last_clean":"2024-09-19T09:25:42.991295+0000","last_became_active":"2024-09-19T09:25:29.297993+0000","last_became_peered":"2024-09-19T09:25:29.297993+0000","last_unstale":"2024-09-19T09:25:42.991295+0000","last_undegraded":"2024-09-19T09:25:42.991295+0000","last_fullsized":"2024-09-19T09:25:42.991295+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:44.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992762+0000","last_change":"2024-09-19T09:25:29.297167+0000","last_active":"2024-09-19T09:25:42.992762+0000","last_peered":"2024-09-19T09:25:42.992762+0000","last_clean":"2024-09-19T09:25:42.992762+0000","last_became_active":"2024-09-19T09:25:29.296777+0000","last_became_peered":"2024-09-19T09:25:29.296777+0000","last_unstale":"2024-09-19T09:25:42.992762+0000","last_undegraded":"2024-09-19T09:25:42.992762+0000","last_fullsized":"2024-09-19T09:25:42.992762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:40.581643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988833+0000","last_change":"2024-09-19T09:25:29.296169+0000","last_active":"2024-09-19T09:25:42.988833+0000","last_peered":"2024-09-19T09:25:42.988833+0000","last_clean":"2024-09-19T09:25:42.988833+0000","last_became_active":"2024-09-19T09:25:29.295806+0000","last_became_peered":"2024-09-19T09:25:29.295806+0000","last_unstale":"2024-09-19T09:25:42.988833+0000","last_undegraded":"2024-09-19T09:25:42.988833+0000","last_fullsized":"2024-09-19T09:25:42.988833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:39.639884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985866+0000","last_change":"2024-09-19T09:25:29.297960+0000","last_active":"2024-09-19T09:25:42.985866+0000","last_peered":"2024-09-19T09:25:42.985866+0000","last_clean":"2024-09-19T09:25:42.985866+0000","last_became_active":"2024-09-19T09:25:29.297577+0000","last_became_peered":"2024-09-19T09:25:29.297577+0000","last_unstale":"2024-09-19T09:25:42.985866+0000","last_undegraded":"2024-09-19T09:25:42.985866+0000","last_fullsized":"2024-09-19T09:25:42.985866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:27.035504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992420+0000","last_change":"2024-09-19T09:25:29.297896+0000","last_active":"2024-09-19T09:25:42.992420+0000","last_peered":"2024-09-19T09:25:42.992420+0000","last_clean":"2024-09-19T09:25:42.992420+0000","last_became_active":"2024-09-19T09:25:29.297594+0000","last_became_peered":"2024-09-19T09:25:29.297594+0000","last_unstale":"2024-09-19T09:25:42.992420+0000","last_undegraded":"2024-09-19T09:25:42.992420+0000","last_fullsized":"2024-09-19T09:25:42.992420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.366325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992205+0000","last_change":"2024-09-19T09:25:29.301851+0000","last_active":"2024-09-19T09:25:42.992205+0000","last_peered":"2024-09-19T09:25:42.992205+0000","last_clean":"2024-09-19T09:25:42.992205+0000","last_became_active":"2024-09-19T09:25:29.299282+0000","last_became_peered":"2024-09-19T09:25:29.299282+0000","last_unstale":"2024-09-19T09:25:42.992205+0000","last_undegraded":"2024-09-19T09:25:42.992205+0000","last_fullsized":"2024-09-19T09:25:42.992205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:00.382114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992658+0000","last_change":"2024-09-19T09:25:29.303316+0000","last_active":"2024-09-19T09:25:42.992658+0000","last_peered":"2024-09-19T09:25:42.992658+0000","last_clean":"2024-09-19T09:25:42.992658+0000","last_became_active":"2024-09-19T09:25:29.302994+0000","last_became_peered":"2024-09-19T09:25:29.302994+0000","last_unstale":"2024-09-19T09:25:42.992658+0000","last_undegraded":"2024-09-19T09:25:42.992658+0000","last_fullsized":"2024-09-19T09:25:42.992658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:46.233991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988501+0000","last_change":"2024-09-19T09:25:29.293004+0000","last_active":"2024-09-19T09:25:42.988501+0000","last_peered":"2024-09-19T09:25:42.988501+0000","last_clean":"2024-09-19T09:25:42.988501+0000","last_became_active":"2024-09-19T09:25:29.292657+0000","last_became_peered":"2024-09-19T09:25:29.292657+0000","last_unstale":"2024-09-19T09:25:42.988501+0000","last_undegraded":"2024-09-19T09:25:42.988501+0000","last_fullsized":"2024-09-19T09:25:42.988501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:50.966477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991936+0000","last_change":"2024-09-19T09:25:29.300434+0000","last_active":"2024-09-19T09:25:42.991936+0000","last_peered":"2024-09-19T09:25:42.991936+0000","last_clean":"2024-09-19T09:25:42.991936+0000","last_became_active":"2024-09-19T09:25:29.300053+0000","last_became_peered":"2024-09-19T09:25:29.300053+0000","last_unstale":"2024-09-19T09:25:42.991936+0000","last_undegraded":"2024-09-19T09:25:42.991936+0000","last_fullsized":"2024-09-19T09:25:42.991936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:13.283298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":46,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:51.982309+0000","last_change":"2024-09-19T09:25:27.280792+0000","last_active":"2024-09-19T09:25:51.982309+0000","last_peered":"2024-09-19T09:25:51.982309+0000","last_clean":"2024-09-19T09:25:51.982309+0000","last_became_active":"2024-09-19T09:25:27.280443+0000","last_became_peered":"2024-09-19T09:25:27.280443+0000","last_unstale":"2024-09-19T09:25:51.982309+0000","last_undegraded":"2024-09-19T09:25:51.982309+0000","last_fullsized":"2024-09-19T09:25:51.982309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.599512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.997114+0000","last_change":"2024-09-19T09:25:29.298975+0000","last_active":"2024-09-19T09:25:42.997114+0000","last_peered":"2024-09-19T09:25:42.997114+0000","last_clean":"2024-09-19T09:25:42.997114+0000","last_became_active":"2024-09-19T09:25:29.298476+0000","last_became_peered":"2024-09-19T09:25:29.298476+0000","last_unstale":"2024-09-19T09:25:42.997114+0000","last_undegraded":"2024-09-19T09:25:42.997114+0000","last_fullsized":"2024-09-19T09:25:42.997114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:09.658894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986623+0000","last_change":"2024-09-19T09:25:27.278257+0000","last_active":"2024-09-19T09:25:42.986623+0000","last_peered":"2024-09-19T09:25:42.986623+0000","last_clean":"2024-09-19T09:25:42.986623+0000","last_became_active":"2024-09-19T09:25:27.277897+0000","last_became_peered":"2024-09-19T09:25:27.277897+0000","last_unstale":"2024-09-19T09:25:42.986623+0000","last_undegraded":"2024-09-19T09:25:42.986623+0000","last_fullsized":"2024-09-19T09:25:42.986623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:47.550314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985617+0000","last_change":"2024-09-19T09:25:27.272215+0000","last_active":"2024-09-19T09:25:42.985617+0000","last_peered":"2024-09-19T09:25:42.985617+0000","last_clean":"2024-09-19T09:25:42.985617+0000","last_became_active":"2024-09-19T09:25:27.271821+0000","last_became_peered":"2024-09-19T09:25:27.271821+0000","last_unstale":"2024-09-19T09:25:42.985617+0000","last_undegraded":"2024-09-19T09:25:42.985617+0000","last_fullsized":"2024-09-19T09:25:42.985617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:30.615730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990922+0000","last_change":"2024-09-19T09:25:29.289347+0000","last_active":"2024-09-19T09:25:42.990922+0000","last_peered":"2024-09-19T09:25:42.990922+0000","last_clean":"2024-09-19T09:25:42.990922+0000","last_became_active":"2024-09-19T09:25:29.289015+0000","last_became_peered":"2024-09-19T09:25:29.289015+0000","last_unstale":"2024-09-19T09:25:42.990922+0000","last_undegraded":"2024-09-19T09:25:42.990922+0000","last_fullsized":"2024-09-19T09:25:42.990922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:18.419334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995432+0000","last_change":"2024-09-19T09:25:27.277342+0000","last_active":"2024-09-19T09:25:42.995432+0000","last_peered":"2024-09-19T09:25:42.995432+0000","last_clean":"2024-09-19T09:25:42.995432+0000","last_became_active":"2024-09-19T09:25:27.277016+0000","last_became_peered":"2024-09-19T09:25:27.277016+0000","last_unstale":"2024-09-19T09:25:42.995432+0000","last_undegraded":"2024-09-19T09:25:42.995432+0000","last_fullsized":"2024-09-19T09:25:42.995432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:56.736156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988740+0000","last_change":"2024-09-19T09:25:29.290924+0000","last_active":"2024-09-19T09:25:42.988740+0000","last_peered":"2024-09-19T09:25:42.988740+0000","last_clean":"2024-09-19T09:25:42.988740+0000","last_became_active":"2024-09-19T09:25:29.290482+0000","last_became_peered":"2024-09-19T09:25:29.290482+0000","last_unstale":"2024-09-19T09:25:42.988740+0000","last_undegraded":"2024-09-19T09:25:42.988740+0000","last_fullsized":"2024-09-19T09:25:42.988740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:16.265040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:50.980337+0000","last_change":"2024-09-19T09:25:27.277604+0000","last_active":"2024-09-19T09:25:50.980337+0000","last_peered":"2024-09-19T09:25:50.980337+0000","last_clean":"2024-09-19T09:25:50.980337+0000","last_became_active":"2024-09-19T09:25:27.277271+0000","last_became_peered":"2024-09-19T09:25:27.277271+0000","last_unstale":"2024-09-19T09:25:50.980337+0000","last_undegraded":"2024-09-19T09:25:50.980337+0000","last_fullsized":"2024-09-19T09:25:50.980337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:32.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.004044+0000","last_change":"2024-09-19T09:25:29.290537+0000","last_active":"2024-09-19T09:25:43.004044+0000","last_peered":"2024-09-19T09:25:43.004044+0000","last_clean":"2024-09-19T09:25:43.004044+0000","last_became_active":"2024-09-19T09:25:29.287984+0000","last_became_peered":"2024-09-19T09:25:29.287984+0000","last_unstale":"2024-09-19T09:25:43.004044+0000","last_undegraded":"2024-09-19T09:25:43.004044+0000","last_fullsized":"2024-09-19T09:25:43.004044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:26.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990018+0000","last_change":"2024-09-19T09:25:27.278560+0000","last_active":"2024-09-19T09:25:42.990018+0000","last_peered":"2024-09-19T09:25:42.990018+0000","last_clean":"2024-09-19T09:25:42.990018+0000","last_became_active":"2024-09-19T09:25:27.277950+0000","last_became_peered":"2024-09-19T09:25:27.277950+0000","last_unstale":"2024-09-19T09:25:42.990018+0000","last_undegraded":"2024-09-19T09:25:42.990018+0000","last_fullsized":"2024-09-19T09:25:42.990018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:28.387725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992413+0000","last_change":"2024-09-19T09:25:29.298897+0000","last_active":"2024-09-19T09:25:42.992413+0000","last_peered":"2024-09-19T09:25:42.992413+0000","last_clean":"2024-09-19T09:25:42.992413+0000","last_became_active":"2024-09-19T09:25:29.298617+0000","last_became_peered":"2024-09-19T09:25:29.298617+0000","last_unstale":"2024-09-19T09:25:42.992413+0000","last_undegraded":"2024-09-19T09:25:42.992413+0000","last_fullsized":"2024-09-19T09:25:42.992413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:53.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994870+0000","last_change":"2024-09-19T09:25:27.324871+0000","last_active":"2024-09-19T09:25:42.994870+0000","last_peered":"2024-09-19T09:25:42.994870+0000","last_clean":"2024-09-19T09:25:42.994870+0000","last_became_active":"2024-09-19T09:25:27.324471+0000","last_became_peered":"2024-09-19T09:25:27.324471+0000","last_unstale":"2024-09-19T09:25:42.994870+0000","last_undegraded":"2024-09-19T09:25:42.994870+0000","last_fullsized":"2024-09-19T09:25:42.994870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:35.936413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.004242+0000","last_change":"2024-09-19T09:25:29.298332+0000","last_active":"2024-09-19T09:25:43.004242+0000","last_peered":"2024-09-19T09:25:43.004242+0000","last_clean":"2024-09-19T09:25:43.004242+0000","last_became_active":"2024-09-19T09:25:29.298006+0000","last_became_peered":"2024-09-19T09:25:29.298006+0000","last_unstale":"2024-09-19T09:25:43.004242+0000","last_undegraded":"2024-09-19T09:25:43.004242+0000","last_fullsized":"2024-09-19T09:25:43.004242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:02.621821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'24","reported_seq":78,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:50.987443+0000","last_change":"2024-09-19T09:25:27.279528+0000","last_active":"2024-09-19T09:25:50.987443+0000","last_peered":"2024-09-19T09:25:50.987443+0000","last_clean":"2024-09-19T09:25:50.987443+0000","last_became_active":"2024-09-19T09:25:27.279149+0000","last_became_peered":"2024-09-19T09:25:27.279149+0000","last_unstale":"2024-09-19T09:25:50.987443+0000","last_undegraded":"2024-09-19T09:25:50.987443+0000","last_fullsized":"2024-09-19T09:25:50.987443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:47.030736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":61,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991057+0000","last_change":"2024-09-19T09:25:29.297538+0000","last_active":"2024-09-19T09:25:42.991057+0000","last_peered":"2024-09-19T09:25:42.991057+0000","last_clean":"2024-09-19T09:25:42.991057+0000","last_became_active":"2024-09-19T09:25:29.297181+0000","last_became_peered":"2024-09-19T09:25:29.297181+0000","last_unstale":"2024-09-19T09:25:42.991057+0000","last_undegraded":"2024-09-19T09:25:42.991057+0000","last_fullsized":"2024-09-19T09:25:42.991057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:53.135438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992860+0000","last_change":"2024-09-19T09:25:27.277090+0000","last_active":"2024-09-19T09:25:42.992860+0000","last_peered":"2024-09-19T09:25:42.992860+0000","last_clean":"2024-09-19T09:25:42.992860+0000","last_became_active":"2024-09-19T09:25:27.276718+0000","last_became_peered":"2024-09-19T09:25:27.276718+0000","last_unstale":"2024-09-19T09:25:42.992860+0000","last_undegraded":"2024-09-19T09:25:42.992860+0000","last_fullsized":"2024-09-19T09:25:42.992860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:25.664261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985642+0000","last_change":"2024-09-19T09:25:29.294393+0000","last_active":"2024-09-19T09:25:42.985642+0000","last_peered":"2024-09-19T09:25:42.985642+0000","last_clean":"2024-09-19T09:25:42.985642+0000","last_became_active":"2024-09-19T09:25:29.294099+0000","last_became_peered":"2024-09-19T09:25:29.294099+0000","last_unstale":"2024-09-19T09:25:42.985642+0000","last_undegraded":"2024-09-19T09:25:42.985642+0000","last_fullsized":"2024-09-19T09:25:42.985642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:17.333736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995398+0000","last_change":"2024-09-19T09:25:27.278593+0000","last_active":"2024-09-19T09:25:42.995398+0000","last_peered":"2024-09-19T09:25:42.995398+0000","last_clean":"2024-09-19T09:25:42.995398+0000","last_became_active":"2024-09-19T09:25:27.278232+0000","last_became_peered":"2024-09-19T09:25:27.278232+0000","last_unstale":"2024-09-19T09:25:42.995398+0000","last_undegraded":"2024-09-19T09:25:42.995398+0000","last_fullsized":"2024-09-19T09:25:42.995398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:05.852671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989855+0000","last_change":"2024-09-19T09:25:29.300024+0000","last_active":"2024-09-19T09:25:42.989855+0000","last_peered":"2024-09-19T09:25:42.989855+0000","last_clean":"2024-09-19T09:25:42.989855+0000","last_became_active":"2024-09-19T09:25:29.299683+0000","last_became_peered":"2024-09-19T09:25:29.299683+0000","last_unstale":"2024-09-19T09:25:42.989855+0000","last_undegraded":"2024-09-19T09:25:42.989855+0000","last_fullsized":"2024-09-19T09:25:42.989855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:42.249399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:50.981839+0000","last_change":"2024-09-19T09:25:27.272739+0000","last_active":"2024-09-19T09:25:50.981839+0000","last_peered":"2024-09-19T09:25:50.981839+0000","last_clean":"2024-09-19T09:25:50.981839+0000","last_became_active":"2024-09-19T09:25:27.272354+0000","last_became_peered":"2024-09-19T09:25:27.272354+0000","last_unstale":"2024-09-19T09:25:50.981839+0000","last_undegraded":"2024-09-19T09:25:50.981839+0000","last_fullsized":"2024-09-19T09:25:50.981839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:27.863840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989616+0000","last_change":"2024-09-19T09:25:29.303460+0000","last_active":"2024-09-19T09:25:42.989616+0000","last_peered":"2024-09-19T09:25:42.989616+0000","last_clean":"2024-09-19T09:25:42.989616+0000","last_became_active":"2024-09-19T09:25:29.303127+0000","last_became_peered":"2024-09-19T09:25:29.303127+0000","last_unstale":"2024-09-19T09:25:42.989616+0000","last_undegraded":"2024-09-19T09:25:42.989616+0000","last_fullsized":"2024-09-19T09:25:42.989616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:35.313924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990876+0000","last_change":"2024-09-19T09:25:27.323238+0000","last_active":"2024-09-19T09:25:42.990876+0000","last_peered":"2024-09-19T09:25:42.990876+0000","last_clean":"2024-09-19T09:25:42.990876+0000","last_became_active":"2024-09-19T09:25:27.322868+0000","last_became_peered":"2024-09-19T09:25:27.322868+0000","last_unstale":"2024-09-19T09:25:42.990876+0000","last_undegraded":"2024-09-19T09:25:42.990876+0000","last_fullsized":"2024-09-19T09:25:42.990876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:22.890280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.005569+0000","last_change":"2024-09-19T09:25:43.005569+0000","last_active":"2024-09-19T09:25:43.005569+0000","last_peered":"2024-09-19T09:25:43.005569+0000","last_clean":"2024-09-19T09:25:43.005569+0000","last_became_active":"2024-09-19T09:25:43.005143+0000","last_became_peered":"2024-09-19T09:25:43.005143+0000","last_unstale":"2024-09-19T09:25:43.005569+0000","last_undegraded":"2024-09-19T09:25:43.005569+0000","last_fullsized":"2024-09-19T09:25:43.005569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:26.297013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991438+0000","last_change":"2024-09-19T09:25:27.282737+0000","last_active":"2024-09-19T09:25:42.991438+0000","last_peered":"2024-09-19T09:25:42.991438+0000","last_clean":"2024-09-19T09:25:42.991438+0000","last_became_active":"2024-09-19T09:25:27.281785+0000","last_became_peered":"2024-09-19T09:25:27.281785+0000","last_unstale":"2024-09-19T09:25:42.991438+0000","last_undegraded":"2024-09-19T09:25:42.991438+0000","last_fullsized":"2024-09-19T09:25:42.991438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:41.552206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988791+0000","last_change":"2024-09-19T09:25:29.300168+0000","last_active":"2024-09-19T09:25:42.988791+0000","last_peered":"2024-09-19T09:25:42.988791+0000","last_clean":"2024-09-19T09:25:42.988791+0000","last_became_active":"2024-09-19T09:25:29.299811+0000","last_became_peered":"2024-09-19T09:25:29.299811+0000","last_unstale":"2024-09-19T09:25:42.988791+0000","last_undegraded":"2024-09-19T09:25:42.988791+0000","last_fullsized":"2024-09-19T09:25:42.988791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:06.691259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985783+0000","last_change":"2024-09-19T09:25:27.278433+0000","last_active":"2024-09-19T09:25:42.985783+0000","last_peered":"2024-09-19T09:25:42.985783+0000","last_clean":"2024-09-19T09:25:42.985783+0000","last_became_active":"2024-09-19T09:25:27.278076+0000","last_became_peered":"2024-09-19T09:25:27.278076+0000","last_unstale":"2024-09-19T09:25:42.985783+0000","last_undegraded":"2024-09-19T09:25:42.985783+0000","last_fullsized":"2024-09-19T09:25:42.985783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:53.799265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990338+0000","last_change":"2024-09-19T09:25:29.295650+0000","last_active":"2024-09-19T09:25:42.990338+0000","last_peered":"2024-09-19T09:25:42.990338+0000","last_clean":"2024-09-19T09:25:42.990338+0000","last_became_active":"2024-09-19T09:25:29.295277+0000","last_became_peered":"2024-09-19T09:25:29.295277+0000","last_unstale":"2024-09-19T09:25:42.990338+0000","last_undegraded":"2024-09-19T09:25:42.990338+0000","last_fullsized":"2024-09-19T09:25:42.990338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:57.690699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996123+0000","last_change":"2024-09-19T09:25:27.279239+0000","last_active":"2024-09-19T09:25:42.996123+0000","last_peered":"2024-09-19T09:25:42.996123+0000","last_clean":"2024-09-19T09:25:42.996123+0000","last_became_active":"2024-09-19T09:25:27.278868+0000","last_became_peered":"2024-09-19T09:25:27.278868+0000","last_unstale":"2024-09-19T09:25:42.996123+0000","last_undegraded":"2024-09-19T09:25:42.996123+0000","last_fullsized":"2024-09-19T09:25:42.996123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:41.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990429+0000","last_change":"2024-09-19T09:25:29.300833+0000","last_active":"2024-09-19T09:25:42.990429+0000","last_peered":"2024-09-19T09:25:42.990429+0000","last_clean":"2024-09-19T09:25:42.990429+0000","last_became_active":"2024-09-19T09:25:29.300472+0000","last_became_peered":"2024-09-19T09:25:29.300472+0000","last_unstale":"2024-09-19T09:25:42.990429+0000","last_undegraded":"2024-09-19T09:25:42.990429+0000","last_fullsized":"2024-09-19T09:25:42.990429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:50.591276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:51.008075+0000","last_change":"2024-09-19T09:25:43.001722+0000","last_active":"2024-09-19T09:25:51.008075+0000","last_peered":"2024-09-19T09:25:51.008075+0000","last_clean":"2024-09-19T09:25:51.008075+0000","last_became_active":"2024-09-19T09:25:42.992796+0000","last_became_peered":"2024-09-19T09:25:42.992796+0000","last_unstale":"2024-09-19T09:25:51.008075+0000","last_undegraded":"2024-09-19T09:25:51.008075+0000","last_fullsized":"2024-09-19T09:25:51.008075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:59.639010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990231+0000","last_change":"2024-09-19T09:25:29.295906+0000","last_active":"2024-09-19T09:25:42.990231+0000","last_peered":"2024-09-19T09:25:42.990231+0000","last_clean":"2024-09-19T09:25:42.990231+0000","last_became_active":"2024-09-19T09:25:29.295565+0000","last_became_peered":"2024-09-19T09:25:29.295565+0000","last_unstale":"2024-09-19T09:25:42.990231+0000","last_undegraded":"2024-09-19T09:25:42.990231+0000","last_fullsized":"2024-09-19T09:25:42.990231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:24.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":334,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989806+0000","last_change":"2024-09-19T09:24:19.256300+0000","last_active":"2024-09-19T09:25:42.989806+0000","last_peered":"2024-09-19T09:25:42.989806+0000","last_clean":"2024-09-19T09:25:42.989806+0000","last_became_active":"2024-09-19T09:24:19.255833+0000","last_became_peered":"2024-09-19T09:24:19.255833+0000","last_unstale":"2024-09-19T09:25:42.989806+0000","last_undegraded":"2024-09-19T09:25:42.989806+0000","last_fullsized":"2024-09-19T09:25:42.989806+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_clean_scrub_stamp":"2024-09-19T09:20:57.798838+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:14:33.343748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989637+0000","last_change":"2024-09-19T09:25:27.272090+0000","last_active":"2024-09-19T09:25:42.989637+0000","last_peered":"2024-09-19T09:25:42.989637+0000","last_clean":"2024-09-19T09:25:42.989637+0000","last_became_active":"2024-09-19T09:25:27.271651+0000","last_became_peered":"2024-09-19T09:25:27.271651+0000","last_unstale":"2024-09-19T09:25:42.989637+0000","last_undegraded":"2024-09-19T09:25:42.989637+0000","last_fullsized":"2024-09-19T09:25:42.989637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:45.847082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:29.300718+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:29.299460+0000","last_became_peered":"2024-09-19T09:25:29.299460+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:43.063909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003299+0000","last_change":"2024-09-19T09:25:43.003299+0000","last_active":"2024-09-19T09:25:43.003299+0000","last_peered":"2024-09-19T09:25:43.003299+0000","last_clean":"2024-09-19T09:25:43.003299+0000","last_became_active":"2024-09-19T09:25:42.999686+0000","last_became_peered":"2024-09-19T09:25:42.999686+0000","last_unstale":"2024-09-19T09:25:43.003299+0000","last_undegraded":"2024-09-19T09:25:43.003299+0000","last_fullsized":"2024-09-19T09:25:43.003299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:05.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985606+0000","last_change":"2024-09-19T09:25:29.297833+0000","last_active":"2024-09-19T09:25:42.985606+0000","last_peered":"2024-09-19T09:25:42.985606+0000","last_clean":"2024-09-19T09:25:42.985606+0000","last_became_active":"2024-09-19T09:25:29.297534+0000","last_became_peered":"2024-09-19T09:25:29.297534+0000","last_unstale":"2024-09-19T09:25:42.985606+0000","last_undegraded":"2024-09-19T09:25:42.985606+0000","last_fullsized":"2024-09-19T09:25:42.985606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:47.156253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:51.982811+0000","last_change":"2024-09-19T09:25:27.322759+0000","last_active":"2024-09-19T09:25:51.982811+0000","last_peered":"2024-09-19T09:25:51.982811+0000","last_clean":"2024-09-19T09:25:51.982811+0000","last_became_active":"2024-09-19T09:25:27.322402+0000","last_became_peered":"2024-09-19T09:25:27.322402+0000","last_unstale":"2024-09-19T09:25:51.982811+0000","last_undegraded":"2024-09-19T09:25:51.982811+0000","last_fullsized":"2024-09-19T09:25:51.982811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:06.276957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.999834+0000","last_change":"2024-09-19T09:25:42.999834+0000","last_active":"2024-09-19T09:25:42.999834+0000","last_peered":"2024-09-19T09:25:42.999834+0000","last_clean":"2024-09-19T09:25:42.999834+0000","last_became_active":"2024-09-19T09:25:42.999450+0000","last_became_peered":"2024-09-19T09:25:42.999450+0000","last_unstale":"2024-09-19T09:25:42.999834+0000","last_undegraded":"2024-09-19T09:25:42.999834+0000","last_fullsized":"2024-09-19T09:25:42.999834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:00.509844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:51.006904+0000","last_change":"2024-09-19T09:25:27.275212+0000","last_active":"2024-09-19T09:25:51.006904+0000","last_peered":"2024-09-19T09:25:51.006904+0000","last_clean":"2024-09-19T09:25:51.006904+0000","last_became_active":"2024-09-19T09:25:27.274802+0000","last_became_peered":"2024-09-19T09:25:27.274802+0000","last_unstale":"2024-09-19T09:25:51.006904+0000","last_undegraded":"2024-09-19T09:25:51.006904+0000","last_fullsized":"2024-09-19T09:25:51.006904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:48.392026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986730+0000","last_change":"2024-09-19T09:25:29.304747+0000","last_active":"2024-09-19T09:25:42.986730+0000","last_peered":"2024-09-19T09:25:42.986730+0000","last_clean":"2024-09-19T09:25:42.986730+0000","last_became_active":"2024-09-19T09:25:29.304386+0000","last_became_peered":"2024-09-19T09:25:29.304386+0000","last_unstale":"2024-09-19T09:25:42.986730+0000","last_undegraded":"2024-09-19T09:25:42.986730+0000","last_fullsized":"2024-09-19T09:25:42.986730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.788697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988854+0000","last_change":"2024-09-19T09:25:27.323083+0000","last_active":"2024-09-19T09:25:42.988854+0000","last_peered":"2024-09-19T09:25:42.988854+0000","last_clean":"2024-09-19T09:25:42.988854+0000","last_became_active":"2024-09-19T09:25:27.322707+0000","last_became_peered":"2024-09-19T09:25:27.322707+0000","last_unstale":"2024-09-19T09:25:42.988854+0000","last_undegraded":"2024-09-19T09:25:42.988854+0000","last_fullsized":"2024-09-19T09:25:42.988854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:24.415116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986687+0000","last_change":"2024-09-19T09:25:29.292472+0000","last_active":"2024-09-19T09:25:42.986687+0000","last_peered":"2024-09-19T09:25:42.986687+0000","last_clean":"2024-09-19T09:25:42.986687+0000","last_became_active":"2024-09-19T09:25:29.292079+0000","last_became_peered":"2024-09-19T09:25:29.292079+0000","last_unstale":"2024-09-19T09:25:42.986687+0000","last_undegraded":"2024-09-19T09:25:42.986687+0000","last_fullsized":"2024-09-19T09:25:42.986687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:35.801594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991224+0000","last_change":"2024-09-19T09:25:27.324839+0000","last_active":"2024-09-19T09:25:42.991224+0000","last_peered":"2024-09-19T09:25:42.991224+0000","last_clean":"2024-09-19T09:25:42.991224+0000","last_became_active":"2024-09-19T09:25:27.324475+0000","last_became_peered":"2024-09-19T09:25:27.324475+0000","last_unstale":"2024-09-19T09:25:42.991224+0000","last_undegraded":"2024-09-19T09:25:42.991224+0000","last_fullsized":"2024-09-19T09:25:42.991224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:32.251753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985998+0000","last_change":"2024-09-19T09:25:29.293025+0000","last_active":"2024-09-19T09:25:42.985998+0000","last_peered":"2024-09-19T09:25:42.985998+0000","last_clean":"2024-09-19T09:25:42.985998+0000","last_became_active":"2024-09-19T09:25:29.292570+0000","last_became_peered":"2024-09-19T09:25:29.292570+0000","last_unstale":"2024-09-19T09:25:42.985998+0000","last_undegraded":"2024-09-19T09:25:42.985998+0000","last_fullsized":"2024-09-19T09:25:42.985998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:16.394524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'13","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:51.295097+0000","last_change":"2024-09-19T09:25:27.278100+0000","last_active":"2024-09-19T09:25:51.295097+0000","last_peered":"2024-09-19T09:25:51.295097+0000","last_clean":"2024-09-19T09:25:51.295097+0000","last_became_active":"2024-09-19T09:25:27.277769+0000","last_became_peered":"2024-09-19T09:25:27.277769+0000","last_unstale":"2024-09-19T09:25:51.295097+0000","last_undegraded":"2024-09-19T09:25:51.295097+0000","last_fullsized":"2024-09-19T09:25:51.295097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:31.885142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991325+0000","last_change":"2024-09-19T09:25:29.293651+0000","last_active":"2024-09-19T09:25:42.991325+0000","last_peered":"2024-09-19T09:25:42.991325+0000","last_clean":"2024-09-19T09:25:42.991325+0000","last_became_active":"2024-09-19T09:25:29.293317+0000","last_became_peered":"2024-09-19T09:25:29.293317+0000","last_unstale":"2024-09-19T09:25:42.991325+0000","last_undegraded":"2024-09-19T09:25:42.991325+0000","last_fullsized":"2024-09-19T09:25:42.991325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:40.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990030+0000","last_change":"2024-09-19T09:25:27.269367+0000","last_active":"2024-09-19T09:25:42.990030+0000","last_peered":"2024-09-19T09:25:42.990030+0000","last_clean":"2024-09-19T09:25:42.990030+0000","last_became_active":"2024-09-19T09:25:27.269011+0000","last_became_peered":"2024-09-19T09:25:27.269011+0000","last_unstale":"2024-09-19T09:25:42.990030+0000","last_undegraded":"2024-09-19T09:25:42.990030+0000","last_fullsized":"2024-09-19T09:25:42.990030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:00.930752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992257+0000","last_change":"2024-09-19T09:25:29.299134+0000","last_active":"2024-09-19T09:25:42.992257+0000","last_peered":"2024-09-19T09:25:42.992257+0000","last_clean":"2024-09-19T09:25:42.992257+0000","last_became_active":"2024-09-19T09:25:29.298648+0000","last_became_peered":"2024-09-19T09:25:29.298648+0000","last_unstale":"2024-09-19T09:25:42.992257+0000","last_undegraded":"2024-09-19T09:25:42.992257+0000","last_fullsized":"2024-09-19T09:25:42.992257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:39.627852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003686+0000","last_change":"2024-09-19T09:25:27.322861+0000","last_active":"2024-09-19T09:25:43.003686+0000","last_peered":"2024-09-19T09:25:43.003686+0000","last_clean":"2024-09-19T09:25:43.003686+0000","last_became_active":"2024-09-19T09:25:27.322560+0000","last_became_peered":"2024-09-19T09:25:27.322560+0000","last_unstale":"2024-09-19T09:25:43.003686+0000","last_undegraded":"2024-09-19T09:25:43.003686+0000","last_fullsized":"2024-09-19T09:25:43.003686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:35.993093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990868+0000","last_change":"2024-09-19T09:25:29.301680+0000","last_active":"2024-09-19T09:25:42.990868+0000","last_peered":"2024-09-19T09:25:42.990868+0000","last_clean":"2024-09-19T09:25:42.990868+0000","last_became_active":"2024-09-19T09:25:29.301331+0000","last_became_peered":"2024-09-19T09:25:29.301331+0000","last_unstale":"2024-09-19T09:25:42.990868+0000","last_undegraded":"2024-09-19T09:25:42.990868+0000","last_fullsized":"2024-09-19T09:25:42.990868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:57.364245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:27.272609+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:27.272169+0000","last_became_peered":"2024-09-19T09:25:27.272169+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.879474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988584+0000","last_change":"2024-09-19T09:25:29.296847+0000","last_active":"2024-09-19T09:25:42.988584+0000","last_peered":"2024-09-19T09:25:42.988584+0000","last_clean":"2024-09-19T09:25:42.988584+0000","last_became_active":"2024-09-19T09:25:29.296465+0000","last_became_peered":"2024-09-19T09:25:29.296465+0000","last_unstale":"2024-09-19T09:25:42.988584+0000","last_undegraded":"2024-09-19T09:25:42.988584+0000","last_fullsized":"2024-09-19T09:25:42.988584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:05:48.553252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.012954+0000","last_change":"2024-09-19T09:25:43.012532+0000","last_active":"2024-09-19T09:25:43.012954+0000","last_peered":"2024-09-19T09:25:43.012954+0000","last_clean":"2024-09-19T09:25:43.012954+0000","last_became_active":"2024-09-19T09:25:42.994996+0000","last_became_peered":"2024-09-19T09:25:42.994996+0000","last_unstale":"2024-09-19T09:25:43.012954+0000","last_undegraded":"2024-09-19T09:25:43.012954+0000","last_fullsized":"2024-09-19T09:25:43.012954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:25.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991072+0000","last_change":"2024-09-19T09:25:29.297633+0000","last_active":"2024-09-19T09:25:42.991072+0000","last_peered":"2024-09-19T09:25:42.991072+0000","last_clean":"2024-09-19T09:25:42.991072+0000","last_became_active":"2024-09-19T09:25:29.297284+0000","last_became_peered":"2024-09-19T09:25:29.297284+0000","last_unstale":"2024-09-19T09:25:42.991072+0000","last_undegraded":"2024-09-19T09:25:42.991072+0000","last_fullsized":"2024-09-19T09:25:42.991072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:50.582546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991172+0000","last_change":"2024-09-19T09:25:27.281778+0000","last_active":"2024-09-19T09:25:42.991172+0000","last_peered":"2024-09-19T09:25:42.991172+0000","last_clean":"2024-09-19T09:25:42.991172+0000","last_became_active":"2024-09-19T09:25:27.281280+0000","last_became_peered":"2024-09-19T09:25:27.281280+0000","last_unstale":"2024-09-19T09:25:42.991172+0000","last_undegraded":"2024-09-19T09:25:42.991172+0000","last_fullsized":"2024-09-19T09:25:42.991172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:46.811968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989808+0000","last_change":"2024-09-19T09:25:29.295663+0000","last_active":"2024-09-19T09:25:42.989808+0000","last_peered":"2024-09-19T09:25:42.989808+0000","last_clean":"2024-09-19T09:25:42.989808+0000","last_became_active":"2024-09-19T09:25:29.295146+0000","last_became_peered":"2024-09-19T09:25:29.295146+0000","last_unstale":"2024-09-19T09:25:42.989808+0000","last_undegraded":"2024-09-19T09:25:42.989808+0000","last_fullsized":"2024-09-19T09:25:42.989808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:36.749339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996010+0000","last_change":"2024-09-19T09:25:29.298004+0000","last_active":"2024-09-19T09:25:42.996010+0000","last_peered":"2024-09-19T09:25:42.996010+0000","last_clean":"2024-09-19T09:25:42.996010+0000","last_became_active":"2024-09-19T09:25:29.297600+0000","last_became_peered":"2024-09-19T09:25:29.297600+0000","last_unstale":"2024-09-19T09:25:42.996010+0000","last_undegraded":"2024-09-19T09:25:42.996010+0000","last_fullsized":"2024-09-19T09:25:42.996010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:09.738615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992323+0000","last_change":"2024-09-19T09:25:27.282876+0000","last_active":"2024-09-19T09:25:42.992323+0000","last_peered":"2024-09-19T09:25:42.992323+0000","last_clean":"2024-09-19T09:25:42.992323+0000","last_became_active":"2024-09-19T09:25:27.282201+0000","last_became_peered":"2024-09-19T09:25:27.282201+0000","last_unstale":"2024-09-19T09:25:42.992323+0000","last_undegraded":"2024-09-19T09:25:42.992323+0000","last_fullsized":"2024-09-19T09:25:42.992323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:08.508662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003650+0000","last_change":"2024-09-19T09:25:29.288379+0000","last_active":"2024-09-19T09:25:43.003650+0000","last_peered":"2024-09-19T09:25:43.003650+0000","last_clean":"2024-09-19T09:25:43.003650+0000","last_became_active":"2024-09-19T09:25:29.287917+0000","last_became_peered":"2024-09-19T09:25:29.287917+0000","last_unstale":"2024-09-19T09:25:43.003650+0000","last_undegraded":"2024-09-19T09:25:43.003650+0000","last_fullsized":"2024-09-19T09:25:43.003650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:03.628233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991076+0000","last_change":"2024-09-19T09:25:27.279551+0000","last_active":"2024-09-19T09:25:42.991076+0000","last_peered":"2024-09-19T09:25:42.991076+0000","last_clean":"2024-09-19T09:25:42.991076+0000","last_became_active":"2024-09-19T09:25:27.278956+0000","last_became_peered":"2024-09-19T09:25:27.278956+0000","last_unstale":"2024-09-19T09:25:42.991076+0000","last_undegraded":"2024-09-19T09:25:42.991076+0000","last_fullsized":"2024-09-19T09:25:42.991076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:17.186453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992345+0000","last_change":"2024-09-19T09:25:29.288657+0000","last_active":"2024-09-19T09:25:42.992345+0000","last_peered":"2024-09-19T09:25:42.992345+0000","last_clean":"2024-09-19T09:25:42.992345+0000","last_became_active":"2024-09-19T09:25:29.288355+0000","last_became_peered":"2024-09-19T09:25:29.288355+0000","last_unstale":"2024-09-19T09:25:42.992345+0000","last_undegraded":"2024-09-19T09:25:42.992345+0000","last_fullsized":"2024-09-19T09:25:42.992345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:46.237558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991315+0000","last_change":"2024-09-19T09:25:27.323312+0000","last_active":"2024-09-19T09:25:42.991315+0000","last_peered":"2024-09-19T09:25:42.991315+0000","last_clean":"2024-09-19T09:25:42.991315+0000","last_became_active":"2024-09-19T09:25:27.322984+0000","last_became_peered":"2024-09-19T09:25:27.322984+0000","last_unstale":"2024-09-19T09:25:42.991315+0000","last_undegraded":"2024-09-19T09:25:42.991315+0000","last_fullsized":"2024-09-19T09:25:42.991315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:48.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991159+0000","last_change":"2024-09-19T09:25:29.301832+0000","last_active":"2024-09-19T09:25:42.991159+0000","last_peered":"2024-09-19T09:25:42.991159+0000","last_clean":"2024-09-19T09:25:42.991159+0000","last_became_active":"2024-09-19T09:25:29.301539+0000","last_became_peered":"2024-09-19T09:25:29.301539+0000","last_unstale":"2024-09-19T09:25:42.991159+0000","last_undegraded":"2024-09-19T09:25:42.991159+0000","last_fullsized":"2024-09-19T09:25:42.991159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:00.244948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:51.006233+0000","last_change":"2024-09-19T09:25:43.009668+0000","last_active":"2024-09-19T09:25:51.006233+0000","last_peered":"2024-09-19T09:25:51.006233+0000","last_clean":"2024-09-19T09:25:51.006233+0000","last_became_active":"2024-09-19T09:25:42.994966+0000","last_became_peered":"2024-09-19T09:25:42.994966+0000","last_unstale":"2024-09-19T09:25:51.006233+0000","last_undegraded":"2024-09-19T09:25:51.006233+0000","last_fullsized":"2024-09-19T09:25:51.006233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:04:05.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54796,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":165,"num_read_kb":143,"num_write":235,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1842,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":164388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612625,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":431057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1068216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005082,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234015,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":430300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1022934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593770,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":901120,"data_stored":430032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822703,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1022498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986227,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":909312,"data_stored":431456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215160,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":476797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":477747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":921600,"data_stored":431371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":47997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:25:56.675 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T09:25:56.675 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T09:25:56.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T09:25:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:57.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:57 smithi016 ceph-mon[30979]: pgmap v288: 97 pgs: 97 active+clean; 630 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 11 op/s; 153 B/s, 0 objects/s recovering 2024-09-19T09:25:57.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:57 smithi016 ceph-mon[30979]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:25:57.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:57 smithi142 ceph-mon[35032]: pgmap v288: 97 pgs: 97 active+clean; 630 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 11 op/s; 153 B/s, 0 objects/s recovering 2024-09-19T09:25:57.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:57 smithi142 ceph-mon[35032]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:25:57.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:57 smithi097 ceph-mon[34864]: pgmap v288: 97 pgs: 97 active+clean; 630 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 11 op/s; 153 B/s, 0 objects/s recovering 2024-09-19T09:25:57.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:57 smithi097 ceph-mon[34864]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:25:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.393+0000 7f4b33746640 1 -- 172.21.15.16:0/3776943486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b2c10e1e0 msgr2=0x7f4b2c1105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.393+0000 7f4b33746640 1 --2- 172.21.15.16:0/3776943486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b2c10e1e0 0x7f4b2c1105d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009ca0 tx=0x7f4b2402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:25:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.393+0000 7f4b33746640 1 -- 172.21.15.16:0/3776943486 shutdown_connections 2024-09-19T09:25:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.393+0000 7f4b33746640 1 --2- 172.21.15.16:0/3776943486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b2c074c50 0x7f4b2c0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.393+0000 7f4b33746640 1 --2- 172.21.15.16:0/3776943486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b2c10e1e0 0x7f4b2c1105d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.393+0000 7f4b33746640 1 --2- 172.21.15.16:0/3776943486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c076220 0x7f4b2c074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.393+0000 7f4b33746640 1 -- 172.21.15.16:0/3776943486 >> 172.21.15.16:0/3776943486 conn(0x7f4b2c0fd390 msgr2=0x7f4b2c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.394+0000 7f4b33746640 1 -- 172.21.15.16:0/3776943486 shutdown_connections 2024-09-19T09:25:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.394+0000 7f4b33746640 1 -- 172.21.15.16:0/3776943486 wait complete. 2024-09-19T09:25:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.394+0000 7f4b33746640 1 Processor -- start 2024-09-19T09:25:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.394+0000 7f4b33746640 1 -- start start 2024-09-19T09:25:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.394+0000 7f4b33746640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b2c074c50 0x7f4b2c19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.394+0000 7f4b33746640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b2c076220 0x7f4b2c19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.394+0000 7f4b33746640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c10e1e0 0x7f4b2c1a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.394+0000 7f4b33746640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c113f00 con 0x7f4b2c076220 2024-09-19T09:25:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.394+0000 7f4b33746640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c113d80 con 0x7f4b2c10e1e0 2024-09-19T09:25:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.394+0000 7f4b33746640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c114080 con 0x7f4b2c074c50 2024-09-19T09:25:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.394+0000 7f4b30ce1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b2c076220 0x7f4b2c19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b30ce1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b2c076220 0x7f4b2c19ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49622/0 (socket says 172.21.15.16:49622) 2024-09-19T09:25:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b30ce1640 1 -- 172.21.15.16:0/777871655 learned_addr learned my addr 172.21.15.16:0/777871655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b314e2640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b2c074c50 0x7f4b2c19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b31ce3640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c10e1e0 0x7f4b2c1a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b30ce1640 1 -- 172.21.15.16:0/777871655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b2c074c50 msgr2=0x7f4b2c19e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b30ce1640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b2c074c50 0x7f4b2c19e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b30ce1640 1 -- 172.21.15.16:0/777871655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c10e1e0 msgr2=0x7f4b2c1a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b30ce1640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c10e1e0 0x7f4b2c1a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b30ce1640 1 -- 172.21.15.16:0/777871655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c1a6110 con 0x7f4b2c076220 2024-09-19T09:25:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b30ce1640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b2c076220 0x7f4b2c19ea20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00b4a0 tx=0x7f4b1c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b227fc640 1 -- 172.21.15.16:0/777871655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c014070 con 0x7f4b2c076220 2024-09-19T09:25:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b33746640 1 -- 172.21.15.16:0/777871655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2c1a6400 con 0x7f4b2c076220 2024-09-19T09:25:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b227fc640 1 -- 172.21.15.16:0/777871655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b1c010af0 con 0x7f4b2c076220 2024-09-19T09:25:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b227fc640 1 -- 172.21.15.16:0/777871655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c017d10 con 0x7f4b2c076220 2024-09-19T09:25:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.395+0000 7f4b33746640 1 -- 172.21.15.16:0/777871655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2c1a6970 con 0x7f4b2c076220 2024-09-19T09:25:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.396+0000 7f4b33746640 1 -- 172.21.15.16:0/777871655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2c1043b0 con 0x7f4b2c076220 2024-09-19T09:25:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.397+0000 7f4b227fc640 1 -- 172.21.15.16:0/777871655 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4b1c017070 con 0x7f4b2c076220 2024-09-19T09:25:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.398+0000 7f4b227fc640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4b180a3300 0x7f4b180a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.398+0000 7f4b314e2640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4b180a3300 0x7f4b180a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.398+0000 7f4b227fc640 1 -- 172.21.15.16:0/777871655 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4b1c0cf160 con 0x7f4b2c076220 2024-09-19T09:25:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.399+0000 7f4b314e2640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4b180a3300 0x7f4b180a57c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4b280058c0 tx=0x7f4b2800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.401+0000 7f4b227fc640 1 -- 172.21.15.16:0/777871655 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4b1c09bdf0 con 0x7f4b2c076220 2024-09-19T09:25:57.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.645+0000 7f4b33746640 1 -- 172.21.15.16:0/777871655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f4b2c02d7c0 con 0x7f4b2c076220 2024-09-19T09:25:57.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.647+0000 7f4b227fc640 1 -- 172.21.15.16:0/777871655 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f4b1c09b6f0 con 0x7f4b2c076220 2024-09-19T09:25:57.648 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-19T09:25:57.648 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQAl7utmBsmKJhAA7tYO0d+Sw7+Hqbtv9Nj2NA== 2024-09-19T09:25:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.649+0000 7f4b33746640 1 -- 172.21.15.16:0/777871655 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4b180a3300 msgr2=0x7f4b180a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.649+0000 7f4b33746640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4b180a3300 0x7f4b180a57c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4b280058c0 tx=0x7f4b2800a2d0 comp rx=0 tx=0).stop 2024-09-19T09:25:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.649+0000 7f4b33746640 1 -- 172.21.15.16:0/777871655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b2c076220 msgr2=0x7f4b2c19ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.649+0000 7f4b33746640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b2c076220 0x7f4b2c19ea20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00b4a0 tx=0x7f4b1c00b960 comp rx=0 tx=0).stop 2024-09-19T09:25:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.650+0000 7f4b33746640 1 -- 172.21.15.16:0/777871655 shutdown_connections 2024-09-19T09:25:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.650+0000 7f4b33746640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c10e1e0 0x7f4b2c1a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.650+0000 7f4b33746640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b2c076220 0x7f4b2c19ea20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.650+0000 7f4b33746640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4b180a3300 0x7f4b180a57c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.650+0000 7f4b33746640 1 --2- 172.21.15.16:0/777871655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b2c074c50 0x7f4b2c19e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.650+0000 7f4b33746640 1 -- 172.21.15.16:0/777871655 >> 172.21.15.16:0/777871655 conn(0x7f4b2c0fd390 msgr2=0x7f4b2c101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.650+0000 7f4b33746640 1 -- 172.21.15.16:0/777871655 shutdown_connections 2024-09-19T09:25:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:57.650+0000 7f4b33746640 1 -- 172.21.15.16:0/777871655 wait complete. 2024-09-19T09:25:58.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:58 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/777871655' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T09:25:58.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:58 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/777871655' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T09:25:58.236 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:25:58.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T09:25:58.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T09:25:58.274 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T09:25:58.274 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T09:25:58.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T09:25:58.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:58 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/777871655' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T09:25:58.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:58 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/777871655' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T09:25:58.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:58 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/777871655' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T09:25:58.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:58 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/777871655' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T09:25:58.569 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:25:59.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.000+0000 7fe1c749e640 1 -- 172.21.15.16:0/2010619213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c01027a0 msgr2=0x7fe1c01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:59.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.000+0000 7fe1c749e640 1 --2- 172.21.15.16:0/2010619213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c01027a0 0x7fe1c01067f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009900 tx=0x7fe1b002a410 comp rx=0 tx=0).stop 2024-09-19T09:25:59.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.000+0000 7fe1c749e640 1 -- 172.21.15.16:0/2010619213 shutdown_connections 2024-09-19T09:25:59.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.000+0000 7fe1c749e640 1 --2- 172.21.15.16:0/2010619213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c0106f20 0x7fe1c010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.000+0000 7fe1c749e640 1 --2- 172.21.15.16:0/2010619213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c01027a0 0x7fe1c01067f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.000+0000 7fe1c749e640 1 --2- 172.21.15.16:0/2010619213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c0101df0 0x7fe1c01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.000+0000 7fe1c749e640 1 -- 172.21.15.16:0/2010619213 >> 172.21.15.16:0/2010619213 conn(0x7fe1c00fd8f0 msgr2=0x7fe1c00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:59.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.000+0000 7fe1c749e640 1 -- 172.21.15.16:0/2010619213 shutdown_connections 2024-09-19T09:25:59.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.000+0000 7fe1c749e640 1 -- 172.21.15.16:0/2010619213 wait complete. 2024-09-19T09:25:59.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c749e640 1 Processor -- start 2024-09-19T09:25:59.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c749e640 1 -- start start 2024-09-19T09:25:59.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c749e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c0101df0 0x7fe1c00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:59.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c749e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c01027a0 0x7fe1c0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:59.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c749e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0106f20 0x7fe1c0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:59.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c749e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c0111f80 con 0x7fe1c0106f20 2024-09-19T09:25:59.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c749e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c0111e00 con 0x7fe1c0101df0 2024-09-19T09:25:59.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c749e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c0112100 con 0x7fe1c01027a0 2024-09-19T09:25:59.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c5a3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0106f20 0x7fe1c0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:59.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c5a3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0106f20 0x7fe1c0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49638/0 (socket says 172.21.15.16:49638) 2024-09-19T09:25:59.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c5a3b640 1 -- 172.21.15.16:0/3161874469 learned_addr learned my addr 172.21.15.16:0/3161874469 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:25:59.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c523a640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c0101df0 0x7fe1c00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:59.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c4a39640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c01027a0 0x7fe1c0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:59.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c5a3b640 1 -- 172.21.15.16:0/3161874469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c01027a0 msgr2=0x7fe1c0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:59.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c5a3b640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c01027a0 0x7fe1c0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c5a3b640 1 -- 172.21.15.16:0/3161874469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c0101df0 msgr2=0x7fe1c00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:59.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c5a3b640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c0101df0 0x7fe1c00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.001+0000 7fe1c5a3b640 1 -- 172.21.15.16:0/3161874469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c01a6920 con 0x7fe1c0106f20 2024-09-19T09:25:59.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.002+0000 7fe1c5a3b640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0106f20 0x7fe1c0074f70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800eb10 tx=0x7fe1b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:59.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.002+0000 7fe1b67fc640 1 -- 172.21.15.16:0/3161874469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8004b20 con 0x7fe1c0106f20 2024-09-19T09:25:59.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.002+0000 7fe1b67fc640 1 -- 172.21.15.16:0/3161874469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b8004cc0 con 0x7fe1c0106f20 2024-09-19T09:25:59.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.002+0000 7fe1b67fc640 1 -- 172.21.15.16:0/3161874469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8010760 con 0x7fe1c0106f20 2024-09-19T09:25:59.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.002+0000 7fe1c749e640 1 -- 172.21.15.16:0/3161874469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c01a6c10 con 0x7fe1c0106f20 2024-09-19T09:25:59.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.002+0000 7fe1c749e640 1 -- 172.21.15.16:0/3161874469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c01a7120 con 0x7fe1c0106f20 2024-09-19T09:25:59.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.003+0000 7fe1c749e640 1 -- 172.21.15.16:0/3161874469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c01021d0 con 0x7fe1c0106f20 2024-09-19T09:25:59.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.004+0000 7fe1b67fc640 1 -- 172.21.15.16:0/3161874469 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe1b8005000 con 0x7fe1c0106f20 2024-09-19T09:25:59.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.005+0000 7fe1b67fc640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe1ac0a32d0 0x7fe1ac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:25:59.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.005+0000 7fe1c523a640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe1ac0a32d0 0x7fe1ac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:25:59.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.005+0000 7fe1b67fc640 1 -- 172.21.15.16:0/3161874469 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe1b80d0470 con 0x7fe1c0106f20 2024-09-19T09:25:59.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.005+0000 7fe1c523a640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe1ac0a32d0 0x7fe1ac0a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00bbe0 tx=0x7fe1bc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:25:59.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.008+0000 7fe1b67fc640 1 -- 172.21.15.16:0/3161874469 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe1b809cff0 con 0x7fe1c0106f20 2024-09-19T09:25:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.232+0000 7fe1c749e640 1 -- 172.21.15.16:0/3161874469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe1c0075a20 con 0x7fe1c0106f20 2024-09-19T09:25:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.237+0000 7fe1b67fc640 1 -- 172.21.15.16:0/3161874469 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264048 (secure 0 0 0) 0x7fe1c0075a20 con 0x7fe1c0106f20 2024-09-19T09:25:59.242 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:25:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.240+0000 7fe1c749e640 1 -- 172.21.15.16:0/3161874469 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe1ac0a32d0 msgr2=0x7fe1ac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.240+0000 7fe1c749e640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe1ac0a32d0 0x7fe1ac0a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00bbe0 tx=0x7fe1bc00b390 comp rx=0 tx=0).stop 2024-09-19T09:25:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.240+0000 7fe1c749e640 1 -- 172.21.15.16:0/3161874469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0106f20 msgr2=0x7fe1c0074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:25:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.240+0000 7fe1c749e640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0106f20 0x7fe1c0074f70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800eb10 tx=0x7fe1b800c2d0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.240+0000 7fe1c749e640 1 -- 172.21.15.16:0/3161874469 shutdown_connections 2024-09-19T09:25:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.240+0000 7fe1c749e640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe1ac0a32d0 0x7fe1ac0a5790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.240+0000 7fe1c749e640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c01027a0 0x7fe1c0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.240+0000 7fe1c749e640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0106f20 0x7fe1c0074f70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.240+0000 7fe1c749e640 1 --2- 172.21.15.16:0/3161874469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1c0101df0 0x7fe1c00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:25:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.240+0000 7fe1c749e640 1 -- 172.21.15.16:0/3161874469 >> 172.21.15.16:0/3161874469 conn(0x7fe1c00fd8f0 msgr2=0x7fe1c00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:25:59.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.240+0000 7fe1c749e640 1 -- 172.21.15.16:0/3161874469 shutdown_connections 2024-09-19T09:25:59.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:25:59.240+0000 7fe1c749e640 1 -- 172.21.15.16:0/3161874469 wait complete. 2024-09-19T09:25:59.251 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:25:59 smithi016 ceph-mon[30979]: pgmap v289: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 14 op/s; 153 B/s, 0 objects/s recovering 2024-09-19T09:25:59.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:25:59 smithi142 ceph-mon[35032]: pgmap v289: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 14 op/s; 153 B/s, 0 objects/s recovering 2024-09-19T09:25:59.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:25:59 smithi097 ceph-mon[34864]: pgmap v289: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 14 op/s; 153 B/s, 0 objects/s recovering 2024-09-19T09:25:59.946 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4287557979},{"type":"v1","addr":"172.21.15.16:6801","nonce":4287557979}]},"active_addr":"172.21.15.16:6801/4287557979","active_change":"2024-09-19T09:18:41.244183+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.16:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":458055934}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1630957839}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1482490412}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2430428028}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":920778761}]}]} 2024-09-19T09:25:59.952 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T09:25:59.953 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T09:25:59.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:26:00.247 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:00.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:00 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3161874469' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T09:26:00.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:00 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3161874469' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T09:26:00.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:00 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3161874469' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T09:26:00.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.690+0000 7f406f2b9640 1 -- 172.21.15.16:0/2144613013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4068102790 msgr2=0x7f40681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:00.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.690+0000 7f406f2b9640 1 --2- 172.21.15.16:0/2144613013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4068102790 0x7f40681067e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4058009900 tx=0x7f405802a410 comp rx=0 tx=0).stop 2024-09-19T09:26:00.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.690+0000 7f406f2b9640 1 -- 172.21.15.16:0/2144613013 shutdown_connections 2024-09-19T09:26:00.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.690+0000 7f406f2b9640 1 --2- 172.21.15.16:0/2144613013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4068106f10 0x7f406810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:00.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.690+0000 7f406f2b9640 1 --2- 172.21.15.16:0/2144613013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4068102790 0x7f40681067e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:00.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.690+0000 7f406f2b9640 1 --2- 172.21.15.16:0/2144613013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4068101de0 0x7f40681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.690+0000 7f406f2b9640 1 -- 172.21.15.16:0/2144613013 >> 172.21.15.16:0/2144613013 conn(0x7f40680fd8c0 msgr2=0x7f40680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.690+0000 7f406f2b9640 1 -- 172.21.15.16:0/2144613013 shutdown_connections 2024-09-19T09:26:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.690+0000 7f406f2b9640 1 -- 172.21.15.16:0/2144613013 wait complete. 2024-09-19T09:26:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.691+0000 7f406f2b9640 1 Processor -- start 2024-09-19T09:26:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.691+0000 7f406f2b9640 1 -- start start 2024-09-19T09:26:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.691+0000 7f406f2b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4068101de0 0x7f40681984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.691+0000 7f406f2b9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4068102790 0x7f40681989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.691+0000 7f406f2b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4068106f10 0x7f406819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.691+0000 7f406f2b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4068111f70 con 0x7f4068106f10 2024-09-19T09:26:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.691+0000 7f406f2b9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4068111df0 con 0x7f4068101de0 2024-09-19T09:26:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.691+0000 7f406f2b9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40681120f0 con 0x7f4068102790 2024-09-19T09:26:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.691+0000 7f406d856640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4068106f10 0x7f406819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:00.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.691+0000 7f406d856640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4068106f10 0x7f406819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49664/0 (socket says 172.21.15.16:49664) 2024-09-19T09:26:00.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.691+0000 7f406d856640 1 -- 172.21.15.16:0/247975879 learned_addr learned my addr 172.21.15.16:0/247975879 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:00.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.691+0000 7f406d055640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4068101de0 0x7f40681984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:00.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.692+0000 7f406c854640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4068102790 0x7f40681989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:00.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.692+0000 7f406d856640 1 -- 172.21.15.16:0/247975879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4068102790 msgr2=0x7f40681989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:00.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.692+0000 7f406d856640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4068102790 0x7f40681989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:00.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.692+0000 7f406d856640 1 -- 172.21.15.16:0/247975879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4068101de0 msgr2=0x7f40681984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:00.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.692+0000 7f406d856640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4068101de0 0x7f40681984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:00.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.692+0000 7f406d856640 1 -- 172.21.15.16:0/247975879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406819d460 con 0x7f4068106f10 2024-09-19T09:26:00.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.692+0000 7f406c854640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4068102790 0x7f40681989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:00.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.692+0000 7f406d856640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4068106f10 0x7f406819cd80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f406000eaa0 tx=0x7f406000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:00.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.692+0000 7f405e7fc640 1 -- 172.21.15.16:0/247975879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406000cdb0 con 0x7f4068106f10 2024-09-19T09:26:00.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.692+0000 7f405e7fc640 1 -- 172.21.15.16:0/247975879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4060005020 con 0x7f4068106f10 2024-09-19T09:26:00.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.692+0000 7f406f2b9640 1 -- 172.21.15.16:0/247975879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406819d750 con 0x7f4068106f10 2024-09-19T09:26:00.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.692+0000 7f405e7fc640 1 -- 172.21.15.16:0/247975879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40600116b0 con 0x7f4068106f10 2024-09-19T09:26:00.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.692+0000 7f406f2b9640 1 -- 172.21.15.16:0/247975879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40681a5030 con 0x7f4068106f10 2024-09-19T09:26:00.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.693+0000 7f406f2b9640 1 -- 172.21.15.16:0/247975879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40681021c0 con 0x7f4068106f10 2024-09-19T09:26:00.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.694+0000 7f405e7fc640 1 -- 172.21.15.16:0/247975879 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4060005560 con 0x7f4068106f10 2024-09-19T09:26:00.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.695+0000 7f405e7fc640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f40540a3350 0x7f40540a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:00.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.695+0000 7f406d055640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f40540a3350 0x7f40540a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:00.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.695+0000 7f405e7fc640 1 -- 172.21.15.16:0/247975879 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f40600d0090 con 0x7f4068106f10 2024-09-19T09:26:00.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.696+0000 7f406d055640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f40540a3350 0x7f40540a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4064005fd0 tx=0x7f4064005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:00.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.699+0000 7f405e7fc640 1 -- 172.21.15.16:0/247975879 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4060016030 con 0x7f4068106f10 2024-09-19T09:26:00.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.876+0000 7f406f2b9640 1 -- 172.21.15.16:0/247975879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f406810a1b0 con 0x7f4068106f10 2024-09-19T09:26:00.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.877+0000 7f405e7fc640 1 -- 172.21.15.16:0/247975879 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22665 (secure 0 0 0) 0x7f406009cc90 con 0x7f4068106f10 2024-09-19T09:26:00.879 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:26:00.880 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","created":"2024-09-19T09:17:48.242238+0000","modified":"2024-09-19T09:25:42.982671+0000","last_up_change":"2024-09-19T09:24:40.190289+0000","last_in_change":"2024-09-19T09:24:17.190380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:20:57.748686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:25:25.694451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:25:28.262997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbf3b28b-2282-4057-9e65-19ceb78ff19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6803","nonce":1578957623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6805","nonce":1578957623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6809","nonce":1578957623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6807","nonce":1578957623}]},"public_addr":"172.21.15.16:6803/1578957623","cluster_addr":"172.21.15.16:6805/1578957623","heartbeat_back_addr":"172.21.15.16:6809/1578957623","heartbeat_front_addr":"172.21.15.16:6807/1578957623","state":["exists","up"]},{"osd":1,"uuid":"c2517aeb-1d79-41ad-a180-369eecb9732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6801","nonce":3804152349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6803","nonce":3804152349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6807","nonce":3804152349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6805","nonce":3804152349}]},"public_addr":"172.21.15.97:6801/3804152349","cluster_addr":"172.21.15.97:6803/3804152349","heartbeat_back_addr":"172.21.15.97:6807/3804152349","heartbeat_front_addr":"172.21.15.97:6805/3804152349","state":["exists","up"]},{"osd":2,"uuid":"3c367518-5085-471a-98cb-ec5bf5facdf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6801","nonce":2692613951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6803","nonce":2692613951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6807","nonce":2692613951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6805","nonce":2692613951}]},"public_addr":"172.21.15.142:6801/2692613951","cluster_addr":"172.21.15.142:6803/2692613951","heartbeat_back_addr":"172.21.15.142:6807/2692613951","heartbeat_front_addr":"172.21.15.142:6805/2692613951","state":["exists","up"]},{"osd":3,"uuid":"e7118925-f0fa-46b3-8552-3e3bb76a6489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6811","nonce":3169290004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6813","nonce":3169290004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6817","nonce":3169290004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6815","nonce":3169290004}]},"public_addr":"172.21.15.16:6811/3169290004","cluster_addr":"172.21.15.16:6813/3169290004","heartbeat_back_addr":"172.21.15.16:6817/3169290004","heartbeat_front_addr":"172.21.15.16:6815/3169290004","state":["exists","up"]},{"osd":4,"uuid":"dc1964f7-af9c-4853-843b-9818f87c7768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6809","nonce":3491353825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6811","nonce":3491353825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6815","nonce":3491353825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6813","nonce":3491353825}]},"public_addr":"172.21.15.97:6809/3491353825","cluster_addr":"172.21.15.97:6811/3491353825","heartbeat_back_addr":"172.21.15.97:6815/3491353825","heartbeat_front_addr":"172.21.15.97:6813/3491353825","state":["exists","up"]},{"osd":5,"uuid":"4d7fbafc-d9ba-4036-a89d-aa295b8a23a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6809","nonce":577099986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6811","nonce":577099986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6815","nonce":577099986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6813","nonce":577099986}]},"public_addr":"172.21.15.142:6809/577099986","cluster_addr":"172.21.15.142:6811/577099986","heartbeat_back_addr":"172.21.15.142:6815/577099986","heartbeat_front_addr":"172.21.15.142:6813/577099986","state":["exists","up"]},{"osd":6,"uuid":"e3f346fc-c027-44f8-bd37-bccf6ec09e3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6819","nonce":4053539536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6821","nonce":4053539536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6825","nonce":4053539536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6823","nonce":4053539536}]},"public_addr":"172.21.15.16:6819/4053539536","cluster_addr":"172.21.15.16:6821/4053539536","heartbeat_back_addr":"172.21.15.16:6825/4053539536","heartbeat_front_addr":"172.21.15.16:6823/4053539536","state":["exists","up"]},{"osd":7,"uuid":"d390b547-8949-4d4b-8481-c49d6ae7e424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6817","nonce":1157122670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6819","nonce":1157122670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6823","nonce":1157122670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6821","nonce":1157122670}]},"public_addr":"172.21.15.97:6817/1157122670","cluster_addr":"172.21.15.97:6819/1157122670","heartbeat_back_addr":"172.21.15.97:6823/1157122670","heartbeat_front_addr":"172.21.15.97:6821/1157122670","state":["exists","up"]},{"osd":8,"uuid":"78a476c4-5deb-4f9e-a266-db4493850732","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6817","nonce":643795004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6819","nonce":643795004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6823","nonce":643795004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6821","nonce":643795004}]},"public_addr":"172.21.15.142:6817/643795004","cluster_addr":"172.21.15.142:6819/643795004","heartbeat_back_addr":"172.21.15.142:6823/643795004","heartbeat_front_addr":"172.21.15.142:6821/643795004","state":["exists","up"]},{"osd":9,"uuid":"0415d187-4a77-42f7-b8bb-2d0b52e9cb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6827","nonce":2651132360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6829","nonce":2651132360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2651132360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6831","nonce":2651132360}]},"public_addr":"172.21.15.16:6827/2651132360","cluster_addr":"172.21.15.16:6829/2651132360","heartbeat_back_addr":"172.21.15.16:6833/2651132360","heartbeat_front_addr":"172.21.15.16:6831/2651132360","state":["exists","up"]},{"osd":10,"uuid":"6fa78f96-cdfe-42a0-8d27-0705a0c7df8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6825","nonce":1848371486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6827","nonce":1848371486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6831","nonce":1848371486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6829","nonce":1848371486}]},"public_addr":"172.21.15.97:6825/1848371486","cluster_addr":"172.21.15.97:6827/1848371486","heartbeat_back_addr":"172.21.15.97:6831/1848371486","heartbeat_front_addr":"172.21.15.97:6829/1848371486","state":["exists","up"]},{"osd":11,"uuid":"d27f20e3-84ce-470c-9889-cd86906b158d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6825","nonce":155083878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6827","nonce":155083878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6831","nonce":155083878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6829","nonce":155083878}]},"public_addr":"172.21.15.142:6825/155083878","cluster_addr":"172.21.15.142:6827/155083878","heartbeat_back_addr":"172.21.15.142:6831/155083878","heartbeat_front_addr":"172.21.15.142:6829/155083878","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:05.088764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:29.928529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:55.236930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:20.379471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:44.984151+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:10.410074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:35.233553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:59.928700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:24.503069+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:49.853073+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:14.006841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:38.546467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3710267992":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3740275561":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/2755421444":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/357390846":"2024-09-20T09:18:17.070034+0000","172.21.15.16:6800/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3104087402":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3546947161":"2024-09-20T09:18:17.070034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:26:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.879+0000 7f406f2b9640 1 -- 172.21.15.16:0/247975879 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f40540a3350 msgr2=0x7f40540a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.879+0000 7f406f2b9640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f40540a3350 0x7f40540a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4064005fd0 tx=0x7f4064005850 comp rx=0 tx=0).stop 2024-09-19T09:26:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.879+0000 7f406f2b9640 1 -- 172.21.15.16:0/247975879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4068106f10 msgr2=0x7f406819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.879+0000 7f406f2b9640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4068106f10 0x7f406819cd80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f406000eaa0 tx=0x7f406000ef60 comp rx=0 tx=0).stop 2024-09-19T09:26:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.879+0000 7f406f2b9640 1 -- 172.21.15.16:0/247975879 shutdown_connections 2024-09-19T09:26:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.879+0000 7f406f2b9640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f40540a3350 0x7f40540a5810 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.879+0000 7f406f2b9640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4068102790 0x7f40681989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.879+0000 7f406f2b9640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4068106f10 0x7f406819cd80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.879+0000 7f406f2b9640 1 --2- 172.21.15.16:0/247975879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4068101de0 0x7f40681984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.879+0000 7f406f2b9640 1 -- 172.21.15.16:0/247975879 >> 172.21.15.16:0/247975879 conn(0x7f40680fd8c0 msgr2=0x7f40680fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.879+0000 7f406f2b9640 1 -- 172.21.15.16:0/247975879 shutdown_connections 2024-09-19T09:26:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:00.879+0000 7f406f2b9640 1 -- 172.21.15.16:0/247975879 wait complete. 2024-09-19T09:26:01.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:01 smithi142 ceph-mon[35032]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.5 KiB/s wr, 12 op/s; 55 B/s, 0 objects/s recovering 2024-09-19T09:26:01.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:01 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/247975879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:26:01.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:01 smithi097 ceph-mon[34864]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.5 KiB/s wr, 12 op/s; 55 B/s, 0 objects/s recovering 2024-09-19T09:26:01.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:01 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/247975879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:26:01.483 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:01 smithi016 ceph-mon[30979]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.5 KiB/s wr, 12 op/s; 55 B/s, 0 objects/s recovering 2024-09-19T09:26:01.483 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:01 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/247975879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:26:01.512 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T09:26:01.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:26:01.768 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:02.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.191+0000 7f20db4cc640 1 -- 172.21.15.16:0/2537437873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d4101de0 msgr2=0x7f20d41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:02.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.191+0000 7f20db4cc640 1 --2- 172.21.15.16:0/2537437873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d4101de0 0x7f20d41021c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009900 tx=0x7f20d002a410 comp rx=0 tx=0).stop 2024-09-19T09:26:02.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.192+0000 7f20db4cc640 1 -- 172.21.15.16:0/2537437873 shutdown_connections 2024-09-19T09:26:02.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.192+0000 7f20db4cc640 1 --2- 172.21.15.16:0/2537437873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20d4106f10 0x7f20d410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.192+0000 7f20db4cc640 1 --2- 172.21.15.16:0/2537437873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d4102790 0x7f20d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.192+0000 7f20db4cc640 1 --2- 172.21.15.16:0/2537437873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d4101de0 0x7f20d41021c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.192+0000 7f20db4cc640 1 -- 172.21.15.16:0/2537437873 >> 172.21.15.16:0/2537437873 conn(0x7f20d40fd8c0 msgr2=0x7f20d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.192+0000 7f20db4cc640 1 -- 172.21.15.16:0/2537437873 shutdown_connections 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.192+0000 7f20db4cc640 1 -- 172.21.15.16:0/2537437873 wait complete. 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.192+0000 7f20db4cc640 1 Processor -- start 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.192+0000 7f20db4cc640 1 -- start start 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20db4cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d4101de0 0x7f20d40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20db4cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d4102790 0x7f20d4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20db4cc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20d4106f10 0x7f20d4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20db4cc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4111f90 con 0x7f20d4102790 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20db4cc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4111e10 con 0x7f20d4101de0 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20db4cc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4112110 con 0x7f20d4106f10 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20d8a67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d4102790 0x7f20d4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20d8a67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d4102790 0x7f20d4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49670/0 (socket says 172.21.15.16:49670) 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20d8a67640 1 -- 172.21.15.16:0/47226575 learned_addr learned my addr 172.21.15.16:0/47226575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:02.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20d9a69640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20d4106f10 0x7f20d4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:02.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20d9268640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d4101de0 0x7f20d40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:02.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20d8a67640 1 -- 172.21.15.16:0/47226575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20d4106f10 msgr2=0x7f20d4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:02.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20d8a67640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20d4106f10 0x7f20d4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20d8a67640 1 -- 172.21.15.16:0/47226575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d4101de0 msgr2=0x7f20d40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:02.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20d8a67640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d4101de0 0x7f20d40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20d8a67640 1 -- 172.21.15.16:0/47226575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d41a6950 con 0x7f20d4102790 2024-09-19T09:26:02.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20d8a67640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d4102790 0x7f20d4074a30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f20c400d950 tx=0x7f20c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:02.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20ca7fc640 1 -- 172.21.15.16:0/47226575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4017030 con 0x7f20d4102790 2024-09-19T09:26:02.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20ca7fc640 1 -- 172.21.15.16:0/47226575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c4004020 con 0x7f20d4102790 2024-09-19T09:26:02.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20db4cc640 1 -- 172.21.15.16:0/47226575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d41a6c40 con 0x7f20d4102790 2024-09-19T09:26:02.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20ca7fc640 1 -- 172.21.15.16:0/47226575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c40047a0 con 0x7f20d4102790 2024-09-19T09:26:02.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.193+0000 7f20db4cc640 1 -- 172.21.15.16:0/47226575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d41a7180 con 0x7f20d4102790 2024-09-19T09:26:02.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.194+0000 7f20db4cc640 1 -- 172.21.15.16:0/47226575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d41021c0 con 0x7f20d4102790 2024-09-19T09:26:02.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.196+0000 7f20ca7fc640 1 -- 172.21.15.16:0/47226575 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f20c40041d0 con 0x7f20d4102790 2024-09-19T09:26:02.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.196+0000 7f20ca7fc640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f20c00a3350 0x7f20c00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:02.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.196+0000 7f20d9268640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f20c00a3350 0x7f20c00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:02.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.196+0000 7f20ca7fc640 1 -- 172.21.15.16:0/47226575 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f20c40cf4c0 con 0x7f20d4102790 2024-09-19T09:26:02.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.197+0000 7f20d9268640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f20c00a3350 0x7f20c00a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20d0005ab0 tx=0x7f20d0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:02.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.200+0000 7f20ca7fc640 1 -- 172.21.15.16:0/47226575 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f20c409c080 con 0x7f20d4102790 2024-09-19T09:26:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.376+0000 7f20db4cc640 1 -- 172.21.15.16:0/47226575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20d410a0f0 con 0x7f20d4102790 2024-09-19T09:26:02.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.377+0000 7f20ca7fc640 1 -- 172.21.15.16:0/47226575 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22665 (secure 0 0 0) 0x7f20c409b980 con 0x7f20d4102790 2024-09-19T09:26:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:26:02.380 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","created":"2024-09-19T09:17:48.242238+0000","modified":"2024-09-19T09:25:42.982671+0000","last_up_change":"2024-09-19T09:24:40.190289+0000","last_in_change":"2024-09-19T09:24:17.190380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:20:57.748686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:25:25.694451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:25:28.262997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbf3b28b-2282-4057-9e65-19ceb78ff19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6803","nonce":1578957623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6805","nonce":1578957623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6809","nonce":1578957623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6807","nonce":1578957623}]},"public_addr":"172.21.15.16:6803/1578957623","cluster_addr":"172.21.15.16:6805/1578957623","heartbeat_back_addr":"172.21.15.16:6809/1578957623","heartbeat_front_addr":"172.21.15.16:6807/1578957623","state":["exists","up"]},{"osd":1,"uuid":"c2517aeb-1d79-41ad-a180-369eecb9732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6801","nonce":3804152349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6803","nonce":3804152349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6807","nonce":3804152349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6805","nonce":3804152349}]},"public_addr":"172.21.15.97:6801/3804152349","cluster_addr":"172.21.15.97:6803/3804152349","heartbeat_back_addr":"172.21.15.97:6807/3804152349","heartbeat_front_addr":"172.21.15.97:6805/3804152349","state":["exists","up"]},{"osd":2,"uuid":"3c367518-5085-471a-98cb-ec5bf5facdf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6801","nonce":2692613951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6803","nonce":2692613951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6807","nonce":2692613951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6805","nonce":2692613951}]},"public_addr":"172.21.15.142:6801/2692613951","cluster_addr":"172.21.15.142:6803/2692613951","heartbeat_back_addr":"172.21.15.142:6807/2692613951","heartbeat_front_addr":"172.21.15.142:6805/2692613951","state":["exists","up"]},{"osd":3,"uuid":"e7118925-f0fa-46b3-8552-3e3bb76a6489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6811","nonce":3169290004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6813","nonce":3169290004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6817","nonce":3169290004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6815","nonce":3169290004}]},"public_addr":"172.21.15.16:6811/3169290004","cluster_addr":"172.21.15.16:6813/3169290004","heartbeat_back_addr":"172.21.15.16:6817/3169290004","heartbeat_front_addr":"172.21.15.16:6815/3169290004","state":["exists","up"]},{"osd":4,"uuid":"dc1964f7-af9c-4853-843b-9818f87c7768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6809","nonce":3491353825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6811","nonce":3491353825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6815","nonce":3491353825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6813","nonce":3491353825}]},"public_addr":"172.21.15.97:6809/3491353825","cluster_addr":"172.21.15.97:6811/3491353825","heartbeat_back_addr":"172.21.15.97:6815/3491353825","heartbeat_front_addr":"172.21.15.97:6813/3491353825","state":["exists","up"]},{"osd":5,"uuid":"4d7fbafc-d9ba-4036-a89d-aa295b8a23a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6809","nonce":577099986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6811","nonce":577099986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6815","nonce":577099986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6813","nonce":577099986}]},"public_addr":"172.21.15.142:6809/577099986","cluster_addr":"172.21.15.142:6811/577099986","heartbeat_back_addr":"172.21.15.142:6815/577099986","heartbeat_front_addr":"172.21.15.142:6813/577099986","state":["exists","up"]},{"osd":6,"uuid":"e3f346fc-c027-44f8-bd37-bccf6ec09e3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6819","nonce":4053539536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6821","nonce":4053539536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6825","nonce":4053539536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6823","nonce":4053539536}]},"public_addr":"172.21.15.16:6819/4053539536","cluster_addr":"172.21.15.16:6821/4053539536","heartbeat_back_addr":"172.21.15.16:6825/4053539536","heartbeat_front_addr":"172.21.15.16:6823/4053539536","state":["exists","up"]},{"osd":7,"uuid":"d390b547-8949-4d4b-8481-c49d6ae7e424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6817","nonce":1157122670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6819","nonce":1157122670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6823","nonce":1157122670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6821","nonce":1157122670}]},"public_addr":"172.21.15.97:6817/1157122670","cluster_addr":"172.21.15.97:6819/1157122670","heartbeat_back_addr":"172.21.15.97:6823/1157122670","heartbeat_front_addr":"172.21.15.97:6821/1157122670","state":["exists","up"]},{"osd":8,"uuid":"78a476c4-5deb-4f9e-a266-db4493850732","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6817","nonce":643795004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6819","nonce":643795004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6823","nonce":643795004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6821","nonce":643795004}]},"public_addr":"172.21.15.142:6817/643795004","cluster_addr":"172.21.15.142:6819/643795004","heartbeat_back_addr":"172.21.15.142:6823/643795004","heartbeat_front_addr":"172.21.15.142:6821/643795004","state":["exists","up"]},{"osd":9,"uuid":"0415d187-4a77-42f7-b8bb-2d0b52e9cb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6827","nonce":2651132360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6829","nonce":2651132360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2651132360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6831","nonce":2651132360}]},"public_addr":"172.21.15.16:6827/2651132360","cluster_addr":"172.21.15.16:6829/2651132360","heartbeat_back_addr":"172.21.15.16:6833/2651132360","heartbeat_front_addr":"172.21.15.16:6831/2651132360","state":["exists","up"]},{"osd":10,"uuid":"6fa78f96-cdfe-42a0-8d27-0705a0c7df8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6825","nonce":1848371486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6827","nonce":1848371486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6831","nonce":1848371486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6829","nonce":1848371486}]},"public_addr":"172.21.15.97:6825/1848371486","cluster_addr":"172.21.15.97:6827/1848371486","heartbeat_back_addr":"172.21.15.97:6831/1848371486","heartbeat_front_addr":"172.21.15.97:6829/1848371486","state":["exists","up"]},{"osd":11,"uuid":"d27f20e3-84ce-470c-9889-cd86906b158d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6825","nonce":155083878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6827","nonce":155083878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6831","nonce":155083878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6829","nonce":155083878}]},"public_addr":"172.21.15.142:6825/155083878","cluster_addr":"172.21.15.142:6827/155083878","heartbeat_back_addr":"172.21.15.142:6831/155083878","heartbeat_front_addr":"172.21.15.142:6829/155083878","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:05.088764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:29.928529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:55.236930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:20.379471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:44.984151+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:10.410074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:35.233553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:59.928700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:24.503069+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:49.853073+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:14.006841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:38.546467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3710267992":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3740275561":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/2755421444":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/357390846":"2024-09-20T09:18:17.070034+0000","172.21.15.16:6800/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3104087402":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3546947161":"2024-09-20T09:18:17.070034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:26:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.379+0000 7f20db4cc640 1 -- 172.21.15.16:0/47226575 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f20c00a3350 msgr2=0x7f20c00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.379+0000 7f20db4cc640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f20c00a3350 0x7f20c00a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f20d0005ab0 tx=0x7f20d0005a20 comp rx=0 tx=0).stop 2024-09-19T09:26:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.379+0000 7f20db4cc640 1 -- 172.21.15.16:0/47226575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d4102790 msgr2=0x7f20d4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.379+0000 7f20db4cc640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d4102790 0x7f20d4074a30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f20c400d950 tx=0x7f20c400de10 comp rx=0 tx=0).stop 2024-09-19T09:26:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.380+0000 7f20db4cc640 1 -- 172.21.15.16:0/47226575 shutdown_connections 2024-09-19T09:26:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.380+0000 7f20db4cc640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f20c00a3350 0x7f20c00a5810 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.380+0000 7f20db4cc640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20d4106f10 0x7f20d4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.380+0000 7f20db4cc640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d4102790 0x7f20d4074a30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.380+0000 7f20db4cc640 1 --2- 172.21.15.16:0/47226575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20d4101de0 0x7f20d40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.380+0000 7f20db4cc640 1 -- 172.21.15.16:0/47226575 >> 172.21.15.16:0/47226575 conn(0x7f20d40fd8c0 msgr2=0x7f20d40fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.380+0000 7f20db4cc640 1 -- 172.21.15.16:0/47226575 shutdown_connections 2024-09-19T09:26:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:02.380+0000 7f20db4cc640 1 -- 172.21.15.16:0/47226575 wait complete. 2024-09-19T09:26:03.084 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T09:26:03.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T09:26:03.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T09:26:03.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T09:26:03.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T09:26:03.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T09:26:03.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T09:26:03.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T09:26:03.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T09:26:03.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T09:26:03.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T09:26:03.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T09:26:03.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:03 smithi142 ceph-mon[35032]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-19T09:26:03.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:03 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/47226575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:26:03.385 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:03 smithi097 ceph-mon[34864]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-19T09:26:03.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:03 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/47226575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:26:03.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:03 smithi016 ceph-mon[30979]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-19T09:26:03.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:03 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/47226575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:26:03.789 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:03.818 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.692+0000 7faac8267640 1 -- 172.21.15.16:0/3251791540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac010c170 msgr2=0x7faac010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.692+0000 7faac8267640 1 --2- 172.21.15.16:0/3251791540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac010c170 0x7faac010e610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faab8009ca0 tx=0x7faab802a950 comp rx=0 tx=0).stop 2024-09-19T09:26:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.692+0000 7faac8267640 1 -- 172.21.15.16:0/3251791540 shutdown_connections 2024-09-19T09:26:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.692+0000 7faac8267640 1 --2- 172.21.15.16:0/3251791540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac0103660 0x7faac010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.692+0000 7faac8267640 1 --2- 172.21.15.16:0/3251791540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac010c170 0x7faac010e610 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.692+0000 7faac8267640 1 --2- 172.21.15.16:0/3251791540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac0102d40 0x7faac0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.692+0000 7faac8267640 1 -- 172.21.15.16:0/3251791540 >> 172.21.15.16:0/3251791540 conn(0x7faac00fc7a0 msgr2=0x7faac00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.692+0000 7faac8267640 1 -- 172.21.15.16:0/3251791540 shutdown_connections 2024-09-19T09:26:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.692+0000 7faac8267640 1 -- 172.21.15.16:0/3251791540 wait complete. 2024-09-19T09:26:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.693+0000 7faac8267640 1 Processor -- start 2024-09-19T09:26:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.693+0000 7faac8267640 1 -- start start 2024-09-19T09:26:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac8267640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac0102d40 0x7faac0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac8267640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac0103660 0x7faac0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac8267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac010c170 0x7faac019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac8267640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faac0111fc0 con 0x7faac0102d40 2024-09-19T09:26:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac8267640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faac0111e40 con 0x7faac010c170 2024-09-19T09:26:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac8267640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faac0112140 con 0x7faac0103660 2024-09-19T09:26:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac6003640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac0102d40 0x7faac0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac6003640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac0102d40 0x7faac0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49686/0 (socket says 172.21.15.16:49686) 2024-09-19T09:26:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac6003640 1 -- 172.21.15.16:0/4254432977 learned_addr learned my addr 172.21.15.16:0/4254432977 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac5802640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac0103660 0x7faac0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac6804640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac010c170 0x7faac019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac6003640 1 -- 172.21.15.16:0/4254432977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac0103660 msgr2=0x7faac0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac6003640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac0103660 0x7faac0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac6003640 1 -- 172.21.15.16:0/4254432977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac010c170 msgr2=0x7faac019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac6003640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac010c170 0x7faac019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac6003640 1 -- 172.21.15.16:0/4254432977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac019d3b0 con 0x7faac0102d40 2024-09-19T09:26:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac6804640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac010c170 0x7faac019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac5802640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faac0103660 0x7faac0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac6003640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac0102d40 0x7faac0198400 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faabc00e9f0 tx=0x7faabc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faab77fe640 1 -- 172.21.15.16:0/4254432977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc00cd80 con 0x7faac0102d40 2024-09-19T09:26:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faab77fe640 1 -- 172.21.15.16:0/4254432977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faabc010910 con 0x7faac0102d40 2024-09-19T09:26:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faac8267640 1 -- 172.21.15.16:0/4254432977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac019d6a0 con 0x7faac0102d40 2024-09-19T09:26:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.694+0000 7faab77fe640 1 -- 172.21.15.16:0/4254432977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc018990 con 0x7faac0102d40 2024-09-19T09:26:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.695+0000 7faac8267640 1 -- 172.21.15.16:0/4254432977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac01a4f80 con 0x7faac0102d40 2024-09-19T09:26:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.695+0000 7faac8267640 1 -- 172.21.15.16:0/4254432977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faac0062980 con 0x7faac0102d40 2024-09-19T09:26:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.697+0000 7faab77fe640 1 -- 172.21.15.16:0/4254432977 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7faabc010ac0 con 0x7faac0102d40 2024-09-19T09:26:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.697+0000 7faab77fe640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7faaac0a3350 0x7faaac0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.697+0000 7faab77fe640 1 -- 172.21.15.16:0/4254432977 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faabc0d0530 con 0x7faac0102d40 2024-09-19T09:26:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.697+0000 7faac5802640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7faaac0a3350 0x7faaac0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.697+0000 7faab77fe640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623] conn(0x7faaac0acf80 0x7faaac0af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.697+0000 7faab77fe640 1 -- 172.21.15.16:0/4254432977 --> [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faabc019ec0 con 0x7faaac0acf80 2024-09-19T09:26:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.697+0000 7faab77fe640 1 -- 172.21.15.16:0/4254432977 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7faabc0d0960 con 0x7faac0102d40 2024-09-19T09:26:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.698+0000 7faac6804640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623] conn(0x7faaac0acf80 0x7faaac0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.698+0000 7faac5802640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7faaac0a3350 0x7faaac0a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faac00fed30 tx=0x7faab000b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.698+0000 7faac6804640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623] conn(0x7faaac0acf80 0x7faaac0af420 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.699+0000 7faab77fe640 1 -- 172.21.15.16:0/4254432977 <== osd.0 v2:172.21.15.16:6802/1578957623 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faabc019ec0 con 0x7faaac0acf80 2024-09-19T09:26:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.713+0000 7faac8267640 1 -- 172.21.15.16:0/4254432977 --> [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faac0062db0 con 0x7faaac0acf80 2024-09-19T09:26:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.713+0000 7faab77fe640 1 -- 172.21.15.16:0/4254432977 <== osd.0 v2:172.21.15.16:6802/1578957623 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faac0062db0 con 0x7faaac0acf80 2024-09-19T09:26:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.713+0000 7faac8267640 1 -- 172.21.15.16:0/4254432977 >> [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623] conn(0x7faaac0acf80 msgr2=0x7faaac0af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.713+0000 7faac8267640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623] conn(0x7faaac0acf80 0x7faaac0af420 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.713+0000 7faac8267640 1 -- 172.21.15.16:0/4254432977 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7faaac0a3350 msgr2=0x7faaac0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.713+0000 7faac8267640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7faaac0a3350 0x7faaac0a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faac00fed30 tx=0x7faab000b610 comp rx=0 tx=0).stop 2024-09-19T09:26:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.713+0000 7faac8267640 1 -- 172.21.15.16:0/4254432977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac0102d40 msgr2=0x7faac0198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.713+0000 7faac8267640 1 --2- 172.21.15.16:0/4254432977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac0102d40 0x7faac0198400 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faabc00e9f0 tx=0x7faabc00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:26:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.713+0000 7faac6804640 1 -- 172.21.15.16:0/4254432977 reap_dead start 2024-09-19T09:26:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.713+0000 7faac8267640 1 -- 172.21.15.16:0/4254432977 shutdown_connections 2024-09-19T09:26:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.713+0000 7faac8267640 1 -- 172.21.15.16:0/4254432977 >> 172.21.15.16:0/4254432977 conn(0x7faac00fc7a0 msgr2=0x7faac00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.713+0000 7faac8267640 1 -- 172.21.15.16:0/4254432977 shutdown_connections 2024-09-19T09:26:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:04.714+0000 7faac8267640 1 -- 172.21.15.16:0/4254432977 wait complete. 2024-09-19T09:26:05.135 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:05.136 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:05.246 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:05.250 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:05.250 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:05.299 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:05 smithi016 ceph-mon[30979]: pgmap v292: 97 pgs: 97 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-09-19T09:26:05.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:05 smithi142 ceph-mon[35032]: pgmap v292: 97 pgs: 97 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-09-19T09:26:05.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:05 smithi097 ceph-mon[34864]: pgmap v292: 97 pgs: 97 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-09-19T09:26:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.464+0000 7f57695d6640 1 -- 172.21.15.16:0/3716475445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5764101de0 msgr2=0x7f57641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.464+0000 7f57695d6640 1 --2- 172.21.15.16:0/3716475445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5764101de0 0x7f57641021c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5754009900 tx=0x7f575402a410 comp rx=0 tx=0).stop 2024-09-19T09:26:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.464+0000 7f57695d6640 1 -- 172.21.15.16:0/3716475445 shutdown_connections 2024-09-19T09:26:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.464+0000 7f57695d6640 1 --2- 172.21.15.16:0/3716475445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5764102790 0x7f57641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.464+0000 7f57695d6640 1 --2- 172.21.15.16:0/3716475445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764106f10 0x7f576410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.464+0000 7f57695d6640 1 --2- 172.21.15.16:0/3716475445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5764101de0 0x7f57641021c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.464+0000 7f57695d6640 1 -- 172.21.15.16:0/3716475445 >> 172.21.15.16:0/3716475445 conn(0x7f57640fd8c0 msgr2=0x7f57640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.464+0000 7f57695d6640 1 -- 172.21.15.16:0/3716475445 shutdown_connections 2024-09-19T09:26:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.464+0000 7f57695d6640 1 -- 172.21.15.16:0/3716475445 wait complete. 2024-09-19T09:26:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.464+0000 7f57695d6640 1 Processor -- start 2024-09-19T09:26:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f57695d6640 1 -- start start 2024-09-19T09:26:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f57695d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5764101de0 0x7f57640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f57695d6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764102790 0x7f5764074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f57695d6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5764106f10 0x7f5764074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f57695d6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5764111f90 con 0x7f5764101de0 2024-09-19T09:26:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f57695d6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5764111e10 con 0x7f5764102790 2024-09-19T09:26:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f57695d6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5764112110 con 0x7f5764106f10 2024-09-19T09:26:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f5762ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5764101de0 0x7f57640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f5762ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5764101de0 0x7f57640744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39134/0 (socket says 172.21.15.16:39134) 2024-09-19T09:26:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f5762ffd640 1 -- 172.21.15.16:0/57409121 learned_addr learned my addr 172.21.15.16:0/57409121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f575bfff640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764102790 0x7f5764074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f5762ffd640 1 -- 172.21.15.16:0/57409121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5764106f10 msgr2=0x7f5764074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f5762ffd640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5764106f10 0x7f5764074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f5762ffd640 1 -- 172.21.15.16:0/57409121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764102790 msgr2=0x7f5764074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f5762ffd640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764102790 0x7f5764074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f5762ffd640 1 -- 172.21.15.16:0/57409121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57641a6890 con 0x7f5764101de0 2024-09-19T09:26:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f575bfff640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5764102790 0x7f5764074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.465+0000 7f5762ffd640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5764101de0 0x7f57640744f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5754005ab0 tx=0x7f57540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.466+0000 7f5760ff9640 1 -- 172.21.15.16:0/57409121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575403f030 con 0x7f5764101de0 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.466+0000 7f5760ff9640 1 -- 172.21.15.16:0/57409121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5754003ce0 con 0x7f5764101de0 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.466+0000 7f5760ff9640 1 -- 172.21.15.16:0/57409121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754035070 con 0x7f5764101de0 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.466+0000 7f57695d6640 1 -- 172.21.15.16:0/57409121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57641a6b20 con 0x7f5764101de0 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.466+0000 7f57695d6640 1 -- 172.21.15.16:0/57409121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57641a7050 con 0x7f5764101de0 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.467+0000 7f57695d6640 1 -- 172.21.15.16:0/57409121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5764062900 con 0x7f5764101de0 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.468+0000 7f5760ff9640 1 -- 172.21.15.16:0/57409121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5754042030 con 0x7f5764101de0 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.468+0000 7f5760ff9640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f57500a3350 0x7f57500a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.469+0000 7f575bfff640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f57500a3350 0x7f57500a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.469+0000 7f5760ff9640 1 -- 172.21.15.16:0/57409121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f57540f0670 con 0x7f5764101de0 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.469+0000 7f5760ff9640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349] conn(0x7f57500acf80 0x7f57500af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.469+0000 7f5760ff9640 1 -- 172.21.15.16:0/57409121 --> [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5754003b30 con 0x7f57500acf80 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.469+0000 7f5760ff9640 1 -- 172.21.15.16:0/57409121 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f57540f0aa0 con 0x7f5764101de0 2024-09-19T09:26:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.469+0000 7f575bfff640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f57500a3350 0x7f57500a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5764075f80 tx=0x7f574c005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.469+0000 7f57637fe640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349] conn(0x7f57500acf80 0x7f57500af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.470+0000 7f57637fe640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349] conn(0x7f57500acf80 0x7f57500af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.470+0000 7f5760ff9640 1 -- 172.21.15.16:0/57409121 <== osd.1 v2:172.21.15.97:6800/3804152349 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5754003b30 con 0x7f57500acf80 2024-09-19T09:26:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.484+0000 7f57695d6640 1 -- 172.21.15.16:0/57409121 --> [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5764062d00 con 0x7f57500acf80 2024-09-19T09:26:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.484+0000 7f5760ff9640 1 -- 172.21.15.16:0/57409121 <== osd.1 v2:172.21.15.97:6800/3804152349 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5764062d00 con 0x7f57500acf80 2024-09-19T09:26:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.485+0000 7f57695d6640 1 -- 172.21.15.16:0/57409121 >> [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349] conn(0x7f57500acf80 msgr2=0x7f57500af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.485+0000 7f57695d6640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349] conn(0x7f57500acf80 0x7f57500af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.485+0000 7f57695d6640 1 -- 172.21.15.16:0/57409121 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f57500a3350 msgr2=0x7f57500a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.485+0000 7f57695d6640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f57500a3350 0x7f57500a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5764075f80 tx=0x7f574c005d90 comp rx=0 tx=0).stop 2024-09-19T09:26:05.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.485+0000 7f57695d6640 1 -- 172.21.15.16:0/57409121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5764101de0 msgr2=0x7f57640744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:05.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.485+0000 7f57695d6640 1 --2- 172.21.15.16:0/57409121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5764101de0 0x7f57640744f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5754005ab0 tx=0x7f57540042f0 comp rx=0 tx=0).stop 2024-09-19T09:26:05.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.485+0000 7f57637fe640 1 -- 172.21.15.16:0/57409121 reap_dead start 2024-09-19T09:26:05.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.485+0000 7f57695d6640 1 -- 172.21.15.16:0/57409121 shutdown_connections 2024-09-19T09:26:05.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.485+0000 7f57695d6640 1 -- 172.21.15.16:0/57409121 >> 172.21.15.16:0/57409121 conn(0x7f57640fd8c0 msgr2=0x7f57640fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:05.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.485+0000 7f57695d6640 1 -- 172.21.15.16:0/57409121 shutdown_connections 2024-09-19T09:26:05.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:05.485+0000 7f57695d6640 1 -- 172.21.15.16:0/57409121 wait complete. 2024-09-19T09:26:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:06.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.599+0000 7fccea7b1640 1 -- 172.21.15.16:0/414825351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce4102790 msgr2=0x7fcce41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:06.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.599+0000 7fccea7b1640 1 --2- 172.21.15.16:0/414825351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce4102790 0x7fcce41067e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009900 tx=0x7fcccc02a410 comp rx=0 tx=0).stop 2024-09-19T09:26:06.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.599+0000 7fccea7b1640 1 -- 172.21.15.16:0/414825351 shutdown_connections 2024-09-19T09:26:06.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.599+0000 7fccea7b1640 1 --2- 172.21.15.16:0/414825351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4106f10 0x7fcce410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:06.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.599+0000 7fccea7b1640 1 --2- 172.21.15.16:0/414825351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce4102790 0x7fcce41067e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:06.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.599+0000 7fccea7b1640 1 --2- 172.21.15.16:0/414825351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce4101de0 0x7fcce41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:06.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.599+0000 7fccea7b1640 1 -- 172.21.15.16:0/414825351 >> 172.21.15.16:0/414825351 conn(0x7fcce40fd8c0 msgr2=0x7fcce40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:06.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.599+0000 7fccea7b1640 1 -- 172.21.15.16:0/414825351 shutdown_connections 2024-09-19T09:26:06.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.599+0000 7fccea7b1640 1 -- 172.21.15.16:0/414825351 wait complete. 2024-09-19T09:26:06.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fccea7b1640 1 Processor -- start 2024-09-19T09:26:06.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fccea7b1640 1 -- start start 2024-09-19T09:26:06.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fccea7b1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4101de0 0x7fcce4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:06.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fccea7b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce4102790 0x7fcce4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:06.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fccea7b1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce4106f10 0x7fcce419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:06.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fccea7b1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcce4111f90 con 0x7fcce4102790 2024-09-19T09:26:06.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fccea7b1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcce4111e10 con 0x7fcce4101de0 2024-09-19T09:26:06.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fccea7b1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcce4112110 con 0x7fcce4106f10 2024-09-19T09:26:06.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fccdbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce4102790 0x7fcce4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:06.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fccdbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce4102790 0x7fcce4198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39192/0 (socket says 172.21.15.16:39192) 2024-09-19T09:26:06.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fccdbfff640 1 -- 172.21.15.16:0/2665923633 learned_addr learned my addr 172.21.15.16:0/2665923633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:06.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fccdbfff640 1 -- 172.21.15.16:0/2665923633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce4106f10 msgr2=0x7fcce419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fcce3fff640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4101de0 0x7fcce4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.600+0000 7fcce8d4e640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce4106f10 0x7fcce419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.601+0000 7fccdbfff640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce4106f10 0x7fcce419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.601+0000 7fccdbfff640 1 -- 172.21.15.16:0/2665923633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4101de0 msgr2=0x7fcce4198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.601+0000 7fccdbfff640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4101de0 0x7fcce4198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.601+0000 7fccdbfff640 1 -- 172.21.15.16:0/2665923633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce419d500 con 0x7fcce4102790 2024-09-19T09:26:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.601+0000 7fcce8d4e640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce4106f10 0x7fcce419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.601+0000 7fcce3fff640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcce4101de0 0x7fcce4198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.601+0000 7fccdbfff640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce4102790 0x7fcce4198a90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcccc005ab0 tx=0x7fcccc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.601+0000 7fcce1ffb640 1 -- 172.21.15.16:0/2665923633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc03f030 con 0x7fcce4102790 2024-09-19T09:26:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.601+0000 7fcce1ffb640 1 -- 172.21.15.16:0/2665923633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcccc003ce0 con 0x7fcce4102790 2024-09-19T09:26:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.601+0000 7fccea7b1640 1 -- 172.21.15.16:0/2665923633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce419d790 con 0x7fcce4102790 2024-09-19T09:26:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.601+0000 7fcce1ffb640 1 -- 172.21.15.16:0/2665923633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc035070 con 0x7fcce4102790 2024-09-19T09:26:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.601+0000 7fccea7b1640 1 -- 172.21.15.16:0/2665923633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce41a50d0 con 0x7fcce4102790 2024-09-19T09:26:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.602+0000 7fccea7b1640 1 -- 172.21.15.16:0/2665923633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcce4062900 con 0x7fcce4102790 2024-09-19T09:26:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.603+0000 7fcce1ffb640 1 -- 172.21.15.16:0/2665923633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fcccc042030 con 0x7fcce4102790 2024-09-19T09:26:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.604+0000 7fcce1ffb640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fccd00a32d0 0x7fccd00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.604+0000 7fcce3fff640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fccd00a32d0 0x7fccd00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.604+0000 7fcce1ffb640 1 -- 172.21.15.16:0/2665923633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fcccc0f06e0 con 0x7fcce4102790 2024-09-19T09:26:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.605+0000 7fcce1ffb640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486] conn(0x7fccd00acf00 0x7fccd00af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.605+0000 7fcce1ffb640 1 -- 172.21.15.16:0/2665923633 --> [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcccc001d30 con 0x7fccd00acf00 2024-09-19T09:26:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.605+0000 7fcce1ffb640 1 -- 172.21.15.16:0/2665923633 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fcccc0f0b10 con 0x7fcce4102790 2024-09-19T09:26:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.605+0000 7fcce3fff640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fccd00a32d0 0x7fccd00a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fccd4006fd0 tx=0x7fccd4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.605+0000 7fcce8d4e640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486] conn(0x7fccd00acf00 0x7fccd00af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.606+0000 7fcce8d4e640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486] conn(0x7fccd00acf00 0x7fccd00af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:06.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.607+0000 7fcce1ffb640 1 -- 172.21.15.16:0/2665923633 <== osd.10 v2:172.21.15.97:6824/1848371486 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcccc001d30 con 0x7fccd00acf00 2024-09-19T09:26:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.620+0000 7fccea7b1640 1 -- 172.21.15.16:0/2665923633 --> [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcce4062d00 con 0x7fccd00acf00 2024-09-19T09:26:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.621+0000 7fcce1ffb640 1 -- 172.21.15.16:0/2665923633 <== osd.10 v2:172.21.15.97:6824/1848371486 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcce4062d00 con 0x7fccd00acf00 2024-09-19T09:26:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.621+0000 7fccea7b1640 1 -- 172.21.15.16:0/2665923633 >> [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486] conn(0x7fccd00acf00 msgr2=0x7fccd00af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.621+0000 7fccea7b1640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486] conn(0x7fccd00acf00 0x7fccd00af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.621+0000 7fccea7b1640 1 -- 172.21.15.16:0/2665923633 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fccd00a32d0 msgr2=0x7fccd00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.621+0000 7fccea7b1640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fccd00a32d0 0x7fccd00a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fccd4006fd0 tx=0x7fccd4008040 comp rx=0 tx=0).stop 2024-09-19T09:26:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.621+0000 7fccea7b1640 1 -- 172.21.15.16:0/2665923633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce4102790 msgr2=0x7fcce4198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.621+0000 7fccea7b1640 1 --2- 172.21.15.16:0/2665923633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce4102790 0x7fcce4198a90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcccc005ab0 tx=0x7fcccc0042f0 comp rx=0 tx=0).stop 2024-09-19T09:26:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.621+0000 7fcce8d4e640 1 -- 172.21.15.16:0/2665923633 reap_dead start 2024-09-19T09:26:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.621+0000 7fccea7b1640 1 -- 172.21.15.16:0/2665923633 shutdown_connections 2024-09-19T09:26:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.621+0000 7fccea7b1640 1 -- 172.21.15.16:0/2665923633 >> 172.21.15.16:0/2665923633 conn(0x7fcce40fd8c0 msgr2=0x7fcce40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.622+0000 7fccea7b1640 1 -- 172.21.15.16:0/2665923633 shutdown_connections 2024-09-19T09:26:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.622+0000 7fccea7b1640 1 -- 172.21.15.16:0/2665923633 wait complete. 2024-09-19T09:26:07.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:07 smithi016 ceph-mon[30979]: pgmap v293: 97 pgs: 97 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T09:26:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.316+0000 7f7809950640 1 -- 172.21.15.16:0/1172740713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7804103660 msgr2=0x7f780410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.316+0000 7f7809950640 1 --2- 172.21.15.16:0/1172740713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7804103660 0x7f780410bb50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009900 tx=0x7f77ec02a480 comp rx=0 tx=0).stop 2024-09-19T09:26:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.316+0000 7f7809950640 1 -- 172.21.15.16:0/1172740713 shutdown_connections 2024-09-19T09:26:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.316+0000 7f7809950640 1 --2- 172.21.15.16:0/1172740713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f780410c170 0x7f780410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.316+0000 7f7809950640 1 --2- 172.21.15.16:0/1172740713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7804103660 0x7f780410bb50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.316+0000 7f7809950640 1 --2- 172.21.15.16:0/1172740713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7804102d40 0x7f7804103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.316+0000 7f7809950640 1 -- 172.21.15.16:0/1172740713 >> 172.21.15.16:0/1172740713 conn(0x7f78040fc7a0 msgr2=0x7f78040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.316+0000 7f7809950640 1 -- 172.21.15.16:0/1172740713 shutdown_connections 2024-09-19T09:26:07.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.316+0000 7f7809950640 1 -- 172.21.15.16:0/1172740713 wait complete. 2024-09-19T09:26:07.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.317+0000 7f7809950640 1 Processor -- start 2024-09-19T09:26:07.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.317+0000 7f7809950640 1 -- start start 2024-09-19T09:26:07.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.317+0000 7f7809950640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7804102d40 0x7f7804198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.317+0000 7f7809950640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7804103660 0x7f7804198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.317+0000 7f7809950640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780410c170 0x7f780419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.317+0000 7f7809950640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7804111fb0 con 0x7f780410c170 2024-09-19T09:26:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.317+0000 7f7809950640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7804111e30 con 0x7f7804103660 2024-09-19T09:26:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.317+0000 7f7809950640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7804112130 con 0x7f7804102d40 2024-09-19T09:26:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.317+0000 7f78037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780410c170 0x7f780419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.317+0000 7f78037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780410c170 0x7f780419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39204/0 (socket says 172.21.15.16:39204) 2024-09-19T09:26:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.317+0000 7f78037fe640 1 -- 172.21.15.16:0/4264772500 learned_addr learned my addr 172.21.15.16:0/4264772500 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f7802ffd640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7804102d40 0x7f7804198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f77fa7fc640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7804103660 0x7f7804198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f78037fe640 1 -- 172.21.15.16:0/4264772500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7804102d40 msgr2=0x7f7804198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f78037fe640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7804102d40 0x7f7804198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f78037fe640 1 -- 172.21.15.16:0/4264772500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7804103660 msgr2=0x7f7804198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f78037fe640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7804103660 0x7f7804198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f78037fe640 1 -- 172.21.15.16:0/4264772500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780419d3e0 con 0x7f780410c170 2024-09-19T09:26:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f77fa7fc640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7804103660 0x7f7804198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f78037fe640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780410c170 0x7f780419cd00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00b970 tx=0x7f77fc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f780894e640 1 -- 172.21.15.16:0/4264772500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc019070 con 0x7f780410c170 2024-09-19T09:26:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f780894e640 1 -- 172.21.15.16:0/4264772500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77fc00cb40 con 0x7f780410c170 2024-09-19T09:26:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f7809950640 1 -- 172.21.15.16:0/4264772500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780419d6d0 con 0x7f780410c170 2024-09-19T09:26:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f780894e640 1 -- 172.21.15.16:0/4264772500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc007650 con 0x7f780410c170 2024-09-19T09:26:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.318+0000 7f7809950640 1 -- 172.21.15.16:0/4264772500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78041a4fb0 con 0x7f780410c170 2024-09-19T09:26:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.319+0000 7f7809950640 1 -- 172.21.15.16:0/4264772500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7804062940 con 0x7f780410c170 2024-09-19T09:26:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.320+0000 7f780894e640 1 -- 172.21.15.16:0/4264772500 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f77fc01c070 con 0x7f780410c170 2024-09-19T09:26:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.321+0000 7f780894e640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f77f00a3350 0x7f77f00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.321+0000 7f7802ffd640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f77f00a3350 0x7f77f00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.321+0000 7f780894e640 1 -- 172.21.15.16:0/4264772500 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f77fc0cf7a0 con 0x7f780410c170 2024-09-19T09:26:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.322+0000 7f780894e640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004] conn(0x7f77f00acf80 0x7f77f00af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.322+0000 7f780894e640 1 -- 172.21.15.16:0/4264772500 --> [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77fc0047d0 con 0x7f77f00acf80 2024-09-19T09:26:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.322+0000 7f780894e640 1 -- 172.21.15.16:0/4264772500 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f77fc0cfb50 con 0x7f780410c170 2024-09-19T09:26:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.322+0000 7f77fa7fc640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004] conn(0x7f77f00acf80 0x7f77f00af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.322+0000 7f7802ffd640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f77f00a3350 0x7f77f00a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f77f4005e10 tx=0x7f77f4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.323+0000 7f77fa7fc640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004] conn(0x7f77f00acf80 0x7f77f00af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:07.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.323+0000 7f780894e640 1 -- 172.21.15.16:0/4264772500 <== osd.3 v2:172.21.15.16:6810/3169290004 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77fc0047d0 con 0x7f77f00acf80 2024-09-19T09:26:07.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.337+0000 7f7809950640 1 -- 172.21.15.16:0/4264772500 --> [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7804062dc0 con 0x7f77f00acf80 2024-09-19T09:26:07.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.338+0000 7f780894e640 1 -- 172.21.15.16:0/4264772500 <== osd.3 v2:172.21.15.16:6810/3169290004 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7804062dc0 con 0x7f77f00acf80 2024-09-19T09:26:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.338+0000 7f7809950640 1 -- 172.21.15.16:0/4264772500 >> [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004] conn(0x7f77f00acf80 msgr2=0x7f77f00af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.338+0000 7f7809950640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004] conn(0x7f77f00acf80 0x7f77f00af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.338+0000 7f7809950640 1 -- 172.21.15.16:0/4264772500 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f77f00a3350 msgr2=0x7f77f00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.338+0000 7f7809950640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f77f00a3350 0x7f77f00a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f77f4005e10 tx=0x7f77f4005da0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.338+0000 7f7809950640 1 -- 172.21.15.16:0/4264772500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780410c170 msgr2=0x7f780419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.338+0000 7f7809950640 1 --2- 172.21.15.16:0/4264772500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780410c170 0x7f780419cd00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00b970 tx=0x7f77fc00be30 comp rx=0 tx=0).stop 2024-09-19T09:26:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.338+0000 7f78037fe640 1 -- 172.21.15.16:0/4264772500 reap_dead start 2024-09-19T09:26:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.338+0000 7f7809950640 1 -- 172.21.15.16:0/4264772500 shutdown_connections 2024-09-19T09:26:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.338+0000 7f7809950640 1 -- 172.21.15.16:0/4264772500 >> 172.21.15.16:0/4264772500 conn(0x7f78040fc7a0 msgr2=0x7f78040fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.338+0000 7f7809950640 1 -- 172.21.15.16:0/4264772500 shutdown_connections 2024-09-19T09:26:07.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.338+0000 7f7809950640 1 -- 172.21.15.16:0/4264772500 wait complete. 2024-09-19T09:26:07.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:07 smithi142 ceph-mon[35032]: pgmap v293: 97 pgs: 97 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T09:26:07.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:07 smithi097 ceph-mon[34864]: pgmap v293: 97 pgs: 97 active+clean; 642 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-19T09:26:07.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.474+0000 7f5b8a3db640 1 -- 172.21.15.16:0/3980872380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b84101de0 msgr2=0x7f5b841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.474+0000 7f5b8a3db640 1 --2- 172.21.15.16:0/3980872380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b84101de0 0x7f5b841021c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009900 tx=0x7f5b7402a410 comp rx=0 tx=0).stop 2024-09-19T09:26:07.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.474+0000 7f5b8a3db640 1 -- 172.21.15.16:0/3980872380 shutdown_connections 2024-09-19T09:26:07.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.474+0000 7f5b8a3db640 1 --2- 172.21.15.16:0/3980872380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b84106f10 0x7f5b8410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.474+0000 7f5b8a3db640 1 --2- 172.21.15.16:0/3980872380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b84102790 0x7f5b841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.474+0000 7f5b8a3db640 1 --2- 172.21.15.16:0/3980872380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b84101de0 0x7f5b841021c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.474+0000 7f5b8a3db640 1 -- 172.21.15.16:0/3980872380 >> 172.21.15.16:0/3980872380 conn(0x7f5b840fd8c0 msgr2=0x7f5b840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:07.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.474+0000 7f5b8a3db640 1 -- 172.21.15.16:0/3980872380 shutdown_connections 2024-09-19T09:26:07.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.474+0000 7f5b8a3db640 1 -- 172.21.15.16:0/3980872380 wait complete. 2024-09-19T09:26:07.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.475+0000 7f5b8a3db640 1 Processor -- start 2024-09-19T09:26:07.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.475+0000 7f5b8a3db640 1 -- start start 2024-09-19T09:26:07.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.475+0000 7f5b8a3db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b84101de0 0x7f5b841984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.475+0000 7f5b8a3db640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b84102790 0x7f5b84198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.475+0000 7f5b8a3db640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b84106f10 0x7f5b8419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.475+0000 7f5b8a3db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84111fc0 con 0x7f5b84101de0 2024-09-19T09:26:07.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.475+0000 7f5b8a3db640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84111e40 con 0x7f5b84102790 2024-09-19T09:26:07.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.475+0000 7f5b8a3db640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84112140 con 0x7f5b84106f10 2024-09-19T09:26:07.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.475+0000 7f5b83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b84101de0 0x7f5b841984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.475+0000 7f5b83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b84101de0 0x7f5b841984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39216/0 (socket says 172.21.15.16:39216) 2024-09-19T09:26:07.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.475+0000 7f5b83fff640 1 -- 172.21.15.16:0/1403673625 learned_addr learned my addr 172.21.15.16:0/1403673625 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:07.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.475+0000 7f5b837fe640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b84102790 0x7f5b84198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b88978640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b84106f10 0x7f5b8419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b83fff640 1 -- 172.21.15.16:0/1403673625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b84106f10 msgr2=0x7f5b8419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b83fff640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b84106f10 0x7f5b8419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b83fff640 1 -- 172.21.15.16:0/1403673625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b84102790 msgr2=0x7f5b84198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b83fff640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b84102790 0x7f5b84198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b83fff640 1 -- 172.21.15.16:0/1403673625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8419d470 con 0x7f5b84101de0 2024-09-19T09:26:07.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b837fe640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b84102790 0x7f5b84198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b88978640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b84106f10 0x7f5b8419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b83fff640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b84101de0 0x7f5b841984c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5b74004770 tx=0x7f5b74003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b817fa640 1 -- 172.21.15.16:0/1403673625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7403f070 con 0x7f5b84101de0 2024-09-19T09:26:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b817fa640 1 -- 172.21.15.16:0/1403673625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b74031ca0 con 0x7f5b84101de0 2024-09-19T09:26:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b8a3db640 1 -- 172.21.15.16:0/1403673625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8419d700 con 0x7f5b84101de0 2024-09-19T09:26:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b817fa640 1 -- 172.21.15.16:0/1403673625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74035030 con 0x7f5b84101de0 2024-09-19T09:26:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.476+0000 7f5b8a3db640 1 -- 172.21.15.16:0/1403673625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b841a5040 con 0x7f5b84101de0 2024-09-19T09:26:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.477+0000 7f5b8a3db640 1 -- 172.21.15.16:0/1403673625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b84062900 con 0x7f5b84101de0 2024-09-19T09:26:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.478+0000 7f5b817fa640 1 -- 172.21.15.16:0/1403673625 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5b74042070 con 0x7f5b84101de0 2024-09-19T09:26:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.479+0000 7f5b817fa640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5b6c0a32d0 0x7f5b6c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.479+0000 7f5b837fe640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5b6c0a32d0 0x7f5b6c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.479+0000 7f5b817fa640 1 -- 172.21.15.16:0/1403673625 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5b740f06b0 con 0x7f5b84101de0 2024-09-19T09:26:07.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.479+0000 7f5b817fa640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986] conn(0x7f5b6c0acf00 0x7f5b6c0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.479+0000 7f5b817fa640 1 -- 172.21.15.16:0/1403673625 --> [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b74038e90 con 0x7f5b6c0acf00 2024-09-19T09:26:07.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.479+0000 7f5b817fa640 1 -- 172.21.15.16:0/1403673625 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f5b740f0ae0 con 0x7f5b84101de0 2024-09-19T09:26:07.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.480+0000 7f5b837fe640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5b6c0a32d0 0x7f5b6c0a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5b780058c0 tx=0x7f5b78005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:07.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.480+0000 7f5b88978640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986] conn(0x7f5b6c0acf00 0x7f5b6c0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.480+0000 7f5b88978640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986] conn(0x7f5b6c0acf00 0x7f5b6c0af3a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:07.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.481+0000 7f5b817fa640 1 -- 172.21.15.16:0/1403673625 <== osd.5 v2:172.21.15.142:6808/577099986 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b74038e90 con 0x7f5b6c0acf00 2024-09-19T09:26:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.495+0000 7f5b8a3db640 1 -- 172.21.15.16:0/1403673625 --> [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b84062d80 con 0x7f5b6c0acf00 2024-09-19T09:26:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.496+0000 7f5b817fa640 1 -- 172.21.15.16:0/1403673625 <== osd.5 v2:172.21.15.142:6808/577099986 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5b84062d80 con 0x7f5b6c0acf00 2024-09-19T09:26:07.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.496+0000 7f5b8a3db640 1 -- 172.21.15.16:0/1403673625 >> [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986] conn(0x7f5b6c0acf00 msgr2=0x7f5b6c0af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.496+0000 7f5b8a3db640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986] conn(0x7f5b6c0acf00 0x7f5b6c0af3a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.496+0000 7f5b8a3db640 1 -- 172.21.15.16:0/1403673625 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5b6c0a32d0 msgr2=0x7f5b6c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.496+0000 7f5b8a3db640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5b6c0a32d0 0x7f5b6c0a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5b780058c0 tx=0x7f5b78005850 comp rx=0 tx=0).stop 2024-09-19T09:26:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.496+0000 7f5b8a3db640 1 -- 172.21.15.16:0/1403673625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b84101de0 msgr2=0x7f5b841984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.496+0000 7f5b8a3db640 1 --2- 172.21.15.16:0/1403673625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b84101de0 0x7f5b841984c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5b74004770 tx=0x7f5b74003c10 comp rx=0 tx=0).stop 2024-09-19T09:26:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.496+0000 7f5b88978640 1 -- 172.21.15.16:0/1403673625 reap_dead start 2024-09-19T09:26:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.496+0000 7f5b8a3db640 1 -- 172.21.15.16:0/1403673625 shutdown_connections 2024-09-19T09:26:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.496+0000 7f5b8a3db640 1 -- 172.21.15.16:0/1403673625 >> 172.21.15.16:0/1403673625 conn(0x7f5b840fd8c0 msgr2=0x7f5b840febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:07.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.496+0000 7f5b8a3db640 1 -- 172.21.15.16:0/1403673625 shutdown_connections 2024-09-19T09:26:07.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.496+0000 7f5b8a3db640 1 -- 172.21.15.16:0/1403673625 wait complete. 2024-09-19T09:26:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.687+0000 7f732ac82640 1 -- 172.21.15.16:0/2015254070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324102570 msgr2=0x7f73241029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.687+0000 7f732ac82640 1 --2- 172.21.15.16:0/2015254070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324102570 0x7f73241029b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7310009900 tx=0x7f731002a480 comp rx=0 tx=0).stop 2024-09-19T09:26:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.687+0000 7f732ac82640 1 -- 172.21.15.16:0/2015254070 shutdown_connections 2024-09-19T09:26:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.687+0000 7f732ac82640 1 --2- 172.21.15.16:0/2015254070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7324102ef0 0x7f732410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.687+0000 7f732ac82640 1 --2- 172.21.15.16:0/2015254070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324102570 0x7f73241029b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.687+0000 7f732ac82640 1 --2- 172.21.15.16:0/2015254070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7324100e20 0x7f7324101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.687+0000 7f732ac82640 1 -- 172.21.15.16:0/2015254070 >> 172.21.15.16:0/2015254070 conn(0x7f73240fc7d0 msgr2=0x7f73240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.687+0000 7f732ac82640 1 -- 172.21.15.16:0/2015254070 shutdown_connections 2024-09-19T09:26:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.687+0000 7f732ac82640 1 -- 172.21.15.16:0/2015254070 wait complete. 2024-09-19T09:26:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f732ac82640 1 Processor -- start 2024-09-19T09:26:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f732ac82640 1 -- start start 2024-09-19T09:26:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f732ac82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324100e20 0x7f73241985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f732ac82640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7324102570 0x7f7324198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f732ac82640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7324102ef0 0x7f732419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f732ac82640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7324110f80 con 0x7f7324100e20 2024-09-19T09:26:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f732ac82640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7324110e00 con 0x7f7324102ef0 2024-09-19T09:26:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f732ac82640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7324111100 con 0x7f7324102570 2024-09-19T09:26:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f7328a1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324100e20 0x7f73241985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f7328a1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324100e20 0x7f73241985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39224/0 (socket says 172.21.15.16:39224) 2024-09-19T09:26:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f7328a1e640 1 -- 172.21.15.16:0/3340504238 learned_addr learned my addr 172.21.15.16:0/3340504238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f7323fff640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7324102570 0x7f7324198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f732921f640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7324102ef0 0x7f732419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f7328a1e640 1 -- 172.21.15.16:0/3340504238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7324102570 msgr2=0x7f7324198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f7328a1e640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7324102570 0x7f7324198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f7328a1e640 1 -- 172.21.15.16:0/3340504238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7324102ef0 msgr2=0x7f732419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f7328a1e640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7324102ef0 0x7f732419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f7328a1e640 1 -- 172.21.15.16:0/3340504238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732419d570 con 0x7f7324100e20 2024-09-19T09:26:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.688+0000 7f732921f640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7324102ef0 0x7f732419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.689+0000 7f7328a1e640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324100e20 0x7f73241985c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f731800d770 tx=0x7f731800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.689+0000 7f7321ffb640 1 -- 172.21.15.16:0/3340504238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318017070 con 0x7f7324100e20 2024-09-19T09:26:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.689+0000 7f732ac82640 1 -- 172.21.15.16:0/3340504238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732419d860 con 0x7f7324100e20 2024-09-19T09:26:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.689+0000 7f7321ffb640 1 -- 172.21.15.16:0/3340504238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7318005500 con 0x7f7324100e20 2024-09-19T09:26:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.689+0000 7f7321ffb640 1 -- 172.21.15.16:0/3340504238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318004e00 con 0x7f7324100e20 2024-09-19T09:26:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.689+0000 7f732ac82640 1 -- 172.21.15.16:0/3340504238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73240fcbb0 con 0x7f7324100e20 2024-09-19T09:26:07.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.690+0000 7f732ac82640 1 -- 172.21.15.16:0/3340504238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7324062980 con 0x7f7324100e20 2024-09-19T09:26:07.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.691+0000 7f7321ffb640 1 -- 172.21.15.16:0/3340504238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7318004fa0 con 0x7f7324100e20 2024-09-19T09:26:07.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.692+0000 7f7321ffb640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f73140a32d0 0x7f73140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.692+0000 7f7323fff640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f73140a32d0 0x7f73140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.692+0000 7f7321ffb640 1 -- 172.21.15.16:0/3340504238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f73180cf950 con 0x7f7324100e20 2024-09-19T09:26:07.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.692+0000 7f7321ffb640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951] conn(0x7f73140acf00 0x7f73140af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.692+0000 7f7321ffb640 1 -- 172.21.15.16:0/3340504238 --> [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7318001d30 con 0x7f73140acf00 2024-09-19T09:26:07.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.692+0000 7f7321ffb640 1 -- 172.21.15.16:0/3340504238 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f73180cfd00 con 0x7f7324100e20 2024-09-19T09:26:07.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.693+0000 7f7323fff640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f73140a32d0 0x7f73140a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7310009230 tx=0x7f73100058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:07.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.693+0000 7f732921f640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951] conn(0x7f73140acf00 0x7f73140af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.693+0000 7f732921f640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951] conn(0x7f73140acf00 0x7f73140af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:07.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.694+0000 7f7321ffb640 1 -- 172.21.15.16:0/3340504238 <== osd.2 v2:172.21.15.142:6800/2692613951 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7318001d30 con 0x7f73140acf00 2024-09-19T09:26:07.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.707+0000 7f732ac82640 1 -- 172.21.15.16:0/3340504238 --> [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7324062e00 con 0x7f73140acf00 2024-09-19T09:26:07.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.708+0000 7f7321ffb640 1 -- 172.21.15.16:0/3340504238 <== osd.2 v2:172.21.15.142:6800/2692613951 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7324062e00 con 0x7f73140acf00 2024-09-19T09:26:07.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.708+0000 7f732ac82640 1 -- 172.21.15.16:0/3340504238 >> [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951] conn(0x7f73140acf00 msgr2=0x7f73140af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.708+0000 7f732ac82640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951] conn(0x7f73140acf00 0x7f73140af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.708+0000 7f732ac82640 1 -- 172.21.15.16:0/3340504238 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f73140a32d0 msgr2=0x7f73140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.708+0000 7f732ac82640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f73140a32d0 0x7f73140a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7310009230 tx=0x7f73100058d0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.708+0000 7f732ac82640 1 -- 172.21.15.16:0/3340504238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324100e20 msgr2=0x7f73241985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.708+0000 7f732ac82640 1 --2- 172.21.15.16:0/3340504238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324100e20 0x7f73241985c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f731800d770 tx=0x7f731800dc30 comp rx=0 tx=0).stop 2024-09-19T09:26:07.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.708+0000 7f732921f640 1 -- 172.21.15.16:0/3340504238 reap_dead start 2024-09-19T09:26:07.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.708+0000 7f732ac82640 1 -- 172.21.15.16:0/3340504238 shutdown_connections 2024-09-19T09:26:07.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.709+0000 7f732ac82640 1 -- 172.21.15.16:0/3340504238 >> 172.21.15.16:0/3340504238 conn(0x7f73240fc7d0 msgr2=0x7f732406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:07.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.709+0000 7f732ac82640 1 -- 172.21.15.16:0/3340504238 shutdown_connections 2024-09-19T09:26:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:07.709+0000 7f732ac82640 1 -- 172.21.15.16:0/3340504238 wait complete. 2024-09-19T09:26:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.010+0000 7fdef6259640 1 -- 172.21.15.16:0/675271060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef0102570 msgr2=0x7fdef01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.010+0000 7fdef6259640 1 --2- 172.21.15.16:0/675271060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef0102570 0x7fdef01029b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009900 tx=0x7fdee402a480 comp rx=0 tx=0).stop 2024-09-19T09:26:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.010+0000 7fdef6259640 1 -- 172.21.15.16:0/675271060 shutdown_connections 2024-09-19T09:26:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.010+0000 7fdef6259640 1 --2- 172.21.15.16:0/675271060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef0102ef0 0x7fdef010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.010+0000 7fdef6259640 1 --2- 172.21.15.16:0/675271060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef0102570 0x7fdef01029b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.010+0000 7fdef6259640 1 --2- 172.21.15.16:0/675271060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef0100e20 0x7fdef0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.010+0000 7fdef6259640 1 -- 172.21.15.16:0/675271060 >> 172.21.15.16:0/675271060 conn(0x7fdef00fc7d0 msgr2=0x7fdef00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.010+0000 7fdef6259640 1 -- 172.21.15.16:0/675271060 shutdown_connections 2024-09-19T09:26:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.010+0000 7fdef6259640 1 -- 172.21.15.16:0/675271060 wait complete. 2024-09-19T09:26:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.011+0000 7fdef6259640 1 Processor -- start 2024-09-19T09:26:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.011+0000 7fdef6259640 1 -- start start 2024-09-19T09:26:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.011+0000 7fdef6259640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef0100e20 0x7fdef0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.011+0000 7fdef6259640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef0102570 0x7fdef0197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.011+0000 7fdef6259640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef0102ef0 0x7fdef019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.011+0000 7fdef6259640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdef0110f50 con 0x7fdef0102ef0 2024-09-19T09:26:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.011+0000 7fdef6259640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdef0110dd0 con 0x7fdef0102570 2024-09-19T09:26:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.011+0000 7fdef6259640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdef01110d0 con 0x7fdef0100e20 2024-09-19T09:26:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.011+0000 7fdeeffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef0102ef0 0x7fdef019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.011+0000 7fdeeffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef0102ef0 0x7fdef019bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39160/0 (socket says 172.21.15.16:39160) 2024-09-19T09:26:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdeeffff640 1 -- 172.21.15.16:0/176641287 learned_addr learned my addr 172.21.15.16:0/176641287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdeeeffd640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef0102570 0x7fdef0197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdeef7fe640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef0100e20 0x7fdef0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdeeffff640 1 -- 172.21.15.16:0/176641287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef0100e20 msgr2=0x7fdef0197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdeeffff640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef0100e20 0x7fdef0197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdeeffff640 1 -- 172.21.15.16:0/176641287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef0102570 msgr2=0x7fdef0197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdeeffff640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef0102570 0x7fdef0197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdeeffff640 1 -- 172.21.15.16:0/176641287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef019c300 con 0x7fdef0102ef0 2024-09-19T09:26:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdeeffff640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef0102ef0 0x7fdef019bc20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdee800cad0 tx=0x7fdee800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdeecff9640 1 -- 172.21.15.16:0/176641287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee800ae20 con 0x7fdef0102ef0 2024-09-19T09:26:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdeecff9640 1 -- 172.21.15.16:0/176641287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee8005020 con 0x7fdef0102ef0 2024-09-19T09:26:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdef6259640 1 -- 172.21.15.16:0/176641287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef019c5f0 con 0x7fdef0102ef0 2024-09-19T09:26:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdeecff9640 1 -- 172.21.15.16:0/176641287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee800f6f0 con 0x7fdef0102ef0 2024-09-19T09:26:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.012+0000 7fdef6259640 1 -- 172.21.15.16:0/176641287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef01a3ed0 con 0x7fdef0102ef0 2024-09-19T09:26:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.013+0000 7fdef6259640 1 -- 172.21.15.16:0/176641287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdef01072c0 con 0x7fdef0102ef0 2024-09-19T09:26:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.014+0000 7fdeecff9640 1 -- 172.21.15.16:0/176641287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdee80051d0 con 0x7fdef0102ef0 2024-09-19T09:26:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.015+0000 7fdeecff9640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fded80a3350 0x7fded80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.015+0000 7fdeef7fe640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fded80a3350 0x7fded80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.015+0000 7fdeecff9640 1 -- 172.21.15.16:0/176641287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fdee80cffa0 con 0x7fdef0102ef0 2024-09-19T09:26:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.015+0000 7fdeecff9640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360] conn(0x7fded80acf80 0x7fded80af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.015+0000 7fdeecff9640 1 -- 172.21.15.16:0/176641287 --> [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fded80afaf0 con 0x7fded80acf80 2024-09-19T09:26:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.015+0000 7fdeecff9640 1 -- 172.21.15.16:0/176641287 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fdee80d03d0 con 0x7fdef0102ef0 2024-09-19T09:26:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.016+0000 7fdeeeffd640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360] conn(0x7fded80acf80 0x7fded80af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.016+0000 7fdeef7fe640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fded80a3350 0x7fded80a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdee0005e10 tx=0x7fdee0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.016+0000 7fdeeeffd640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360] conn(0x7fded80acf80 0x7fded80af420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.017+0000 7fdeecff9640 1 -- 172.21.15.16:0/176641287 <== osd.9 v2:172.21.15.16:6826/2651132360 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fded80afaf0 con 0x7fded80acf80 2024-09-19T09:26:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.030+0000 7fdef6259640 1 -- 172.21.15.16:0/176641287 --> [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdef01021b0 con 0x7fded80acf80 2024-09-19T09:26:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.031+0000 7fdeecff9640 1 -- 172.21.15.16:0/176641287 <== osd.9 v2:172.21.15.16:6826/2651132360 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdef01021b0 con 0x7fded80acf80 2024-09-19T09:26:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.031+0000 7fdef6259640 1 -- 172.21.15.16:0/176641287 >> [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360] conn(0x7fded80acf80 msgr2=0x7fded80af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.031+0000 7fdef6259640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360] conn(0x7fded80acf80 0x7fded80af420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.031+0000 7fdef6259640 1 -- 172.21.15.16:0/176641287 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fded80a3350 msgr2=0x7fded80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.031+0000 7fdef6259640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fded80a3350 0x7fded80a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdee0005e10 tx=0x7fdee0005da0 comp rx=0 tx=0).stop 2024-09-19T09:26:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.031+0000 7fdef6259640 1 -- 172.21.15.16:0/176641287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef0102ef0 msgr2=0x7fdef019bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.031+0000 7fdef6259640 1 --2- 172.21.15.16:0/176641287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef0102ef0 0x7fdef019bc20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdee800cad0 tx=0x7fdee800cf90 comp rx=0 tx=0).stop 2024-09-19T09:26:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.031+0000 7fdeeffff640 1 -- 172.21.15.16:0/176641287 reap_dead start 2024-09-19T09:26:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.031+0000 7fdef6259640 1 -- 172.21.15.16:0/176641287 shutdown_connections 2024-09-19T09:26:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.031+0000 7fdef6259640 1 -- 172.21.15.16:0/176641287 >> 172.21.15.16:0/176641287 conn(0x7fdef00fc7d0 msgr2=0x7fdef00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.031+0000 7fdef6259640 1 -- 172.21.15.16:0/176641287 shutdown_connections 2024-09-19T09:26:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:06.031+0000 7fdef6259640 1 -- 172.21.15.16:0/176641287 wait complete. 2024-09-19T09:26:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.472+0000 7f48f1e6a640 1 -- 172.21.15.16:0/3661850026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec106f10 msgr2=0x7f48ec10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.472+0000 7f48f1e6a640 1 --2- 172.21.15.16:0/3661850026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec106f10 0x7f48ec10ea90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009ca0 tx=0x7f48e402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:26:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.472+0000 7f48f1e6a640 1 -- 172.21.15.16:0/3661850026 shutdown_connections 2024-09-19T09:26:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.472+0000 7f48f1e6a640 1 --2- 172.21.15.16:0/3661850026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec106f10 0x7f48ec10ea90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.472+0000 7f48f1e6a640 1 --2- 172.21.15.16:0/3661850026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48ec102790 0x7f48ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.472+0000 7f48f1e6a640 1 --2- 172.21.15.16:0/3661850026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ec101de0 0x7f48ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.472+0000 7f48f1e6a640 1 -- 172.21.15.16:0/3661850026 >> 172.21.15.16:0/3661850026 conn(0x7f48ec0fd8c0 msgr2=0x7f48ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.472+0000 7f48f1e6a640 1 -- 172.21.15.16:0/3661850026 shutdown_connections 2024-09-19T09:26:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.472+0000 7f48f1e6a640 1 -- 172.21.15.16:0/3661850026 wait complete. 2024-09-19T09:26:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.473+0000 7f48f1e6a640 1 Processor -- start 2024-09-19T09:26:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.473+0000 7f48f1e6a640 1 -- start start 2024-09-19T09:26:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.473+0000 7f48f1e6a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48ec101de0 0x7f48ec198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.473+0000 7f48f1e6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec102790 0x7f48ec198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.473+0000 7f48f1e6a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ec106f10 0x7f48ec19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.473+0000 7f48f1e6a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec111f90 con 0x7f48ec102790 2024-09-19T09:26:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.473+0000 7f48f1e6a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec111e10 con 0x7f48ec101de0 2024-09-19T09:26:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.473+0000 7f48f1e6a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec112110 con 0x7f48ec106f10 2024-09-19T09:26:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.473+0000 7f48eaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec102790 0x7f48ec198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:08.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.473+0000 7f48eaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec102790 0x7f48ec198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39246/0 (socket says 172.21.15.16:39246) 2024-09-19T09:26:08.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.473+0000 7f48eaffd640 1 -- 172.21.15.16:0/1667506421 learned_addr learned my addr 172.21.15.16:0/1667506421 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:08.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.474+0000 7f48ebfff640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ec106f10 0x7f48ec19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:08.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.474+0000 7f48eb7fe640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48ec101de0 0x7f48ec198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:08.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.474+0000 7f48eaffd640 1 -- 172.21.15.16:0/1667506421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ec106f10 msgr2=0x7f48ec19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:08.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.474+0000 7f48eaffd640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48ec106f10 0x7f48ec19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:08.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.474+0000 7f48eaffd640 1 -- 172.21.15.16:0/1667506421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48ec101de0 msgr2=0x7f48ec198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:08.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.474+0000 7f48eaffd640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48ec101de0 0x7f48ec198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:08.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.474+0000 7f48eaffd640 1 -- 172.21.15.16:0/1667506421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec19d3c0 con 0x7f48ec102790 2024-09-19T09:26:08.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.474+0000 7f48eb7fe640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48ec101de0 0x7f48ec198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:08.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.474+0000 7f48eaffd640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec102790 0x7f48ec198950 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f48e000c920 tx=0x7f48e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.474+0000 7f48e8ff9640 1 -- 172.21.15.16:0/1667506421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0007c70 con 0x7f48ec102790 2024-09-19T09:26:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.474+0000 7f48e8ff9640 1 -- 172.21.15.16:0/1667506421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e0007e10 con 0x7f48ec102790 2024-09-19T09:26:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.474+0000 7f48f1e6a640 1 -- 172.21.15.16:0/1667506421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ec19d6b0 con 0x7f48ec102790 2024-09-19T09:26:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.474+0000 7f48e8ff9640 1 -- 172.21.15.16:0/1667506421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0018890 con 0x7f48ec102790 2024-09-19T09:26:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.475+0000 7f48f1e6a640 1 -- 172.21.15.16:0/1667506421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ec1a4f90 con 0x7f48ec102790 2024-09-19T09:26:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.476+0000 7f48f1e6a640 1 -- 172.21.15.16:0/1667506421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f48ec0fee30 con 0x7f48ec102790 2024-09-19T09:26:08.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.477+0000 7f48e8ff9640 1 -- 172.21.15.16:0/1667506421 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f48e0018a30 con 0x7f48ec102790 2024-09-19T09:26:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.478+0000 7f48e8ff9640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f48d40a32d0 0x7f48d40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.478+0000 7f48eb7fe640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f48d40a32d0 0x7f48d40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.478+0000 7f48e8ff9640 1 -- 172.21.15.16:0/1667506421 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f48e00d0070 con 0x7f48ec102790 2024-09-19T09:26:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.478+0000 7f48e8ff9640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536] conn(0x7f48d40acf00 0x7f48d40af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.478+0000 7f48e8ff9640 1 -- 172.21.15.16:0/1667506421 --> [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48d40afa70 con 0x7f48d40acf00 2024-09-19T09:26:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.478+0000 7f48e8ff9640 1 -- 172.21.15.16:0/1667506421 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f48e00d0420 con 0x7f48ec102790 2024-09-19T09:26:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.478+0000 7f48ebfff640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536] conn(0x7f48d40acf00 0x7f48d40af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.478+0000 7f48eb7fe640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f48d40a32d0 0x7f48d40a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009930 tx=0x7f48dc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.479+0000 7f48ebfff640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536] conn(0x7f48d40acf00 0x7f48d40af3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.479+0000 7f48e8ff9640 1 -- 172.21.15.16:0/1667506421 <== osd.6 v2:172.21.15.16:6818/4053539536 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48d40afa70 con 0x7f48d40acf00 2024-09-19T09:26:08.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.493+0000 7f48f1e6a640 1 -- 172.21.15.16:0/1667506421 --> [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f48ec1032c0 con 0x7f48d40acf00 2024-09-19T09:26:08.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.493+0000 7f48e8ff9640 1 -- 172.21.15.16:0/1667506421 <== osd.6 v2:172.21.15.16:6818/4053539536 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f48ec1032c0 con 0x7f48d40acf00 2024-09-19T09:26:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.494+0000 7f48f1e6a640 1 -- 172.21.15.16:0/1667506421 >> [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536] conn(0x7f48d40acf00 msgr2=0x7f48d40af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.494+0000 7f48f1e6a640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536] conn(0x7f48d40acf00 0x7f48d40af3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.494+0000 7f48f1e6a640 1 -- 172.21.15.16:0/1667506421 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f48d40a32d0 msgr2=0x7f48d40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.494+0000 7f48f1e6a640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f48d40a32d0 0x7f48d40a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009930 tx=0x7f48dc008040 comp rx=0 tx=0).stop 2024-09-19T09:26:08.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.494+0000 7f48f1e6a640 1 -- 172.21.15.16:0/1667506421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec102790 msgr2=0x7f48ec198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.494+0000 7f48f1e6a640 1 --2- 172.21.15.16:0/1667506421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec102790 0x7f48ec198950 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f48e000c920 tx=0x7f48e000cde0 comp rx=0 tx=0).stop 2024-09-19T09:26:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.494+0000 7f48ebfff640 1 -- 172.21.15.16:0/1667506421 reap_dead start 2024-09-19T09:26:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.494+0000 7f48f1e6a640 1 -- 172.21.15.16:0/1667506421 shutdown_connections 2024-09-19T09:26:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.494+0000 7f48f1e6a640 1 -- 172.21.15.16:0/1667506421 >> 172.21.15.16:0/1667506421 conn(0x7f48ec0fd8c0 msgr2=0x7f48ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.494+0000 7f48f1e6a640 1 -- 172.21.15.16:0/1667506421 shutdown_connections 2024-09-19T09:26:08.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:08.494+0000 7f48f1e6a640 1 -- 172.21.15.16:0/1667506421 wait complete. 2024-09-19T09:26:09.186 INFO:teuthology.orchestra.run.smithi016.stdout:34359738443 2024-09-19T09:26:09.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T09:26:09.315 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:09 smithi016 ceph-mon[30979]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-19T09:26:09.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:09 smithi142 ceph-mon[35032]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-19T09:26:09.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:09 smithi097 ceph-mon[34864]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-19T09:26:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.702+0000 7fa5e429f640 1 -- 172.21.15.16:0/3538136001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc1082a0 msgr2=0x7fa5dc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.702+0000 7fa5e429f640 1 --2- 172.21.15.16:0/3538136001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc1082a0 0x7fa5dc10ec40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009cd0 tx=0x7fa5cc02a910 comp rx=0 tx=0).stop 2024-09-19T09:26:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 -- 172.21.15.16:0/3538136001 shutdown_connections 2024-09-19T09:26:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 --2- 172.21.15.16:0/3538136001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc1082a0 0x7fa5dc10ec40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 --2- 172.21.15.16:0/3538136001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5dc1036e0 0x7fa5dc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 --2- 172.21.15.16:0/3538136001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5dc102d30 0x7fa5dc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 -- 172.21.15.16:0/3538136001 >> 172.21.15.16:0/3538136001 conn(0x7fa5dc0fc7d0 msgr2=0x7fa5dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 -- 172.21.15.16:0/3538136001 shutdown_connections 2024-09-19T09:26:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 -- 172.21.15.16:0/3538136001 wait complete. 2024-09-19T09:26:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 Processor -- start 2024-09-19T09:26:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 -- start start 2024-09-19T09:26:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc102d30 0x7fa5dc1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5dc1036e0 0x7fa5dc1a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5dc1082a0 0x7fa5dc19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc112100 con 0x7fa5dc102d30 2024-09-19T09:26:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc111f80 con 0x7fa5dc1082a0 2024-09-19T09:26:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e429f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc112280 con 0x7fa5dc1036e0 2024-09-19T09:26:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.703+0000 7fa5e203b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc102d30 0x7fa5dc1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5e203b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc102d30 0x7fa5dc1a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39256/0 (socket says 172.21.15.16:39256) 2024-09-19T09:26:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5e203b640 1 -- 172.21.15.16:0/3109777827 learned_addr learned my addr 172.21.15.16:0/3109777827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5e183a640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5dc1036e0 0x7fa5dc1a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5e283c640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5dc1082a0 0x7fa5dc19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5e203b640 1 -- 172.21.15.16:0/3109777827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5dc1036e0 msgr2=0x7fa5dc1a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5e203b640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5dc1036e0 0x7fa5dc1a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5e203b640 1 -- 172.21.15.16:0/3109777827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5dc1082a0 msgr2=0x7fa5dc19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5e203b640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5dc1082a0 0x7fa5dc19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5e203b640 1 -- 172.21.15.16:0/3109777827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc19ad00 con 0x7fa5dc102d30 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5e203b640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc102d30 0x7fa5dc1a0410 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400c970 tx=0x7fa5d400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5db7fe640 1 -- 172.21.15.16:0/3109777827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4007bb0 con 0x7fa5dc102d30 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5e429f640 1 -- 172.21.15.16:0/3109777827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5dc19aff0 con 0x7fa5dc102d30 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5db7fe640 1 -- 172.21.15.16:0/3109777827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5d4007d50 con 0x7fa5dc102d30 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5db7fe640 1 -- 172.21.15.16:0/3109777827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d400f450 con 0x7fa5dc102d30 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.704+0000 7fa5e429f640 1 -- 172.21.15.16:0/3109777827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5dc1a72e0 con 0x7fa5dc102d30 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.705+0000 7fa5e429f640 1 -- 172.21.15.16:0/3109777827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5dc062980 con 0x7fa5dc102d30 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.706+0000 7fa5db7fe640 1 -- 172.21.15.16:0/3109777827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa5d4004ee0 con 0x7fa5dc102d30 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.707+0000 7fa5db7fe640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa5c40a32d0 0x7fa5c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.707+0000 7fa5db7fe640 1 -- 172.21.15.16:0/3109777827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa5d40cfbf0 con 0x7fa5dc102d30 2024-09-19T09:26:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.707+0000 7fa5e183a640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa5c40a32d0 0x7fa5c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:09.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.707+0000 7fa5db7fe640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004] conn(0x7fa5c40acf00 0x7fa5c40af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:09.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.707+0000 7fa5db7fe640 1 -- 172.21.15.16:0/3109777827 --> [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5d4012ec0 con 0x7fa5c40acf00 2024-09-19T09:26:09.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.707+0000 7fa5db7fe640 1 -- 172.21.15.16:0/3109777827 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fa5d40d0020 con 0x7fa5dc102d30 2024-09-19T09:26:09.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.707+0000 7fa5e283c640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004] conn(0x7fa5c40acf00 0x7fa5c40af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:09.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.708+0000 7fa5e183a640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa5c40a32d0 0x7fa5c40a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0fde60 tx=0x7fa5d0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:09.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.708+0000 7fa5e283c640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004] conn(0x7fa5c40acf00 0x7fa5c40af3a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:09.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.709+0000 7fa5db7fe640 1 -- 172.21.15.16:0/3109777827 <== osd.8 v2:172.21.15.142:6816/643795004 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa5d4012ec0 con 0x7fa5c40acf00 2024-09-19T09:26:09.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.722+0000 7fa5e429f640 1 -- 172.21.15.16:0/3109777827 --> [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5dc062e00 con 0x7fa5c40acf00 2024-09-19T09:26:09.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.723+0000 7fa5db7fe640 1 -- 172.21.15.16:0/3109777827 <== osd.8 v2:172.21.15.142:6816/643795004 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa5dc062e00 con 0x7fa5c40acf00 2024-09-19T09:26:09.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.723+0000 7fa5e429f640 1 -- 172.21.15.16:0/3109777827 >> [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004] conn(0x7fa5c40acf00 msgr2=0x7fa5c40af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:09.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.723+0000 7fa5e429f640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004] conn(0x7fa5c40acf00 0x7fa5c40af3a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:09.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.723+0000 7fa5e429f640 1 -- 172.21.15.16:0/3109777827 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa5c40a32d0 msgr2=0x7fa5c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:09.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.723+0000 7fa5e429f640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa5c40a32d0 0x7fa5c40a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0fde60 tx=0x7fa5d0009110 comp rx=0 tx=0).stop 2024-09-19T09:26:09.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.723+0000 7fa5e429f640 1 -- 172.21.15.16:0/3109777827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc102d30 msgr2=0x7fa5dc1a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:09.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.723+0000 7fa5e429f640 1 --2- 172.21.15.16:0/3109777827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5dc102d30 0x7fa5dc1a0410 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400c970 tx=0x7fa5d400cd70 comp rx=0 tx=0).stop 2024-09-19T09:26:09.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.723+0000 7fa5e283c640 1 -- 172.21.15.16:0/3109777827 reap_dead start 2024-09-19T09:26:09.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.723+0000 7fa5e429f640 1 -- 172.21.15.16:0/3109777827 shutdown_connections 2024-09-19T09:26:09.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.723+0000 7fa5e429f640 1 -- 172.21.15.16:0/3109777827 >> 172.21.15.16:0/3109777827 conn(0x7fa5dc0fc7d0 msgr2=0x7fa5dc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:09.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.724+0000 7fa5e429f640 1 -- 172.21.15.16:0/3109777827 shutdown_connections 2024-09-19T09:26:09.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:09.724+0000 7fa5e429f640 1 -- 172.21.15.16:0/3109777827 wait complete. 2024-09-19T09:26:10.754 INFO:teuthology.orchestra.run.smithi016.stdout:283467841561 2024-09-19T09:26:10.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T09:26:11.304 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:11 smithi016 ceph-mon[30979]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T09:26:11.333 INFO:teuthology.orchestra.run.smithi016.stdout:261993005085 2024-09-19T09:26:11.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T09:26:11.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:11 smithi142 ceph-mon[35032]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T09:26:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:11 smithi097 ceph-mon[34864]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-19T09:26:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.874+0000 7fdae4f70640 1 -- 172.21.15.16:0/2327326866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdae0102570 msgr2=0x7fdae01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.874+0000 7fdae4f70640 1 --2- 172.21.15.16:0/2327326866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdae0102570 0x7fdae01029b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009900 tx=0x7fdac802a480 comp rx=0 tx=0).stop 2024-09-19T09:26:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.874+0000 7fdae4f70640 1 -- 172.21.15.16:0/2327326866 shutdown_connections 2024-09-19T09:26:11.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.874+0000 7fdae4f70640 1 --2- 172.21.15.16:0/2327326866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdae0102ef0 0x7fdae010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:11.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.874+0000 7fdae4f70640 1 --2- 172.21.15.16:0/2327326866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdae0102570 0x7fdae01029b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:11.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.874+0000 7fdae4f70640 1 --2- 172.21.15.16:0/2327326866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae0100e20 0x7fdae0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:11.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.875+0000 7fdae4f70640 1 -- 172.21.15.16:0/2327326866 >> 172.21.15.16:0/2327326866 conn(0x7fdae00fc7d0 msgr2=0x7fdae00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:11.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.875+0000 7fdae4f70640 1 -- 172.21.15.16:0/2327326866 shutdown_connections 2024-09-19T09:26:11.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.875+0000 7fdae4f70640 1 -- 172.21.15.16:0/2327326866 wait complete. 2024-09-19T09:26:11.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.875+0000 7fdae4f70640 1 Processor -- start 2024-09-19T09:26:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.875+0000 7fdae4f70640 1 -- start start 2024-09-19T09:26:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.875+0000 7fdae4f70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdae0100e20 0x7fdae01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdae4f70640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdae0102570 0x7fdae0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdae4f70640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae0102ef0 0x7fdae01a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdae4f70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0110f80 con 0x7fdae0100e20 2024-09-19T09:26:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdae4f70640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0110e00 con 0x7fdae0102ef0 2024-09-19T09:26:11.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdae4f70640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0111100 con 0x7fdae0102570 2024-09-19T09:26:11.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdade59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdae0100e20 0x7fdae01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:11.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdade59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdae0100e20 0x7fdae01992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39274/0 (socket says 172.21.15.16:39274) 2024-09-19T09:26:11.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdade59c640 1 -- 172.21.15.16:0/4135787819 learned_addr learned my addr 172.21.15.16:0/4135787819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:11.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdad7fff640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdae0102570 0x7fdae0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:11.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdaded9d640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae0102ef0 0x7fdae01a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:11.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdade59c640 1 -- 172.21.15.16:0/4135787819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdae0102570 msgr2=0x7fdae0199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:11.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdade59c640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdae0102570 0x7fdae0199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:11.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdade59c640 1 -- 172.21.15.16:0/4135787819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae0102ef0 msgr2=0x7fdae01a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:11.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdade59c640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae0102ef0 0x7fdae01a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:11.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdade59c640 1 -- 172.21.15.16:0/4135787819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae01a48b0 con 0x7fdae0100e20 2024-09-19T09:26:11.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdaded9d640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdae0102ef0 0x7fdae01a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:11.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdade59c640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdae0100e20 0x7fdae01992f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdad000c920 tx=0x7fdad000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:11.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdad77fe640 1 -- 172.21.15.16:0/4135787819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0007c70 con 0x7fdae0100e20 2024-09-19T09:26:11.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdad77fe640 1 -- 172.21.15.16:0/4135787819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad0007e10 con 0x7fdae0100e20 2024-09-19T09:26:11.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdae4f70640 1 -- 172.21.15.16:0/4135787819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae01a4a80 con 0x7fdae0100e20 2024-09-19T09:26:11.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.876+0000 7fdad77fe640 1 -- 172.21.15.16:0/4135787819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0018890 con 0x7fdae0100e20 2024-09-19T09:26:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.877+0000 7fdae4f70640 1 -- 172.21.15.16:0/4135787819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae01a4f90 con 0x7fdae0100e20 2024-09-19T09:26:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.877+0000 7fdae4f70640 1 -- 172.21.15.16:0/4135787819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdae0107250 con 0x7fdae0100e20 2024-09-19T09:26:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.879+0000 7fdad77fe640 1 -- 172.21.15.16:0/4135787819 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdad0018a30 con 0x7fdae0100e20 2024-09-19T09:26:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.879+0000 7fdad77fe640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdacc0a32d0 0x7fdacc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.879+0000 7fdad77fe640 1 -- 172.21.15.16:0/4135787819 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fdad00d00b0 con 0x7fdae0100e20 2024-09-19T09:26:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.879+0000 7fdad7fff640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdacc0a32d0 0x7fdacc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.880+0000 7fdad77fe640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825] conn(0x7fdacc0acf00 0x7fdacc0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.880+0000 7fdad77fe640 1 -- 172.21.15.16:0/4135787819 --> [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdacc0afa70 con 0x7fdacc0acf00 2024-09-19T09:26:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.880+0000 7fdad77fe640 1 -- 172.21.15.16:0/4135787819 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fdad00d04e0 con 0x7fdae0100e20 2024-09-19T09:26:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.880+0000 7fdad7fff640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdacc0a32d0 0x7fdacc0a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdae006aac0 tx=0x7fdac8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.880+0000 7fdaded9d640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825] conn(0x7fdacc0acf00 0x7fdacc0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.881+0000 7fdaded9d640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825] conn(0x7fdacc0acf00 0x7fdacc0af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.881+0000 7fdad77fe640 1 -- 172.21.15.16:0/4135787819 <== osd.4 v2:172.21.15.97:6808/3491353825 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdacc0afa70 con 0x7fdacc0acf00 2024-09-19T09:26:11.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.895+0000 7fdae4f70640 1 -- 172.21.15.16:0/4135787819 --> [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdae01021b0 con 0x7fdacc0acf00 2024-09-19T09:26:11.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.896+0000 7fdad77fe640 1 -- 172.21.15.16:0/4135787819 <== osd.4 v2:172.21.15.97:6808/3491353825 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdae01021b0 con 0x7fdacc0acf00 2024-09-19T09:26:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.896+0000 7fdae4f70640 1 -- 172.21.15.16:0/4135787819 >> [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825] conn(0x7fdacc0acf00 msgr2=0x7fdacc0af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.896+0000 7fdae4f70640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825] conn(0x7fdacc0acf00 0x7fdacc0af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.896+0000 7fdae4f70640 1 -- 172.21.15.16:0/4135787819 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdacc0a32d0 msgr2=0x7fdacc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.896+0000 7fdae4f70640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdacc0a32d0 0x7fdacc0a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdae006aac0 tx=0x7fdac8005a20 comp rx=0 tx=0).stop 2024-09-19T09:26:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.896+0000 7fdae4f70640 1 -- 172.21.15.16:0/4135787819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdae0100e20 msgr2=0x7fdae01992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.896+0000 7fdae4f70640 1 --2- 172.21.15.16:0/4135787819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdae0100e20 0x7fdae01992f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdad000c920 tx=0x7fdad000cde0 comp rx=0 tx=0).stop 2024-09-19T09:26:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.897+0000 7fdaded9d640 1 -- 172.21.15.16:0/4135787819 reap_dead start 2024-09-19T09:26:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.897+0000 7fdae4f70640 1 -- 172.21.15.16:0/4135787819 shutdown_connections 2024-09-19T09:26:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.897+0000 7fdae4f70640 1 -- 172.21.15.16:0/4135787819 >> 172.21.15.16:0/4135787819 conn(0x7fdae00fc7d0 msgr2=0x7fdae0106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.897+0000 7fdae4f70640 1 -- 172.21.15.16:0/4135787819 shutdown_connections 2024-09-19T09:26:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:11.897+0000 7fdae4f70640 1 -- 172.21.15.16:0/4135787819 wait complete. 2024-09-19T09:26:12.301 INFO:teuthology.orchestra.run.smithi016.stdout:231928234019 2024-09-19T09:26:12.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T09:26:12.476 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:12.872 INFO:teuthology.orchestra.run.smithi016.stdout:55834574918 2024-09-19T09:26:12.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T09:26:13.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:13 smithi016 ceph-mon[30979]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T09:26:13.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:13 smithi142 ceph-mon[35032]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T09:26:13.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:13 smithi097 ceph-mon[34864]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T09:26:13.650 INFO:teuthology.orchestra.run.smithi016.stdout:77309411394 2024-09-19T09:26:13.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T09:26:13.979 INFO:teuthology.orchestra.run.smithi016.stdout:154618822706 2024-09-19T09:26:13.979 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T09:26:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.171+0000 7f9b66dab640 1 -- 172.21.15.16:0/3649777614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b60106a80 msgr2=0x7f9b60106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.171+0000 7f9b66dab640 1 --2- 172.21.15.16:0/3649777614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b60106a80 0x7f9b60106e60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009930 tx=0x7f9b4802a420 comp rx=0 tx=0).stop 2024-09-19T09:26:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.171+0000 7f9b66dab640 1 -- 172.21.15.16:0/3649777614 shutdown_connections 2024-09-19T09:26:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.171+0000 7f9b66dab640 1 --2- 172.21.15.16:0/3649777614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b60101420 0x7f9b6010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.171+0000 7f9b66dab640 1 --2- 172.21.15.16:0/3649777614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b60100a80 0x7f9b60100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.171+0000 7f9b66dab640 1 --2- 172.21.15.16:0/3649777614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b60106a80 0x7f9b60106e60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.171+0000 7f9b66dab640 1 -- 172.21.15.16:0/3649777614 >> 172.21.15.16:0/3649777614 conn(0x7f9b600fc7a0 msgr2=0x7f9b600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.171+0000 7f9b66dab640 1 -- 172.21.15.16:0/3649777614 shutdown_connections 2024-09-19T09:26:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.171+0000 7f9b66dab640 1 -- 172.21.15.16:0/3649777614 wait complete. 2024-09-19T09:26:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.172+0000 7f9b66dab640 1 Processor -- start 2024-09-19T09:26:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.172+0000 7f9b66dab640 1 -- start start 2024-09-19T09:26:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.172+0000 7f9b66dab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b60100a80 0x7f9b601984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.172+0000 7f9b66dab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b60101420 0x7f9b60198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.172+0000 7f9b66dab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b60106a80 0x7f9b6019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.172+0000 7f9b66dab640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60111fb0 con 0x7f9b60100a80 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.172+0000 7f9b66dab640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60111e30 con 0x7f9b60101420 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.172+0000 7f9b66dab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60112130 con 0x7f9b60106a80 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.172+0000 7f9b64b47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b60100a80 0x7f9b601984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.172+0000 7f9b64b47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b60100a80 0x7f9b601984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39302/0 (socket says 172.21.15.16:39302) 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.172+0000 7f9b64b47640 1 -- 172.21.15.16:0/1502967228 learned_addr learned my addr 172.21.15.16:0/1502967228 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.173+0000 7f9b5ffff640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b60101420 0x7f9b60198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.173+0000 7f9b65348640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b60106a80 0x7f9b6019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.173+0000 7f9b64b47640 1 -- 172.21.15.16:0/1502967228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b60106a80 msgr2=0x7f9b6019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.173+0000 7f9b64b47640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b60106a80 0x7f9b6019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.173+0000 7f9b64b47640 1 -- 172.21.15.16:0/1502967228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b60101420 msgr2=0x7f9b60198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.173+0000 7f9b64b47640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b60101420 0x7f9b60198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.173+0000 7f9b64b47640 1 -- 172.21.15.16:0/1502967228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6019d470 con 0x7f9b60100a80 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.173+0000 7f9b65348640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b60106a80 0x7f9b6019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:14.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.173+0000 7f9b64b47640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b60100a80 0x7f9b601984c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009230 tx=0x7f9b48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:14.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.173+0000 7f9b5dffb640 1 -- 172.21.15.16:0/1502967228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4803f030 con 0x7f9b60100a80 2024-09-19T09:26:14.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.173+0000 7f9b5dffb640 1 -- 172.21.15.16:0/1502967228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b48003ad0 con 0x7f9b60100a80 2024-09-19T09:26:14.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.173+0000 7f9b66dab640 1 -- 172.21.15.16:0/1502967228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6019d700 con 0x7f9b60100a80 2024-09-19T09:26:14.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.173+0000 7f9b5dffb640 1 -- 172.21.15.16:0/1502967228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48035070 con 0x7f9b60100a80 2024-09-19T09:26:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.174+0000 7f9b66dab640 1 -- 172.21.15.16:0/1502967228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6010e190 con 0x7f9b60100a80 2024-09-19T09:26:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.175+0000 7f9b66dab640 1 -- 172.21.15.16:0/1502967228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b60062940 con 0x7f9b60100a80 2024-09-19T09:26:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.177+0000 7f9b5dffb640 1 -- 172.21.15.16:0/1502967228 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9b48042030 con 0x7f9b60100a80 2024-09-19T09:26:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.178+0000 7f9b5dffb640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9b4c0a32d0 0x7f9b4c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.178+0000 7f9b5dffb640 1 -- 172.21.15.16:0/1502967228 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9b480eff90 con 0x7f9b60100a80 2024-09-19T09:26:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.178+0000 7f9b5ffff640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9b4c0a32d0 0x7f9b4c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.178+0000 7f9b5dffb640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878] conn(0x7f9b4c0acf00 0x7f9b4c0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.178+0000 7f9b5dffb640 1 -- 172.21.15.16:0/1502967228 --> [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b4c0afa70 con 0x7f9b4c0acf00 2024-09-19T09:26:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.178+0000 7f9b5dffb640 1 -- 172.21.15.16:0/1502967228 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f9b480f25d0 con 0x7f9b60100a80 2024-09-19T09:26:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.178+0000 7f9b5ffff640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9b4c0a32d0 0x7f9b4c0a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9b60106160 tx=0x7f9b5000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.179+0000 7f9b65348640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878] conn(0x7f9b4c0acf00 0x7f9b4c0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.179+0000 7f9b65348640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878] conn(0x7f9b4c0acf00 0x7f9b4c0af3a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.180+0000 7f9b5dffb640 1 -- 172.21.15.16:0/1502967228 <== osd.11 v2:172.21.15.142:6824/155083878 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b4c0afa70 con 0x7f9b4c0acf00 2024-09-19T09:26:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.195+0000 7f9b66dab640 1 -- 172.21.15.16:0/1502967228 --> [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b60062df0 con 0x7f9b4c0acf00 2024-09-19T09:26:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.196+0000 7f9b5dffb640 1 -- 172.21.15.16:0/1502967228 <== osd.11 v2:172.21.15.142:6824/155083878 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9b60062df0 con 0x7f9b4c0acf00 2024-09-19T09:26:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.196+0000 7f9b66dab640 1 -- 172.21.15.16:0/1502967228 >> [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878] conn(0x7f9b4c0acf00 msgr2=0x7f9b4c0af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.196+0000 7f9b66dab640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878] conn(0x7f9b4c0acf00 0x7f9b4c0af3a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.197+0000 7f9b66dab640 1 -- 172.21.15.16:0/1502967228 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9b4c0a32d0 msgr2=0x7f9b4c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.197+0000 7f9b66dab640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9b4c0a32d0 0x7f9b4c0a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9b60106160 tx=0x7f9b5000b390 comp rx=0 tx=0).stop 2024-09-19T09:26:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.197+0000 7f9b66dab640 1 -- 172.21.15.16:0/1502967228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b60100a80 msgr2=0x7f9b601984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.197+0000 7f9b66dab640 1 --2- 172.21.15.16:0/1502967228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b60100a80 0x7f9b601984c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009230 tx=0x7f9b48004270 comp rx=0 tx=0).stop 2024-09-19T09:26:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.197+0000 7f9b65348640 1 -- 172.21.15.16:0/1502967228 reap_dead start 2024-09-19T09:26:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.197+0000 7f9b66dab640 1 -- 172.21.15.16:0/1502967228 shutdown_connections 2024-09-19T09:26:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.197+0000 7f9b66dab640 1 -- 172.21.15.16:0/1502967228 >> 172.21.15.16:0/1502967228 conn(0x7f9b600fc7a0 msgr2=0x7f9b600fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.197+0000 7f9b66dab640 1 -- 172.21.15.16:0/1502967228 shutdown_connections 2024-09-19T09:26:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.197+0000 7f9b66dab640 1 -- 172.21.15.16:0/1502967228 wait complete. 2024-09-19T09:26:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.743+0000 7fdc86e7f640 1 -- 172.21.15.16:0/2752805864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc80101420 msgr2=0x7fdc8010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.743+0000 7fdc86e7f640 1 --2- 172.21.15.16:0/2752805864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc80101420 0x7fdc8010d950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009cd0 tx=0x7fdc7802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:26:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.743+0000 7fdc86e7f640 1 -- 172.21.15.16:0/2752805864 shutdown_connections 2024-09-19T09:26:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.743+0000 7fdc86e7f640 1 --2- 172.21.15.16:0/2752805864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc80101420 0x7fdc8010d950 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.743+0000 7fdc86e7f640 1 --2- 172.21.15.16:0/2752805864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc80100a80 0x7fdc80100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.743+0000 7fdc86e7f640 1 --2- 172.21.15.16:0/2752805864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc80106a80 0x7fdc80106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.743+0000 7fdc86e7f640 1 -- 172.21.15.16:0/2752805864 >> 172.21.15.16:0/2752805864 conn(0x7fdc800fc7a0 msgr2=0x7fdc800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.743+0000 7fdc86e7f640 1 -- 172.21.15.16:0/2752805864 shutdown_connections 2024-09-19T09:26:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.743+0000 7fdc86e7f640 1 -- 172.21.15.16:0/2752805864 wait complete. 2024-09-19T09:26:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc86e7f640 1 Processor -- start 2024-09-19T09:26:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc86e7f640 1 -- start start 2024-09-19T09:26:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc86e7f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc80100a80 0x7fdc80198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc86e7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc80101420 0x7fdc801989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc86e7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc80106a80 0x7fdc8019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc86e7f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc80110e80 con 0x7fdc80106a80 2024-09-19T09:26:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc86e7f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdc80110d00 con 0x7fdc80101420 2024-09-19T09:26:14.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc86e7f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc80111000 con 0x7fdc80100a80 2024-09-19T09:26:14.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc8541c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc80106a80 0x7fdc8019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:14.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc8541c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc80106a80 0x7fdc8019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39330/0 (socket says 172.21.15.16:39330) 2024-09-19T09:26:14.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc8541c640 1 -- 172.21.15.16:0/3786559666 learned_addr learned my addr 172.21.15.16:0/3786559666 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:14.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc84c1b640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc80100a80 0x7fdc80198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:14.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc7ffff640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc80101420 0x7fdc801989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:14.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc8541c640 1 -- 172.21.15.16:0/3786559666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc80100a80 msgr2=0x7fdc80198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:14.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc8541c640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc80100a80 0x7fdc80198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:14.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc8541c640 1 -- 172.21.15.16:0/3786559666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc80101420 msgr2=0x7fdc801989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:14.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc8541c640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc80101420 0x7fdc801989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:14.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc8541c640 1 -- 172.21.15.16:0/3786559666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8019d420 con 0x7fdc80106a80 2024-09-19T09:26:14.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.744+0000 7fdc7ffff640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc80101420 0x7fdc801989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:14.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.745+0000 7fdc8541c640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc80106a80 0x7fdc8019cd40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdc78005cc0 tx=0x7fdc78032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:14.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.745+0000 7fdc7dffb640 1 -- 172.21.15.16:0/3786559666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78004040 con 0x7fdc80106a80 2024-09-19T09:26:14.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.745+0000 7fdc86e7f640 1 -- 172.21.15.16:0/3786559666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8019d6b0 con 0x7fdc80106a80 2024-09-19T09:26:14.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.745+0000 7fdc7dffb640 1 -- 172.21.15.16:0/3786559666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc780046c0 con 0x7fdc80106a80 2024-09-19T09:26:14.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.745+0000 7fdc7dffb640 1 -- 172.21.15.16:0/3786559666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78030e00 con 0x7fdc80106a80 2024-09-19T09:26:14.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.745+0000 7fdc86e7f640 1 -- 172.21.15.16:0/3786559666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc801a4ff0 con 0x7fdc80106a80 2024-09-19T09:26:14.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.746+0000 7fdc86e7f640 1 -- 172.21.15.16:0/3786559666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc80062900 con 0x7fdc80106a80 2024-09-19T09:26:14.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.747+0000 7fdc7dffb640 1 -- 172.21.15.16:0/3786559666 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdc780305a0 con 0x7fdc80106a80 2024-09-19T09:26:14.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.748+0000 7fdc7dffb640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdc700a32d0 0x7fdc700a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:14.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.748+0000 7fdc84c1b640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdc700a32d0 0x7fdc700a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:14.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.748+0000 7fdc7dffb640 1 -- 172.21.15.16:0/3786559666 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fdc7802e030 con 0x7fdc80106a80 2024-09-19T09:26:14.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.748+0000 7fdc7dffb640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670] conn(0x7fdc700acf00 0x7fdc700af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:14.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.748+0000 7fdc7dffb640 1 -- 172.21.15.16:0/3786559666 --> [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc7804fe50 con 0x7fdc700acf00 2024-09-19T09:26:14.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.748+0000 7fdc7dffb640 1 -- 172.21.15.16:0/3786559666 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fdc780f1710 con 0x7fdc80106a80 2024-09-19T09:26:14.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.748+0000 7fdc84c1b640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdc700a32d0 0x7fdc700a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009b50 tx=0x7fdc74005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:14.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.749+0000 7fdc7ffff640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670] conn(0x7fdc700acf00 0x7fdc700af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:14.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.749+0000 7fdc7ffff640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670] conn(0x7fdc700acf00 0x7fdc700af3a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:14.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.750+0000 7fdc7dffb640 1 -- 172.21.15.16:0/3786559666 <== osd.7 v2:172.21.15.97:6816/1157122670 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc7804fe50 con 0x7fdc700acf00 2024-09-19T09:26:14.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.764+0000 7fdc86e7f640 1 -- 172.21.15.16:0/3786559666 --> [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdc80062d30 con 0x7fdc700acf00 2024-09-19T09:26:14.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.765+0000 7fdc7dffb640 1 -- 172.21.15.16:0/3786559666 <== osd.7 v2:172.21.15.97:6816/1157122670 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdc80062d30 con 0x7fdc700acf00 2024-09-19T09:26:14.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.765+0000 7fdc86e7f640 1 -- 172.21.15.16:0/3786559666 >> [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670] conn(0x7fdc700acf00 msgr2=0x7fdc700af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:14.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.765+0000 7fdc86e7f640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670] conn(0x7fdc700acf00 0x7fdc700af3a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:14.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.765+0000 7fdc86e7f640 1 -- 172.21.15.16:0/3786559666 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdc700a32d0 msgr2=0x7fdc700a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.765+0000 7fdc86e7f640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fdc700a32d0 0x7fdc700a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009b50 tx=0x7fdc74005f90 comp rx=0 tx=0).stop 2024-09-19T09:26:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.765+0000 7fdc86e7f640 1 -- 172.21.15.16:0/3786559666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc80106a80 msgr2=0x7fdc8019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.765+0000 7fdc86e7f640 1 --2- 172.21.15.16:0/3786559666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc80106a80 0x7fdc8019cd40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdc78005cc0 tx=0x7fdc78032f40 comp rx=0 tx=0).stop 2024-09-19T09:26:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.765+0000 7fdc8541c640 1 -- 172.21.15.16:0/3786559666 reap_dead start 2024-09-19T09:26:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.765+0000 7fdc86e7f640 1 -- 172.21.15.16:0/3786559666 shutdown_connections 2024-09-19T09:26:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.765+0000 7fdc86e7f640 1 -- 172.21.15.16:0/3786559666 >> 172.21.15.16:0/3786559666 conn(0x7fdc800fc7a0 msgr2=0x7fdc8006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.765+0000 7fdc86e7f640 1 -- 172.21.15.16:0/3786559666 shutdown_connections 2024-09-19T09:26:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:14.765+0000 7fdc86e7f640 1 -- 172.21.15.16:0/3786559666 wait complete. 2024-09-19T09:26:15.300 INFO:teuthology.orchestra.run.smithi016.stdout:184683593774 2024-09-19T09:26:15.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T09:26:15.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:15 smithi016 ceph-mon[30979]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-19T09:26:15.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:15 smithi142 ceph-mon[35032]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-19T09:26:15.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:15 smithi097 ceph-mon[34864]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-19T09:26:15.721 INFO:teuthology.orchestra.run.smithi016.stdout:313532612630 2024-09-19T09:26:15.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T09:26:16.136 INFO:teuthology.orchestra.run.smithi016.stdout:103079215164 2024-09-19T09:26:16.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T09:26:16.877 INFO:teuthology.orchestra.run.smithi016.stdout:133143986231 2024-09-19T09:26:16.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T09:26:17.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:17 smithi016 ceph-mon[30979]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T09:26:17.303 INFO:teuthology.orchestra.run.smithi016.stdout:201863462954 2024-09-19T09:26:17.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T09:26:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:17.343 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:17.362 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:17.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:17 smithi142 ceph-mon[35032]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T09:26:17.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:17 smithi097 ceph-mon[34864]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T09:26:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.944+0000 7fd397894640 1 -- 172.21.15.16:0/135088341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390102d40 msgr2=0x7fd390103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.944+0000 7fd397894640 1 --2- 172.21.15.16:0/135088341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390102d40 0x7fd390103120 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd38c009900 tx=0x7fd38c02a480 comp rx=0 tx=0).stop 2024-09-19T09:26:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.944+0000 7fd397894640 1 -- 172.21.15.16:0/135088341 shutdown_connections 2024-09-19T09:26:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.944+0000 7fd397894640 1 --2- 172.21.15.16:0/135088341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd39010c170 0x7fd39010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.944+0000 7fd397894640 1 --2- 172.21.15.16:0/135088341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd390103660 0x7fd39010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.944+0000 7fd397894640 1 --2- 172.21.15.16:0/135088341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390102d40 0x7fd390103120 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.944+0000 7fd397894640 1 -- 172.21.15.16:0/135088341 >> 172.21.15.16:0/135088341 conn(0x7fd3900fc7a0 msgr2=0x7fd3900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.944+0000 7fd397894640 1 -- 172.21.15.16:0/135088341 shutdown_connections 2024-09-19T09:26:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.944+0000 7fd397894640 1 -- 172.21.15.16:0/135088341 wait complete. 2024-09-19T09:26:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd397894640 1 Processor -- start 2024-09-19T09:26:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd397894640 1 -- start start 2024-09-19T09:26:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd397894640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390102d40 0x7fd39019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd397894640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390103660 0x7fd39019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd397894640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd39010c170 0x7fd39019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd397894640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd390112000 con 0x7fd390103660 2024-09-19T09:26:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd397894640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd390111e80 con 0x7fd39010c170 2024-09-19T09:26:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd397894640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd390112180 con 0x7fd390102d40 2024-09-19T09:26:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd394e2f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390103660 0x7fd39019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:17.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd394e2f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390103660 0x7fd39019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33788/0 (socket says 172.21.15.16:33788) 2024-09-19T09:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd394e2f640 1 -- 172.21.15.16:0/3387283704 learned_addr learned my addr 172.21.15.16:0/3387283704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd395630640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390102d40 0x7fd39019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd394e2f640 1 -- 172.21.15.16:0/3387283704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390102d40 msgr2=0x7fd39019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd394e2f640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390102d40 0x7fd39019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd394e2f640 1 -- 172.21.15.16:0/3387283704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd39010c170 msgr2=0x7fd39019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd395e31640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd39010c170 0x7fd39019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd394e2f640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd39010c170 0x7fd39019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd394e2f640 1 -- 172.21.15.16:0/3387283704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39019f5f0 con 0x7fd390103660 2024-09-19T09:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.945+0000 7fd395630640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390102d40 0x7fd39019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.946+0000 7fd394e2f640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390103660 0x7fd39019ab80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd38000b3f0 tx=0x7fd38000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.946+0000 7fd3867fc640 1 -- 172.21.15.16:0/3387283704 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380015020 con 0x7fd390103660 2024-09-19T09:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.946+0000 7fd3867fc640 1 -- 172.21.15.16:0/3387283704 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd380011d10 con 0x7fd390103660 2024-09-19T09:26:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.946+0000 7fd397894640 1 -- 172.21.15.16:0/3387283704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39019f8e0 con 0x7fd390103660 2024-09-19T09:26:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.946+0000 7fd3867fc640 1 -- 172.21.15.16:0/3387283704 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38000f2e0 con 0x7fd390103660 2024-09-19T09:26:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.946+0000 7fd397894640 1 -- 172.21.15.16:0/3387283704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3901a71c0 con 0x7fd390103660 2024-09-19T09:26:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.947+0000 7fd397894640 1 -- 172.21.15.16:0/3387283704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd390103120 con 0x7fd390103660 2024-09-19T09:26:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.948+0000 7fd3867fc640 1 -- 172.21.15.16:0/3387283704 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd380018020 con 0x7fd390103660 2024-09-19T09:26:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.949+0000 7fd3867fc640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd37c0a32d0 0x7fd37c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.949+0000 7fd395630640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd37c0a32d0 0x7fd37c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.949+0000 7fd3867fc640 1 -- 172.21.15.16:0/3387283704 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd3800cf2e0 con 0x7fd390103660 2024-09-19T09:26:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.949+0000 7fd395630640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd37c0a32d0 0x7fd37c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd38c000c00 tx=0x7fd38c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:17.953+0000 7fd3867fc640 1 -- 172.21.15.16:0/3387283704 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd38001f020 con 0x7fd390103660 2024-09-19T09:26:18.034 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:18.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.143+0000 7fd397894640 1 -- 172.21.15.16:0/3387283704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd3901a7460 con 0x7fd390103660 2024-09-19T09:26:18.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.143+0000 7fd3867fc640 1 -- 172.21.15.16:0/3387283704 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd38009bf70 con 0x7fd390103660 2024-09-19T09:26:18.144 INFO:teuthology.orchestra.run.smithi016.stdout:154618822708 2024-09-19T09:26:18.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.145+0000 7fd397894640 1 -- 172.21.15.16:0/3387283704 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd37c0a32d0 msgr2=0x7fd37c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:18.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.145+0000 7fd397894640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd37c0a32d0 0x7fd37c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd38c000c00 tx=0x7fd38c005a20 comp rx=0 tx=0).stop 2024-09-19T09:26:18.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.145+0000 7fd397894640 1 -- 172.21.15.16:0/3387283704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390103660 msgr2=0x7fd39019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.145+0000 7fd397894640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390103660 0x7fd39019ab80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd38000b3f0 tx=0x7fd38000b8b0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.145+0000 7fd397894640 1 -- 172.21.15.16:0/3387283704 shutdown_connections 2024-09-19T09:26:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.145+0000 7fd397894640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd39010c170 0x7fd39019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.145+0000 7fd397894640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd390103660 0x7fd39019ab80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.145+0000 7fd397894640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd37c0a32d0 0x7fd37c0a5790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.145+0000 7fd397894640 1 --2- 172.21.15.16:0/3387283704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390102d40 0x7fd39019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.145+0000 7fd397894640 1 -- 172.21.15.16:0/3387283704 >> 172.21.15.16:0/3387283704 conn(0x7fd3900fc7a0 msgr2=0x7fd39010d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.145+0000 7fd397894640 1 -- 172.21.15.16:0/3387283704 shutdown_connections 2024-09-19T09:26:18.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:18.145+0000 7fd397894640 1 -- 172.21.15.16:0/3387283704 wait complete. 2024-09-19T09:26:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.044+0000 7fb974537640 1 -- 172.21.15.16:0/1300658950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c103660 msgr2=0x7fb96c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.044+0000 7fb974537640 1 --2- 172.21.15.16:0/1300658950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c103660 0x7fb96c10bb50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009900 tx=0x7fb95c02a480 comp rx=0 tx=0).stop 2024-09-19T09:26:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.044+0000 7fb974537640 1 -- 172.21.15.16:0/1300658950 shutdown_connections 2024-09-19T09:26:19.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.044+0000 7fb974537640 1 --2- 172.21.15.16:0/1300658950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb96c10c170 0x7fb96c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.044+0000 7fb974537640 1 --2- 172.21.15.16:0/1300658950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c103660 0x7fb96c10bb50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.044+0000 7fb974537640 1 --2- 172.21.15.16:0/1300658950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb96c102d40 0x7fb96c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.044+0000 7fb974537640 1 -- 172.21.15.16:0/1300658950 >> 172.21.15.16:0/1300658950 conn(0x7fb96c0fc7a0 msgr2=0x7fb96c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.044+0000 7fb974537640 1 -- 172.21.15.16:0/1300658950 shutdown_connections 2024-09-19T09:26:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.044+0000 7fb974537640 1 -- 172.21.15.16:0/1300658950 wait complete. 2024-09-19T09:26:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb974537640 1 Processor -- start 2024-09-19T09:26:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb974537640 1 -- start start 2024-09-19T09:26:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb974537640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c102d40 0x7fb96c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb974537640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb96c103660 0x7fb96c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb974537640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb96c10c170 0x7fb96c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:19.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb974537640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c111f70 con 0x7fb96c102d40 2024-09-19T09:26:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb974537640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c111df0 con 0x7fb96c10c170 2024-09-19T09:26:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb974537640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c1120f0 con 0x7fb96c103660 2024-09-19T09:26:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb9722d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c102d40 0x7fb96c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb9722d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c102d40 0x7fb96c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33796/0 (socket says 172.21.15.16:33796) 2024-09-19T09:26:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb9722d3640 1 -- 172.21.15.16:0/1162600968 learned_addr learned my addr 172.21.15.16:0/1162600968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb971ad2640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb96c103660 0x7fb96c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb972ad4640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb96c10c170 0x7fb96c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb9722d3640 1 -- 172.21.15.16:0/1162600968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb96c103660 msgr2=0x7fb96c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb9722d3640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb96c103660 0x7fb96c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb9722d3640 1 -- 172.21.15.16:0/1162600968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb96c10c170 msgr2=0x7fb96c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:19.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb9722d3640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb96c10c170 0x7fb96c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb9722d3640 1 -- 172.21.15.16:0/1162600968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c19d420 con 0x7fb96c102d40 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb971ad2640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb96c103660 0x7fb96c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb972ad4640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb96c10c170 0x7fb96c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.045+0000 7fb9722d3640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c102d40 0x7fb96c198470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb96800d950 tx=0x7fb96800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.046+0000 7fb9637fe640 1 -- 172.21.15.16:0/1162600968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968017070 con 0x7fb96c102d40 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.046+0000 7fb9637fe640 1 -- 172.21.15.16:0/1162600968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb968009c30 con 0x7fb96c102d40 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.046+0000 7fb974537640 1 -- 172.21.15.16:0/1162600968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c19d710 con 0x7fb96c102d40 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.046+0000 7fb9637fe640 1 -- 172.21.15.16:0/1162600968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96800f6b0 con 0x7fb96c102d40 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.046+0000 7fb974537640 1 -- 172.21.15.16:0/1162600968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c1a4ff0 con 0x7fb96c102d40 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.047+0000 7fb974537640 1 -- 172.21.15.16:0/1162600968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96c103120 con 0x7fb96c102d40 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.048+0000 7fb9637fe640 1 -- 172.21.15.16:0/1162600968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb9680109d0 con 0x7fb96c102d40 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.049+0000 7fb9637fe640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9580a3350 0x7fb9580a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.049+0000 7fb971ad2640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9580a3350 0x7fb9580a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:19.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.049+0000 7fb9637fe640 1 -- 172.21.15.16:0/1162600968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb9680cf6c0 con 0x7fb96c102d40 2024-09-19T09:26:19.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.049+0000 7fb971ad2640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9580a3350 0x7fb9580a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb96c10d690 tx=0x7fb95c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:19.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.052+0000 7fb9637fe640 1 -- 172.21.15.16:0/1162600968 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb968009de0 con 0x7fb96c102d40 2024-09-19T09:26:19.189 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:19 smithi016 ceph-mon[30979]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T09:26:19.189 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:19 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3387283704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T09:26:19.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.232+0000 7fb974537640 1 -- 172.21.15.16:0/1162600968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb96c1997d0 con 0x7fb96c102d40 2024-09-19T09:26:19.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.233+0000 7fb9637fe640 1 -- 172.21.15.16:0/1162600968 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb96c1997d0 con 0x7fb96c102d40 2024-09-19T09:26:19.234 INFO:teuthology.orchestra.run.smithi016.stdout:55834574921 2024-09-19T09:26:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.235+0000 7fb974537640 1 -- 172.21.15.16:0/1162600968 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9580a3350 msgr2=0x7fb9580a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.235+0000 7fb974537640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9580a3350 0x7fb9580a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb96c10d690 tx=0x7fb95c005a20 comp rx=0 tx=0).stop 2024-09-19T09:26:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.235+0000 7fb974537640 1 -- 172.21.15.16:0/1162600968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c102d40 msgr2=0x7fb96c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.235+0000 7fb974537640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c102d40 0x7fb96c198470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb96800d950 tx=0x7fb96800de10 comp rx=0 tx=0).stop 2024-09-19T09:26:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.235+0000 7fb974537640 1 -- 172.21.15.16:0/1162600968 shutdown_connections 2024-09-19T09:26:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.235+0000 7fb974537640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9580a3350 0x7fb9580a5810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.235+0000 7fb974537640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb96c103660 0x7fb96c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.235+0000 7fb974537640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb96c10c170 0x7fb96c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.235+0000 7fb974537640 1 --2- 172.21.15.16:0/1162600968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c102d40 0x7fb96c198470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.235+0000 7fb974537640 1 -- 172.21.15.16:0/1162600968 >> 172.21.15.16:0/1162600968 conn(0x7fb96c0fc7a0 msgr2=0x7fb96c0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.235+0000 7fb974537640 1 -- 172.21.15.16:0/1162600968 shutdown_connections 2024-09-19T09:26:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.235+0000 7fb974537640 1 -- 172.21.15.16:0/1162600968 wait complete. 2024-09-19T09:26:19.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.273+0000 7f2d90466640 1 -- 172.21.15.16:0/2037344943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88101e10 msgr2=0x7f2d881021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:19.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.273+0000 7f2d90466640 1 --2- 172.21.15.16:0/2037344943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88101e10 0x7f2d881021f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009900 tx=0x7f2d8402a410 comp rx=0 tx=0).stop 2024-09-19T09:26:19.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.273+0000 7f2d90466640 1 -- 172.21.15.16:0/2037344943 shutdown_connections 2024-09-19T09:26:19.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.273+0000 7f2d90466640 1 --2- 172.21.15.16:0/2037344943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d88106f40 0x7f2d8810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.273+0000 7f2d90466640 1 --2- 172.21.15.16:0/2037344943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d881027c0 0x7f2d88106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.273+0000 7f2d90466640 1 --2- 172.21.15.16:0/2037344943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88101e10 0x7f2d881021f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.273+0000 7f2d90466640 1 -- 172.21.15.16:0/2037344943 >> 172.21.15.16:0/2037344943 conn(0x7f2d880fd8d0 msgr2=0x7f2d880ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:19.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.273+0000 7f2d90466640 1 -- 172.21.15.16:0/2037344943 shutdown_connections 2024-09-19T09:26:19.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.273+0000 7f2d90466640 1 -- 172.21.15.16:0/2037344943 wait complete. 2024-09-19T09:26:19.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.274+0000 7f2d90466640 1 Processor -- start 2024-09-19T09:26:19.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.274+0000 7f2d90466640 1 -- start start 2024-09-19T09:26:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.274+0000 7f2d90466640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d88101e10 0x7f2d88198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.274+0000 7f2d90466640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d881027c0 0x7f2d88198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.274+0000 7f2d90466640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88106f40 0x7f2d8819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.274+0000 7f2d90466640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d88111fa0 con 0x7f2d88106f40 2024-09-19T09:26:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.274+0000 7f2d90466640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d88111e20 con 0x7f2d881027c0 2024-09-19T09:26:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.274+0000 7f2d90466640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d88112120 con 0x7f2d88101e10 2024-09-19T09:26:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.274+0000 7f2d8ea03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88106f40 0x7f2d8819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.274+0000 7f2d8ea03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88106f40 0x7f2d8819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33804/0 (socket says 172.21.15.16:33804) 2024-09-19T09:26:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.274+0000 7f2d8ea03640 1 -- 172.21.15.16:0/4179550386 learned_addr learned my addr 172.21.15.16:0/4179550386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d8da01640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d881027c0 0x7f2d88198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d8e202640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d88101e10 0x7f2d88198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:19.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d8ea03640 1 -- 172.21.15.16:0/4179550386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d88101e10 msgr2=0x7f2d88198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d8ea03640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d88101e10 0x7f2d88198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d8ea03640 1 -- 172.21.15.16:0/4179550386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d881027c0 msgr2=0x7f2d88198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d8ea03640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d881027c0 0x7f2d88198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d8ea03640 1 -- 172.21.15.16:0/4179550386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8819d3f0 con 0x7f2d88106f40 2024-09-19T09:26:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d8e202640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d88101e10 0x7f2d88198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d8da01640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d881027c0 0x7f2d88198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d8ea03640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88106f40 0x7f2d8819cd10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000eaa0 tx=0x7f2d8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d7f7fe640 1 -- 172.21.15.16:0/4179550386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8000cdb0 con 0x7f2d88106f40 2024-09-19T09:26:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d90466640 1 -- 172.21.15.16:0/4179550386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8819d6e0 con 0x7f2d88106f40 2024-09-19T09:26:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d7f7fe640 1 -- 172.21.15.16:0/4179550386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d80005020 con 0x7f2d88106f40 2024-09-19T09:26:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d7f7fe640 1 -- 172.21.15.16:0/4179550386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d800116d0 con 0x7f2d88106f40 2024-09-19T09:26:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.275+0000 7f2d90466640 1 -- 172.21.15.16:0/4179550386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d881a4fc0 con 0x7f2d88106f40 2024-09-19T09:26:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.276+0000 7f2d90466640 1 -- 172.21.15.16:0/4179550386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d881021f0 con 0x7f2d88106f40 2024-09-19T09:26:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.277+0000 7f2d7f7fe640 1 -- 172.21.15.16:0/4179550386 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2d800054d0 con 0x7f2d88106f40 2024-09-19T09:26:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.278+0000 7f2d7f7fe640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2d740a32d0 0x7f2d740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.278+0000 7f2d8e202640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2d740a32d0 0x7f2d740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.278+0000 7f2d7f7fe640 1 -- 172.21.15.16:0/4179550386 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2d800cfd90 con 0x7f2d88106f40 2024-09-19T09:26:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.279+0000 7f2d8e202640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2d740a32d0 0x7f2d740a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2d84000c00 tx=0x7f2d84032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:19.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.282+0000 7f2d7f7fe640 1 -- 172.21.15.16:0/4179550386 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2d80016030 con 0x7f2d88106f40 2024-09-19T09:26:19.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:19 smithi142 ceph-mon[35032]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T09:26:19.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:19 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3387283704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T09:26:19.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:19 smithi097 ceph-mon[34864]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T09:26:19.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:19 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3387283704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T09:26:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.460+0000 7f2d90466640 1 -- 172.21.15.16:0/4179550386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f2d8810a180 con 0x7f2d88106f40 2024-09-19T09:26:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.461+0000 7f2d7f7fe640 1 -- 172.21.15.16:0/4179550386 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2d8009c910 con 0x7f2d88106f40 2024-09-19T09:26:19.462 INFO:teuthology.orchestra.run.smithi016.stdout:283467841564 2024-09-19T09:26:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.463+0000 7f2d90466640 1 -- 172.21.15.16:0/4179550386 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2d740a32d0 msgr2=0x7f2d740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.463+0000 7f2d90466640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2d740a32d0 0x7f2d740a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2d84000c00 tx=0x7f2d84032c70 comp rx=0 tx=0).stop 2024-09-19T09:26:19.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.463+0000 7f2d90466640 1 -- 172.21.15.16:0/4179550386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88106f40 msgr2=0x7f2d8819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.463+0000 7f2d90466640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88106f40 0x7f2d8819cd10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000eaa0 tx=0x7f2d8000ef60 comp rx=0 tx=0).stop 2024-09-19T09:26:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.463+0000 7f2d90466640 1 -- 172.21.15.16:0/4179550386 shutdown_connections 2024-09-19T09:26:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.463+0000 7f2d90466640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88106f40 0x7f2d8819cd10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.463+0000 7f2d90466640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d881027c0 0x7f2d88198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.463+0000 7f2d90466640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2d740a32d0 0x7f2d740a5790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.463+0000 7f2d90466640 1 --2- 172.21.15.16:0/4179550386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d88101e10 0x7f2d88198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.463+0000 7f2d90466640 1 -- 172.21.15.16:0/4179550386 >> 172.21.15.16:0/4179550386 conn(0x7f2d880fd8d0 msgr2=0x7f2d880fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.463+0000 7f2d90466640 1 -- 172.21.15.16:0/4179550386 shutdown_connections 2024-09-19T09:26:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:19.463+0000 7f2d90466640 1 -- 172.21.15.16:0/4179550386 wait complete. 2024-09-19T09:26:20.243 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:20.244 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:20.284 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574921 for osd.1 2024-09-19T09:26:20.284 DEBUG:teuthology.parallel:result is None 2024-09-19T09:26:20.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:20 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1162600968' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T09:26:20.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:20 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/4179550386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T09:26:20.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:20 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1162600968' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T09:26:20.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:20 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/4179550386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T09:26:20.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:20 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1162600968' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T09:26:20.424 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:20 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/4179550386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T09:26:21.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.011+0000 7f9e7251d640 1 -- 172.21.15.16:0/1845486070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c0ffb60 msgr2=0x7f9e6c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:21.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.011+0000 7f9e7251d640 1 --2- 172.21.15.16:0/1845486070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c0ffb60 0x7f9e6c0fffc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009900 tx=0x7f9e6002a410 comp rx=0 tx=0).stop 2024-09-19T09:26:21.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.011+0000 7f9e7251d640 1 -- 172.21.15.16:0/1845486070 shutdown_connections 2024-09-19T09:26:21.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.011+0000 7f9e7251d640 1 --2- 172.21.15.16:0/1845486070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6c100500 0x7f9e6c10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:21.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.011+0000 7f9e7251d640 1 --2- 172.21.15.16:0/1845486070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c0ffb60 0x7f9e6c0fffc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.011+0000 7f9e7251d640 1 --2- 172.21.15.16:0/1845486070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e6c106b80 0x7f9e6c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.011+0000 7f9e7251d640 1 -- 172.21.15.16:0/1845486070 >> 172.21.15.16:0/1845486070 conn(0x7f9e6c0fba90 msgr2=0x7f9e6c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.011+0000 7f9e7251d640 1 -- 172.21.15.16:0/1845486070 shutdown_connections 2024-09-19T09:26:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.011+0000 7f9e7251d640 1 -- 172.21.15.16:0/1845486070 wait complete. 2024-09-19T09:26:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.012+0000 7f9e7251d640 1 Processor -- start 2024-09-19T09:26:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.012+0000 7f9e7251d640 1 -- start start 2024-09-19T09:26:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.012+0000 7f9e7251d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e6c0ffb60 0x7f9e6c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.012+0000 7f9e7251d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6c100500 0x7f9e6c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.012+0000 7f9e7251d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c106b80 0x7f9e6c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.012+0000 7f9e7251d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c111ff0 con 0x7f9e6c106b80 2024-09-19T09:26:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.012+0000 7f9e7251d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c111e70 con 0x7f9e6c0ffb60 2024-09-19T09:26:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.012+0000 7f9e7251d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c112170 con 0x7f9e6c100500 2024-09-19T09:26:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e70aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c106b80 0x7f9e6c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:21.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e70aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c106b80 0x7f9e6c19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33826/0 (socket says 172.21.15.16:33826) 2024-09-19T09:26:21.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e70aba640 1 -- 172.21.15.16:0/2710080735 learned_addr learned my addr 172.21.15.16:0/2710080735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:21.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e6bfff640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e6c0ffb60 0x7f9e6c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:21.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e6b7fe640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6c100500 0x7f9e6c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:21.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e70aba640 1 -- 172.21.15.16:0/2710080735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6c100500 msgr2=0x7f9e6c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:21.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e70aba640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6c100500 0x7f9e6c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:21.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e70aba640 1 -- 172.21.15.16:0/2710080735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e6c0ffb60 msgr2=0x7f9e6c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:21.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e70aba640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e6c0ffb60 0x7f9e6c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:21.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e70aba640 1 -- 172.21.15.16:0/2710080735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c19f5d0 con 0x7f9e6c106b80 2024-09-19T09:26:21.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e6b7fe640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6c100500 0x7f9e6c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:21.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e70aba640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c106b80 0x7f9e6c19eef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400b9e0 tx=0x7f9e6400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:21.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e697fa640 1 -- 172.21.15.16:0/2710080735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64019070 con 0x7f9e6c106b80 2024-09-19T09:26:21.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e7251d640 1 -- 172.21.15.16:0/2710080735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6c19f8c0 con 0x7f9e6c106b80 2024-09-19T09:26:21.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e697fa640 1 -- 172.21.15.16:0/2710080735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e64014440 con 0x7f9e6c106b80 2024-09-19T09:26:21.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e697fa640 1 -- 172.21.15.16:0/2710080735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e640126b0 con 0x7f9e6c106b80 2024-09-19T09:26:21.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.013+0000 7f9e7251d640 1 -- 172.21.15.16:0/2710080735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6c1a71a0 con 0x7f9e6c106b80 2024-09-19T09:26:21.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.015+0000 7f9e7251d640 1 -- 172.21.15.16:0/2710080735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e6c0fffc0 con 0x7f9e6c106b80 2024-09-19T09:26:21.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.016+0000 7f9e697fa640 1 -- 172.21.15.16:0/2710080735 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9e6401c070 con 0x7f9e6c106b80 2024-09-19T09:26:21.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.016+0000 7f9e697fa640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9e540a32d0 0x7f9e540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:21.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.016+0000 7f9e6bfff640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9e540a32d0 0x7f9e540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:21.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.016+0000 7f9e697fa640 1 -- 172.21.15.16:0/2710080735 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9e640cf000 con 0x7f9e6c106b80 2024-09-19T09:26:21.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.017+0000 7f9e6bfff640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9e540a32d0 0x7f9e540a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00abe0 tx=0x7f9e5c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:21.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.020+0000 7f9e697fa640 1 -- 172.21.15.16:0/2710080735 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9e6409bcd0 con 0x7f9e6c106b80 2024-09-19T09:26:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.200+0000 7f9e7251d640 1 -- 172.21.15.16:0/2710080735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9e6c1a7430 con 0x7f9e6c106b80 2024-09-19T09:26:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.201+0000 7f9e697fa640 1 -- 172.21.15.16:0/2710080735 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9e6409b5d0 con 0x7f9e6c106b80 2024-09-19T09:26:21.202 INFO:teuthology.orchestra.run.smithi016.stdout:77309411396 2024-09-19T09:26:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.203+0000 7f9e7251d640 1 -- 172.21.15.16:0/2710080735 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9e540a32d0 msgr2=0x7f9e540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:21.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.203+0000 7f9e7251d640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9e540a32d0 0x7f9e540a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00abe0 tx=0x7f9e5c00a2d0 comp rx=0 tx=0).stop 2024-09-19T09:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.203+0000 7f9e7251d640 1 -- 172.21.15.16:0/2710080735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c106b80 msgr2=0x7f9e6c19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.203+0000 7f9e7251d640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c106b80 0x7f9e6c19eef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400b9e0 tx=0x7f9e6400bde0 comp rx=0 tx=0).stop 2024-09-19T09:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.203+0000 7f9e7251d640 1 -- 172.21.15.16:0/2710080735 shutdown_connections 2024-09-19T09:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.203+0000 7f9e7251d640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9e540a32d0 0x7f9e540a5790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.203+0000 7f9e7251d640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6c100500 0x7f9e6c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.203+0000 7f9e7251d640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6c106b80 0x7f9e6c19eef0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.203+0000 7f9e7251d640 1 --2- 172.21.15.16:0/2710080735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e6c0ffb60 0x7f9e6c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.203+0000 7f9e7251d640 1 -- 172.21.15.16:0/2710080735 >> 172.21.15.16:0/2710080735 conn(0x7f9e6c0fba90 msgr2=0x7f9e6c10d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.203+0000 7f9e7251d640 1 -- 172.21.15.16:0/2710080735 shutdown_connections 2024-09-19T09:26:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.203+0000 7f9e7251d640 1 -- 172.21.15.16:0/2710080735 wait complete. 2024-09-19T09:26:21.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:21 smithi016 ceph-mon[30979]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T09:26:21.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:21 smithi142 ceph-mon[35032]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T09:26:21.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:21 smithi097 ceph-mon[34864]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-19T09:26:21.929 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822706 got 154618822708 for osd.5 2024-09-19T09:26:21.929 DEBUG:teuthology.parallel:result is None 2024-09-19T09:26:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.564+0000 7f0c20182640 1 -- 172.21.15.16:0/1568720049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c18074590 msgr2=0x7f0c18074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.564+0000 7f0c20182640 1 --2- 172.21.15.16:0/1568720049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c18074590 0x7f0c18074970 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009900 tx=0x7f0c1402a480 comp rx=0 tx=0).stop 2024-09-19T09:26:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.564+0000 7f0c20182640 1 -- 172.21.15.16:0/1568720049 shutdown_connections 2024-09-19T09:26:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.564+0000 7f0c20182640 1 --2- 172.21.15.16:0/1568720049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c18112b90 0x7f0c18114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.564+0000 7f0c20182640 1 --2- 172.21.15.16:0/1568720049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c18074eb0 0x7f0c18112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.564+0000 7f0c20182640 1 --2- 172.21.15.16:0/1568720049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c18074590 0x7f0c18074970 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.564+0000 7f0c20182640 1 -- 172.21.15.16:0/1568720049 >> 172.21.15.16:0/1568720049 conn(0x7f0c180fd8c0 msgr2=0x7f0c180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.564+0000 7f0c20182640 1 -- 172.21.15.16:0/1568720049 shutdown_connections 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.564+0000 7f0c20182640 1 -- 172.21.15.16:0/1568720049 wait complete. 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.565+0000 7f0c20182640 1 Processor -- start 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.565+0000 7f0c20182640 1 -- start start 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.565+0000 7f0c20182640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c18074590 0x7f0c181a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.565+0000 7f0c20182640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c18074eb0 0x7f0c181a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.565+0000 7f0c20182640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c18112b90 0x7f0c1819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.565+0000 7f0c20182640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c18118650 con 0x7f0c18074590 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.565+0000 7f0c20182640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0c181184d0 con 0x7f0c18074eb0 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.565+0000 7f0c20182640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c181187d0 con 0x7f0c18112b90 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.565+0000 7f0c1df1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c18074590 0x7f0c181a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.565+0000 7f0c1df1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c18074590 0x7f0c181a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33844/0 (socket says 172.21.15.16:33844) 2024-09-19T09:26:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.565+0000 7f0c1df1e640 1 -- 172.21.15.16:0/3274528977 learned_addr learned my addr 172.21.15.16:0/3274528977 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:22.115 INFO:teuthology.orchestra.run.smithi016.stdout:261993005088 2024-09-19T09:26:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.566+0000 7f0c1df1e640 1 -- 172.21.15.16:0/3274528977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c18112b90 msgr2=0x7f0c1819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.566+0000 7f0c1df1e640 1 --2- 172.21.15.16:0/3274528977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c18112b90 0x7f0c1819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.566+0000 7f0c1df1e640 1 -- 172.21.15.16:0/3274528977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c18074eb0 msgr2=0x7f0c181a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.566+0000 7f0c1df1e640 1 --2- 172.21.15.16:0/3274528977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c18074eb0 0x7f0c181a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.566+0000 7f0c1df1e640 1 -- 172.21.15.16:0/3274528977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1819f1f0 con 0x7f0c18074590 2024-09-19T09:26:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.566+0000 7f0c1df1e640 1 --2- 172.21.15.16:0/3274528977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c18074590 0x7f0c181a4900 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0c14005ab0 tx=0x7f0c140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.566+0000 7f0c0effd640 1 -- 172.21.15.16:0/3274528977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1403f030 con 0x7f0c18074590 2024-09-19T09:26:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.566+0000 7f0c0effd640 1 -- 172.21.15.16:0/3274528977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c14003ce0 con 0x7f0c18074590 2024-09-19T09:26:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.566+0000 7f0c20182640 1 -- 172.21.15.16:0/3274528977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1819f480 con 0x7f0c18074590 2024-09-19T09:26:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.566+0000 7f0c0effd640 1 -- 172.21.15.16:0/3274528977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14035070 con 0x7f0c18074590 2024-09-19T09:26:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.566+0000 7f0c20182640 1 -- 172.21.15.16:0/3274528977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c181ab7d0 con 0x7f0c18074590 2024-09-19T09:26:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.567+0000 7f0c20182640 1 -- 172.21.15.16:0/3274528977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c180749f0 con 0x7f0c18074590 2024-09-19T09:26:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.568+0000 7f0c0effd640 1 -- 172.21.15.16:0/3274528977 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0c14042030 con 0x7f0c18074590 2024-09-19T09:26:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.569+0000 7f0c0effd640 1 --2- 172.21.15.16:0/3274528977 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0c040a32d0 0x7f0c040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.569+0000 7f0c1d71d640 1 --2- 172.21.15.16:0/3274528977 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0c040a32d0 0x7f0c040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.569+0000 7f0c0effd640 1 -- 172.21.15.16:0/3274528977 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0c140f0730 con 0x7f0c18074590 2024-09-19T09:26:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.570+0000 7f0c1d71d640 1 --2- 172.21.15.16:0/3274528977 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0c040a32d0 0x7f0c040a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0c181001e0 tx=0x7f0c08006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.573+0000 7f0c0effd640 1 -- 172.21.15.16:0/3274528977 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0c140bd2b0 con 0x7f0c18074590 2024-09-19T09:26:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.752+0000 7f0c20182640 1 -- 172.21.15.16:0/3274528977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f0c1819fff0 con 0x7f0c18074590 2024-09-19T09:26:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.752+0000 7f0c0effd640 1 -- 172.21.15.16:0/3274528977 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0c140bcbb0 con 0x7f0c18074590 2024-09-19T09:26:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.754+0000 7f0c20182640 1 -- 172.21.15.16:0/3274528977 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0c040a32d0 msgr2=0x7f0c040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.754+0000 7f0c20182640 1 --2- 172.21.15.16:0/3274528977 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0c040a32d0 0x7f0c040a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0c181001e0 tx=0x7f0c08006c90 comp rx=0 tx=0).stop 2024-09-19T09:26:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.754+0000 7f0c20182640 1 -- 172.21.15.16:0/3274528977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c18074590 msgr2=0x7f0c181a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.754+0000 7f0c20182640 1 --2- 172.21.15.16:0/3274528977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c18074590 0x7f0c181a4900 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0c14005ab0 tx=0x7f0c140042f0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.754+0000 7f0c20182640 1 -- 172.21.15.16:0/3274528977 shutdown_connections 2024-09-19T09:26:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.754+0000 7f0c20182640 1 --2- 172.21.15.16:0/3274528977 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0c040a32d0 0x7f0c040a5790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.754+0000 7f0c20182640 1 --2- 172.21.15.16:0/3274528977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c18112b90 0x7f0c1819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.754+0000 7f0c20182640 1 --2- 172.21.15.16:0/3274528977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c18074eb0 0x7f0c181a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.754+0000 7f0c20182640 1 --2- 172.21.15.16:0/3274528977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c18074590 0x7f0c181a4900 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:22.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.754+0000 7f0c20182640 1 -- 172.21.15.16:0/3274528977 >> 172.21.15.16:0/3274528977 conn(0x7f0c180fd8c0 msgr2=0x7f0c180ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:22.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.754+0000 7f0c20182640 1 -- 172.21.15.16:0/3274528977 shutdown_connections 2024-09-19T09:26:22.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:21.754+0000 7f0c20182640 1 -- 172.21.15.16:0/3274528977 wait complete. 2024-09-19T09:26:22.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:22 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2710080735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T09:26:22.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:22 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3274528977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T09:26:22.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:22 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2710080735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T09:26:22.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:22 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3274528977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T09:26:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:22 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2710080735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T09:26:22.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:22 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3274528977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T09:26:23.170 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:23 smithi016 ceph-mon[30979]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:23.170 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:23 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/527595925' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T09:26:23.229 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411396 for osd.2 2024-09-19T09:26:23.229 DEBUG:teuthology.parallel:result is None 2024-09-19T09:26:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:23 smithi142 ceph-mon[35032]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:23 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/527595925' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T09:26:23.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:23 smithi097 ceph-mon[34864]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:23.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:23 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/527595925' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T09:26:23.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.516+0000 7fe69768c640 1 -- 172.21.15.16:0/3548788945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6901036e0 msgr2=0x7fe690107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:23.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.516+0000 7fe69768c640 1 --2- 172.21.15.16:0/3548788945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6901036e0 0x7fe690107b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe680009930 tx=0x7fe68002a420 comp rx=0 tx=0).stop 2024-09-19T09:26:23.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.516+0000 7fe69768c640 1 -- 172.21.15.16:0/3548788945 shutdown_connections 2024-09-19T09:26:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.516+0000 7fe69768c640 1 --2- 172.21.15.16:0/3548788945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6901082a0 0x7fe69010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.516+0000 7fe69768c640 1 --2- 172.21.15.16:0/3548788945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6901036e0 0x7fe690107b90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.516+0000 7fe69768c640 1 --2- 172.21.15.16:0/3548788945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe690102d30 0x7fe690103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.516+0000 7fe69768c640 1 -- 172.21.15.16:0/3548788945 >> 172.21.15.16:0/3548788945 conn(0x7fe6900fc7d0 msgr2=0x7fe6900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.516+0000 7fe69768c640 1 -- 172.21.15.16:0/3548788945 shutdown_connections 2024-09-19T09:26:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.516+0000 7fe69768c640 1 -- 172.21.15.16:0/3548788945 wait complete. 2024-09-19T09:26:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe69768c640 1 Processor -- start 2024-09-19T09:26:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe69768c640 1 -- start start 2024-09-19T09:26:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe69768c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690102d30 0x7fe6901985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe69768c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6901036e0 0x7fe690198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe69768c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6901082a0 0x7fe69019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe69768c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe690112060 con 0x7fe690102d30 2024-09-19T09:26:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe69768c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe690111ee0 con 0x7fe6901036e0 2024-09-19T09:26:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe69768c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901121e0 con 0x7fe6901082a0 2024-09-19T09:26:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe695428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690102d30 0x7fe6901985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe695428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690102d30 0x7fe6901985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33902/0 (socket says 172.21.15.16:33902) 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe695428640 1 -- 172.21.15.16:0/1129698649 learned_addr learned my addr 172.21.15.16:0/1129698649 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe694c27640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6901036e0 0x7fe690198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe695c29640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6901082a0 0x7fe69019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.517+0000 7fe695428640 1 -- 172.21.15.16:0/1129698649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6901082a0 msgr2=0x7fe69019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.518+0000 7fe695428640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6901082a0 0x7fe69019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.518+0000 7fe695428640 1 -- 172.21.15.16:0/1129698649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6901036e0 msgr2=0x7fe690198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.518+0000 7fe695428640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6901036e0 0x7fe690198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.518+0000 7fe695428640 1 -- 172.21.15.16:0/1129698649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69019d570 con 0x7fe690102d30 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.518+0000 7fe695c29640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6901082a0 0x7fe69019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.518+0000 7fe694c27640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6901036e0 0x7fe690198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.518+0000 7fe695428640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690102d30 0x7fe6901985c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00d950 tx=0x7fe68c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.518+0000 7fe6867fc640 1 -- 172.21.15.16:0/1129698649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c017030 con 0x7fe690102d30 2024-09-19T09:26:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.518+0000 7fe6867fc640 1 -- 172.21.15.16:0/1129698649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68c004020 con 0x7fe690102d30 2024-09-19T09:26:23.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.518+0000 7fe69768c640 1 -- 172.21.15.16:0/1129698649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69019d860 con 0x7fe690102d30 2024-09-19T09:26:23.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.518+0000 7fe6867fc640 1 -- 172.21.15.16:0/1129698649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0047a0 con 0x7fe690102d30 2024-09-19T09:26:23.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.518+0000 7fe69768c640 1 -- 172.21.15.16:0/1129698649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6901a5140 con 0x7fe690102d30 2024-09-19T09:26:23.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.520+0000 7fe69768c640 1 -- 172.21.15.16:0/1129698649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe690103110 con 0x7fe690102d30 2024-09-19T09:26:23.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.521+0000 7fe6867fc640 1 -- 172.21.15.16:0/1129698649 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe68c0041d0 con 0x7fe690102d30 2024-09-19T09:26:23.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.521+0000 7fe6867fc640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe67c0a32d0 0x7fe67c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:23.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.522+0000 7fe6867fc640 1 -- 172.21.15.16:0/1129698649 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe68c0cf550 con 0x7fe690102d30 2024-09-19T09:26:23.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.522+0000 7fe694c27640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe67c0a32d0 0x7fe67c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:23.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.522+0000 7fe694c27640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe67c0a32d0 0x7fe67c0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe68002a940 tx=0x7fe680032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:23.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.526+0000 7fe6867fc640 1 -- 172.21.15.16:0/1129698649 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe68c09c110 con 0x7fe690102d30 2024-09-19T09:26:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.095+0000 7f36f60db640 1 -- 172.21.15.16:0/2171719466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36f0101420 msgr2=0x7f36f010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.095+0000 7f36f60db640 1 --2- 172.21.15.16:0/2171719466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36f0101420 0x7f36f010d950 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009cd0 tx=0x7f36e802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.095+0000 7f36f60db640 1 -- 172.21.15.16:0/2171719466 shutdown_connections 2024-09-19T09:26:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.095+0000 7f36f60db640 1 --2- 172.21.15.16:0/2171719466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36f0101420 0x7f36f010d950 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.095+0000 7f36f60db640 1 --2- 172.21.15.16:0/2171719466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36f0100a80 0x7f36f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.095+0000 7f36f60db640 1 --2- 172.21.15.16:0/2171719466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f0106a80 0x7f36f0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.095+0000 7f36f60db640 1 -- 172.21.15.16:0/2171719466 >> 172.21.15.16:0/2171719466 conn(0x7f36f00fc7a0 msgr2=0x7f36f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.096+0000 7f36f60db640 1 -- 172.21.15.16:0/2171719466 shutdown_connections 2024-09-19T09:26:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.096+0000 7f36f60db640 1 -- 172.21.15.16:0/2171719466 wait complete. 2024-09-19T09:26:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.096+0000 7f36f60db640 1 Processor -- start 2024-09-19T09:26:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.096+0000 7f36f60db640 1 -- start start 2024-09-19T09:26:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.096+0000 7f36f60db640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36f0100a80 0x7f36f0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.096+0000 7f36f60db640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f0101420 0x7f36f01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36f60db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36f0106a80 0x7f36f01a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36f60db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0110e80 con 0x7f36f0106a80 2024-09-19T09:26:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36f60db640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0110d00 con 0x7f36f0100a80 2024-09-19T09:26:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36f60db640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0111000 con 0x7f36f0101420 2024-09-19T09:26:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36f0106a80 0x7f36f01a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36f0106a80 0x7f36f01a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33870/0 (socket says 172.21.15.16:33870) 2024-09-19T09:26:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36effff640 1 -- 172.21.15.16:0/527595925 learned_addr learned my addr 172.21.15.16:0/527595925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36ef7fe640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36f0100a80 0x7f36f0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36eeffd640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f0101420 0x7f36f01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36effff640 1 -- 172.21.15.16:0/527595925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f0101420 msgr2=0x7f36f01997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36effff640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f0101420 0x7f36f01997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36effff640 1 -- 172.21.15.16:0/527595925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36f0100a80 msgr2=0x7f36f0199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36effff640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36f0100a80 0x7f36f0199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36effff640 1 -- 172.21.15.16:0/527595925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f01a4850 con 0x7f36f0106a80 2024-09-19T09:26:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36eeffd640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f0101420 0x7f36f01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36effff640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36f0106a80 0x7f36f01a4310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f36e8006010 tx=0x7f36e8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36ecff9640 1 -- 172.21.15.16:0/527595925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8004450 con 0x7f36f0106a80 2024-09-19T09:26:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36f60db640 1 -- 172.21.15.16:0/527595925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f01a4a20 con 0x7f36f0106a80 2024-09-19T09:26:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36ecff9640 1 -- 172.21.15.16:0/527595925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e8003e10 con 0x7f36f0106a80 2024-09-19T09:26:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.097+0000 7f36ecff9640 1 -- 172.21.15.16:0/527595925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e80466e0 con 0x7f36f0106a80 2024-09-19T09:26:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.098+0000 7f36f60db640 1 -- 172.21.15.16:0/527595925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f01a4ed0 con 0x7f36f0106a80 2024-09-19T09:26:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.098+0000 7f36f60db640 1 -- 172.21.15.16:0/527595925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f0100ee0 con 0x7f36f0106a80 2024-09-19T09:26:23.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.100+0000 7f36ecff9640 1 -- 172.21.15.16:0/527595925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f36e80305a0 con 0x7f36f0106a80 2024-09-19T09:26:23.638 INFO:teuthology.orchestra.run.smithi016.stdout:184683593776 2024-09-19T09:26:23.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.100+0000 7f36ecff9640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f36d80a32d0 0x7f36d80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:23.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.101+0000 7f36ef7fe640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f36d80a32d0 0x7f36d80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:23.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.101+0000 7f36ecff9640 1 -- 172.21.15.16:0/527595925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f36e802e030 con 0x7f36f0106a80 2024-09-19T09:26:23.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.101+0000 7f36ef7fe640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f36d80a32d0 0x7f36d80a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f36e00058c0 tx=0x7f36e000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:23.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.104+0000 7f36ecff9640 1 -- 172.21.15.16:0/527595925 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f36e804b060 con 0x7f36f0106a80 2024-09-19T09:26:23.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.279+0000 7f36f60db640 1 -- 172.21.15.16:0/527595925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f36f0108ff0 con 0x7f36f0106a80 2024-09-19T09:26:23.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.280+0000 7f36ecff9640 1 -- 172.21.15.16:0/527595925 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f36e80bdf10 con 0x7f36f0106a80 2024-09-19T09:26:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.282+0000 7f36f60db640 1 -- 172.21.15.16:0/527595925 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f36d80a32d0 msgr2=0x7f36d80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.282+0000 7f36f60db640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f36d80a32d0 0x7f36d80a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f36e00058c0 tx=0x7f36e000a2d0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.282+0000 7f36f60db640 1 -- 172.21.15.16:0/527595925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36f0106a80 msgr2=0x7f36f01a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.282+0000 7f36f60db640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36f0106a80 0x7f36f01a4310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f36e8006010 tx=0x7f36e8003c20 comp rx=0 tx=0).stop 2024-09-19T09:26:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.282+0000 7f36f60db640 1 -- 172.21.15.16:0/527595925 shutdown_connections 2024-09-19T09:26:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.282+0000 7f36f60db640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f36d80a32d0 0x7f36d80a5790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.282+0000 7f36f60db640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f0101420 0x7f36f01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.282+0000 7f36f60db640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36f0106a80 0x7f36f01a4310 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.282+0000 7f36f60db640 1 --2- 172.21.15.16:0/527595925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36f0100a80 0x7f36f0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.282+0000 7f36f60db640 1 -- 172.21.15.16:0/527595925 >> 172.21.15.16:0/527595925 conn(0x7f36f00fc7a0 msgr2=0x7f36f00fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.282+0000 7f36f60db640 1 -- 172.21.15.16:0/527595925 shutdown_connections 2024-09-19T09:26:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:22.282+0000 7f36f60db640 1 -- 172.21.15.16:0/527595925 wait complete. 2024-09-19T09:26:23.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.707+0000 7fe69768c640 1 -- 172.21.15.16:0/1129698649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe690199920 con 0x7fe690102d30 2024-09-19T09:26:23.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.707+0000 7fe6867fc640 1 -- 172.21.15.16:0/1129698649 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe68c09ba10 con 0x7fe690102d30 2024-09-19T09:26:23.708 INFO:teuthology.orchestra.run.smithi016.stdout:34359738446 2024-09-19T09:26:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.709+0000 7fe69768c640 1 -- 172.21.15.16:0/1129698649 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe67c0a32d0 msgr2=0x7fe67c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.709+0000 7fe69768c640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe67c0a32d0 0x7fe67c0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe68002a940 tx=0x7fe680032c70 comp rx=0 tx=0).stop 2024-09-19T09:26:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.709+0000 7fe69768c640 1 -- 172.21.15.16:0/1129698649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690102d30 msgr2=0x7fe6901985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.709+0000 7fe69768c640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690102d30 0x7fe6901985c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00d950 tx=0x7fe68c00de10 comp rx=0 tx=0).stop 2024-09-19T09:26:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.710+0000 7fe69768c640 1 -- 172.21.15.16:0/1129698649 shutdown_connections 2024-09-19T09:26:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.710+0000 7fe69768c640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe67c0a32d0 0x7fe67c0a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.710+0000 7fe69768c640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6901082a0 0x7fe69019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.710+0000 7fe69768c640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6901036e0 0x7fe690198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.710+0000 7fe69768c640 1 --2- 172.21.15.16:0/1129698649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe690102d30 0x7fe6901985c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.710+0000 7fe69768c640 1 -- 172.21.15.16:0/1129698649 >> 172.21.15.16:0/1129698649 conn(0x7fe6900fc7d0 msgr2=0x7fe690105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.710+0000 7fe69768c640 1 -- 172.21.15.16:0/1129698649 shutdown_connections 2024-09-19T09:26:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.710+0000 7fe69768c640 1 -- 172.21.15.16:0/1129698649 wait complete. 2024-09-19T09:26:24.237 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593774 got 184683593776 for osd.6 2024-09-19T09:26:24.238 DEBUG:teuthology.parallel:result is None 2024-09-19T09:26:24.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:24 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2437966014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T09:26:24.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:24 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1129698649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T09:26:24.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:24 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2437966014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T09:26:24.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:24 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1129698649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T09:26:24.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:24 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2437966014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T09:26:24.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:24 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1129698649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T09:26:24.600 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005085 got 261993005088 for osd.9 2024-09-19T09:26:24.600 DEBUG:teuthology.parallel:result is None 2024-09-19T09:26:25.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:25 smithi142 ceph-mon[35032]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:26:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:25 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:25 smithi097 ceph-mon[34864]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:26:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:25 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:25.432 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:25 smithi016 ceph-mon[30979]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:26:25.433 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:25 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:26:25.461 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738446 for osd.0 2024-09-19T09:26:25.461 DEBUG:teuthology.parallel:result is None 2024-09-19T09:26:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.406+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2115799375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8101420 msgr2=0x7f38f810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.406+0000 7f38fe0ee640 1 --2- 172.21.15.16:0/2115799375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8101420 0x7f38f810d950 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009cd0 tx=0x7f38f002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.406+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2115799375 shutdown_connections 2024-09-19T09:26:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.406+0000 7f38fe0ee640 1 --2- 172.21.15.16:0/2115799375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8101420 0x7f38f810d950 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.406+0000 7f38fe0ee640 1 --2- 172.21.15.16:0/2115799375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38f8100a80 0x7f38f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.406+0000 7f38fe0ee640 1 --2- 172.21.15.16:0/2115799375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f8106a80 0x7f38f8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.406+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2115799375 >> 172.21.15.16:0/2115799375 conn(0x7f38f80fc7a0 msgr2=0x7f38f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.406+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2115799375 shutdown_connections 2024-09-19T09:26:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.406+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2115799375 wait complete. 2024-09-19T09:26:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.406+0000 7f38fe0ee640 1 Processor -- start 2024-09-19T09:26:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38fe0ee640 1 -- start start 2024-09-19T09:26:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38fe0ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8100a80 0x7f38f81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38fe0ee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f8101420 0x7f38f81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38fe0ee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38f8106a80 0x7f38f819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38fe0ee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38f8111fa0 con 0x7f38f8100a80 2024-09-19T09:26:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38fe0ee640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f38f8111e20 con 0x7f38f8106a80 2024-09-19T09:26:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38fe0ee640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f38f8112120 con 0x7f38f8101420 2024-09-19T09:26:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8100a80 0x7f38f81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8100a80 0x7f38f81983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33888/0 (socket says 172.21.15.16:33888) 2024-09-19T09:26:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38f77fe640 1 -- 172.21.15.16:0/2437966014 learned_addr learned my addr 172.21.15.16:0/2437966014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38f7fff640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38f8106a80 0x7f38f819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38f6ffd640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f8101420 0x7f38f81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38f77fe640 1 -- 172.21.15.16:0/2437966014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f8101420 msgr2=0x7f38f81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38f77fe640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f8101420 0x7f38f81988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38f77fe640 1 -- 172.21.15.16:0/2437966014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38f8106a80 msgr2=0x7f38f819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38f77fe640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38f8106a80 0x7f38f819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38f77fe640 1 -- 172.21.15.16:0/2437966014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f819d350 con 0x7f38f8100a80 2024-09-19T09:26:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.407+0000 7f38f7fff640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38f8106a80 0x7f38f819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.408+0000 7f38f77fe640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8100a80 0x7f38f81983a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c920 tx=0x7f38e800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.408+0000 7f38f4ff9640 1 -- 172.21.15.16:0/2437966014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8007c70 con 0x7f38f8100a80 2024-09-19T09:26:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.408+0000 7f38f4ff9640 1 -- 172.21.15.16:0/2437966014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e8007e10 con 0x7f38f8100a80 2024-09-19T09:26:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.408+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2437966014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f819d640 con 0x7f38f8100a80 2024-09-19T09:26:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.408+0000 7f38f4ff9640 1 -- 172.21.15.16:0/2437966014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e8018890 con 0x7f38f8100a80 2024-09-19T09:26:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.408+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2437966014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f81a4f20 con 0x7f38f8100a80 2024-09-19T09:26:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.409+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2437966014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f8100ee0 con 0x7f38f8100a80 2024-09-19T09:26:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.410+0000 7f38f4ff9640 1 -- 172.21.15.16:0/2437966014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f38e8018a70 con 0x7f38f8100a80 2024-09-19T09:26:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.411+0000 7f38f4ff9640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f38e00a32d0 0x7f38e00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.411+0000 7f38f6ffd640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f38e00a32d0 0x7f38e00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.411+0000 7f38f4ff9640 1 -- 172.21.15.16:0/2437966014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f38e80cfc60 con 0x7f38f8100a80 2024-09-19T09:26:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.411+0000 7f38f6ffd640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f38e00a32d0 0x7f38e00a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38ec005fd0 tx=0x7f38ec005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.415+0000 7f38f4ff9640 1 -- 172.21.15.16:0/2437966014 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f38e8014030 con 0x7f38f8100a80 2024-09-19T09:26:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.595+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2437966014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f38f81a51c0 con 0x7f38f8100a80 2024-09-19T09:26:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.595+0000 7f38f4ff9640 1 -- 172.21.15.16:0/2437966014 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f38e809c7e0 con 0x7f38f8100a80 2024-09-19T09:26:25.554 INFO:teuthology.orchestra.run.smithi016.stdout:313532612632 2024-09-19T09:26:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.597+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2437966014 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f38e00a32d0 msgr2=0x7f38e00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.597+0000 7f38fe0ee640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f38e00a32d0 0x7f38e00a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38ec005fd0 tx=0x7f38ec005850 comp rx=0 tx=0).stop 2024-09-19T09:26:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.597+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2437966014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8100a80 msgr2=0x7f38f81983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.597+0000 7f38fe0ee640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8100a80 0x7f38f81983a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c920 tx=0x7f38e800cde0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.598+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2437966014 shutdown_connections 2024-09-19T09:26:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.598+0000 7f38fe0ee640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f38e00a32d0 0x7f38e00a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.598+0000 7f38fe0ee640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f8101420 0x7f38f81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.598+0000 7f38fe0ee640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38f8106a80 0x7f38f819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.598+0000 7f38fe0ee640 1 --2- 172.21.15.16:0/2437966014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8100a80 0x7f38f81983a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.598+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2437966014 >> 172.21.15.16:0/2437966014 conn(0x7f38f80fc7a0 msgr2=0x7f38f80fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.598+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2437966014 shutdown_connections 2024-09-19T09:26:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:23.598+0000 7f38fe0ee640 1 -- 172.21.15.16:0/2437966014 wait complete. 2024-09-19T09:26:26.158 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612630 got 313532612632 for osd.11 2024-09-19T09:26:26.158 DEBUG:teuthology.parallel:result is None 2024-09-19T09:26:26.289 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:26.290 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:26.290 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:26.292 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:26.342 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841561 got 283467841564 for osd.10 2024-09-19T09:26:26.343 DEBUG:teuthology.parallel:result is None 2024-09-19T09:26:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.213+0000 7f6dca6ff640 1 -- 172.21.15.16:0/4165349719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc4101de0 msgr2=0x7f6dc41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.213+0000 7f6dca6ff640 1 --2- 172.21.15.16:0/4165349719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc4101de0 0x7f6dc41021c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009900 tx=0x7f6db402a410 comp rx=0 tx=0).stop 2024-09-19T09:26:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.213+0000 7f6dca6ff640 1 -- 172.21.15.16:0/4165349719 shutdown_connections 2024-09-19T09:26:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.213+0000 7f6dca6ff640 1 --2- 172.21.15.16:0/4165349719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dc4106f10 0x7f6dc410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.213+0000 7f6dca6ff640 1 --2- 172.21.15.16:0/4165349719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc4102790 0x7f6dc41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.213+0000 7f6dca6ff640 1 --2- 172.21.15.16:0/4165349719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc4101de0 0x7f6dc41021c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.213+0000 7f6dca6ff640 1 -- 172.21.15.16:0/4165349719 >> 172.21.15.16:0/4165349719 conn(0x7f6dc40fd8c0 msgr2=0x7f6dc40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.214+0000 7f6dca6ff640 1 -- 172.21.15.16:0/4165349719 shutdown_connections 2024-09-19T09:26:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.214+0000 7f6dca6ff640 1 -- 172.21.15.16:0/4165349719 wait complete. 2024-09-19T09:26:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.214+0000 7f6dca6ff640 1 Processor -- start 2024-09-19T09:26:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.214+0000 7f6dca6ff640 1 -- start start 2024-09-19T09:26:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dca6ff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dc4101de0 0x7f6dc41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dca6ff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc4102790 0x7f6dc41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dca6ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc4106f10 0x7f6dc419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dca6ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc4111f90 con 0x7f6dc4106f10 2024-09-19T09:26:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dca6ff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc4111e10 con 0x7f6dc4102790 2024-09-19T09:26:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dca6ff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc4112110 con 0x7f6dc4101de0 2024-09-19T09:26:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dc8c9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc4106f10 0x7f6dc419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dc8c9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc4106f10 0x7f6dc419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55356/0 (socket says 172.21.15.16:55356) 2024-09-19T09:26:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dc8c9c640 1 -- 172.21.15.16:0/104976815 learned_addr learned my addr 172.21.15.16:0/104976815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dc37fe640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc4102790 0x7f6dc41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dc3fff640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dc4101de0 0x7f6dc41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dc8c9c640 1 -- 172.21.15.16:0/104976815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dc4101de0 msgr2=0x7f6dc41984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dc8c9c640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dc4101de0 0x7f6dc41984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dc8c9c640 1 -- 172.21.15.16:0/104976815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc4102790 msgr2=0x7f6dc41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dc8c9c640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc4102790 0x7f6dc41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dc8c9c640 1 -- 172.21.15.16:0/104976815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc419d460 con 0x7f6dc4106f10 2024-09-19T09:26:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dc8c9c640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc4106f10 0x7f6dc419cd80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00b970 tx=0x7f6dbc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.215+0000 7f6dc17fa640 1 -- 172.21.15.16:0/104976815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc019070 con 0x7f6dc4106f10 2024-09-19T09:26:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.216+0000 7f6dc17fa640 1 -- 172.21.15.16:0/104976815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dbc00cb40 con 0x7f6dc4106f10 2024-09-19T09:26:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.216+0000 7f6dc17fa640 1 -- 172.21.15.16:0/104976815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc007650 con 0x7f6dc4106f10 2024-09-19T09:26:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.216+0000 7f6dca6ff640 1 -- 172.21.15.16:0/104976815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc419d750 con 0x7f6dc4106f10 2024-09-19T09:26:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.216+0000 7f6dca6ff640 1 -- 172.21.15.16:0/104976815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc41a5030 con 0x7f6dc4106f10 2024-09-19T09:26:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.217+0000 7f6dca6ff640 1 -- 172.21.15.16:0/104976815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc41021c0 con 0x7f6dc4106f10 2024-09-19T09:26:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.218+0000 7f6dc17fa640 1 -- 172.21.15.16:0/104976815 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6dbc01c070 con 0x7f6dc4106f10 2024-09-19T09:26:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.219+0000 7f6dc17fa640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6dac0a3350 0x7f6dac0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.219+0000 7f6dc3fff640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6dac0a3350 0x7f6dac0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.219+0000 7f6dc17fa640 1 -- 172.21.15.16:0/104976815 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6dbc0cf380 con 0x7f6dc4106f10 2024-09-19T09:26:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.219+0000 7f6dc3fff640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6dac0a3350 0x7f6dac0a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009230 tx=0x7f6db40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.222+0000 7f6dc17fa640 1 -- 172.21.15.16:0/104976815 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6dbc09ba00 con 0x7f6dc4106f10 2024-09-19T09:26:27.332 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:27 smithi016 ceph-mon[30979]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:27.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:27 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:27.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:27 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:27.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:27 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:27.334 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:27 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:27.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:27 smithi142 ceph-mon[35032]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:27.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:27 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:27.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:27 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:27.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:27 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:27.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:27 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:27.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:27 smithi097 ceph-mon[34864]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:27.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:27 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:27.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:27 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:27.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:27 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:27.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:27 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.402+0000 7f6dca6ff640 1 -- 172.21.15.16:0/104976815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6dc4109b60 con 0x7f6dc4106f10 2024-09-19T09:26:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.402+0000 7f6dc17fa640 1 -- 172.21.15.16:0/104976815 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6dbc09fd00 con 0x7f6dc4106f10 2024-09-19T09:26:27.403 INFO:teuthology.orchestra.run.smithi016.stdout:133143986234 2024-09-19T09:26:27.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.404+0000 7f6dca6ff640 1 -- 172.21.15.16:0/104976815 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6dac0a3350 msgr2=0x7f6dac0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.404+0000 7f6dca6ff640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6dac0a3350 0x7f6dac0a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009230 tx=0x7f6db40058d0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.404+0000 7f6dca6ff640 1 -- 172.21.15.16:0/104976815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc4106f10 msgr2=0x7f6dc419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.404+0000 7f6dca6ff640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc4106f10 0x7f6dc419cd80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00b970 tx=0x7f6dbc00be30 comp rx=0 tx=0).stop 2024-09-19T09:26:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.404+0000 7f6dca6ff640 1 -- 172.21.15.16:0/104976815 shutdown_connections 2024-09-19T09:26:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.404+0000 7f6dca6ff640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dc4106f10 0x7f6dc419cd80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.404+0000 7f6dca6ff640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc4102790 0x7f6dc41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.404+0000 7f6dca6ff640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6dac0a3350 0x7f6dac0a5810 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.404+0000 7f6dca6ff640 1 --2- 172.21.15.16:0/104976815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dc4101de0 0x7f6dc41984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.404+0000 7f6dca6ff640 1 -- 172.21.15.16:0/104976815 >> 172.21.15.16:0/104976815 conn(0x7f6dc40fd8c0 msgr2=0x7f6dc40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.404+0000 7f6dca6ff640 1 -- 172.21.15.16:0/104976815 shutdown_connections 2024-09-19T09:26:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.404+0000 7f6dca6ff640 1 -- 172.21.15.16:0/104976815 wait complete. 2024-09-19T09:26:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.415+0000 7f95bae48640 1 -- 172.21.15.16:0/491243476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b4106f20 msgr2=0x7f95b410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.415+0000 7f95bae48640 1 --2- 172.21.15.16:0/491243476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b4106f20 0x7f95b410eaa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009ca0 tx=0x7f95ac02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.415+0000 7f95bae48640 1 -- 172.21.15.16:0/491243476 shutdown_connections 2024-09-19T09:26:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.415+0000 7f95bae48640 1 --2- 172.21.15.16:0/491243476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95b41027a0 0x7f95b41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.415+0000 7f95bae48640 1 --2- 172.21.15.16:0/491243476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b4106f20 0x7f95b410eaa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.415+0000 7f95bae48640 1 --2- 172.21.15.16:0/491243476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95b4101df0 0x7f95b41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.415+0000 7f95bae48640 1 -- 172.21.15.16:0/491243476 >> 172.21.15.16:0/491243476 conn(0x7f95b40fd8f0 msgr2=0x7f95b40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.415+0000 7f95bae48640 1 -- 172.21.15.16:0/491243476 shutdown_connections 2024-09-19T09:26:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.415+0000 7f95bae48640 1 -- 172.21.15.16:0/491243476 wait complete. 2024-09-19T09:26:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.415+0000 7f95bae48640 1 Processor -- start 2024-09-19T09:26:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.415+0000 7f95bae48640 1 -- start start 2024-09-19T09:26:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95bae48640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95b4101df0 0x7f95b4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95bae48640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95b41027a0 0x7f95b4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95bae48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b4106f20 0x7f95b419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95bae48640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4111f80 con 0x7f95b4106f20 2024-09-19T09:26:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95bae48640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4111e00 con 0x7f95b41027a0 2024-09-19T09:26:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95bae48640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4112100 con 0x7f95b4101df0 2024-09-19T09:26:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b93e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b4106f20 0x7f95b419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b93e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b4106f20 0x7f95b419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55364/0 (socket says 172.21.15.16:55364) 2024-09-19T09:26:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b93e5640 1 -- 172.21.15.16:0/2965059806 learned_addr learned my addr 172.21.15.16:0/2965059806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b3fff640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95b41027a0 0x7f95b4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b8be4640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95b4101df0 0x7f95b4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b93e5640 1 -- 172.21.15.16:0/2965059806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95b4101df0 msgr2=0x7f95b4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b93e5640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95b4101df0 0x7f95b4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b93e5640 1 -- 172.21.15.16:0/2965059806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95b41027a0 msgr2=0x7f95b4198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b93e5640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95b41027a0 0x7f95b4198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b93e5640 1 -- 172.21.15.16:0/2965059806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b419d3b0 con 0x7f95b4106f20 2024-09-19T09:26:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b3fff640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95b41027a0 0x7f95b4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b93e5640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b4106f20 0x7f95b419ccd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f95ac031a20 tx=0x7f95ac031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b1ffb640 1 -- 172.21.15.16:0/2965059806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac003c30 con 0x7f95b4106f20 2024-09-19T09:26:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95b1ffb640 1 -- 172.21.15.16:0/2965059806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ac003dd0 con 0x7f95b4106f20 2024-09-19T09:26:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.416+0000 7f95bae48640 1 -- 172.21.15.16:0/2965059806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b419d640 con 0x7f95b4106f20 2024-09-19T09:26:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.417+0000 7f95b1ffb640 1 -- 172.21.15.16:0/2965059806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac02fc40 con 0x7f95b4106f20 2024-09-19T09:26:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.417+0000 7f95bae48640 1 -- 172.21.15.16:0/2965059806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b41a4f80 con 0x7f95b4106f20 2024-09-19T09:26:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.418+0000 7f95bae48640 1 -- 172.21.15.16:0/2965059806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b41021d0 con 0x7f95b4106f20 2024-09-19T09:26:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.419+0000 7f95b1ffb640 1 -- 172.21.15.16:0/2965059806 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f95ac004050 con 0x7f95b4106f20 2024-09-19T09:26:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.420+0000 7f95b1ffb640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f95a40a32d0 0x7f95a40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.420+0000 7f95b1ffb640 1 -- 172.21.15.16:0/2965059806 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f95ac0f0ca0 con 0x7f95b4106f20 2024-09-19T09:26:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.420+0000 7f95b8be4640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f95a40a32d0 0x7f95a40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.420+0000 7f95b8be4640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f95a40a32d0 0x7f95a40a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009bd0 tx=0x7f95a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.423+0000 7f95b1ffb640 1 -- 172.21.15.16:0/2965059806 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f95ac0bd9c0 con 0x7f95b4106f20 2024-09-19T09:26:27.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.604+0000 7f95bae48640 1 -- 172.21.15.16:0/2965059806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f95b4199760 con 0x7f95b4106f20 2024-09-19T09:26:27.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.605+0000 7f95b1ffb640 1 -- 172.21.15.16:0/2965059806 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f95b4199760 con 0x7f95b4106f20 2024-09-19T09:26:27.606 INFO:teuthology.orchestra.run.smithi016.stdout:201863462956 2024-09-19T09:26:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.607+0000 7f95bae48640 1 -- 172.21.15.16:0/2965059806 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f95a40a32d0 msgr2=0x7f95a40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.607+0000 7f95bae48640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f95a40a32d0 0x7f95a40a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009bd0 tx=0x7f95a8005f90 comp rx=0 tx=0).stop 2024-09-19T09:26:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.607+0000 7f95bae48640 1 -- 172.21.15.16:0/2965059806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b4106f20 msgr2=0x7f95b419ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.607+0000 7f95bae48640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b4106f20 0x7f95b419ccd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f95ac031a20 tx=0x7f95ac031a50 comp rx=0 tx=0).stop 2024-09-19T09:26:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.607+0000 7f95bae48640 1 -- 172.21.15.16:0/2965059806 shutdown_connections 2024-09-19T09:26:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.607+0000 7f95bae48640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95b4106f20 0x7f95b419ccd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.607+0000 7f95bae48640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95b41027a0 0x7f95b4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.607+0000 7f95bae48640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f95a40a32d0 0x7f95a40a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.607+0000 7f95bae48640 1 --2- 172.21.15.16:0/2965059806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95b4101df0 0x7f95b4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.607+0000 7f95bae48640 1 -- 172.21.15.16:0/2965059806 >> 172.21.15.16:0/2965059806 conn(0x7f95b40fd8f0 msgr2=0x7f95b40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.607+0000 7f95bae48640 1 -- 172.21.15.16:0/2965059806 shutdown_connections 2024-09-19T09:26:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:27.607+0000 7f95bae48640 1 -- 172.21.15.16:0/2965059806 wait complete. 2024-09-19T09:26:28.025 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986231 got 133143986234 for osd.4 2024-09-19T09:26:28.025 DEBUG:teuthology.parallel:result is None 2024-09-19T09:26:28.248 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:28 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/104976815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T09:26:28.248 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:28 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2965059806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T09:26:28.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:28 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/104976815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T09:26:28.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:28 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2965059806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T09:26:28.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:28 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/104976815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T09:26:28.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:28 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2965059806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T09:26:28.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.660+0000 7f4200046640 1 -- 172.21.15.16:0/799298660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f8100a80 msgr2=0x7f41f8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:28.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.660+0000 7f4200046640 1 --2- 172.21.15.16:0/799298660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f8100a80 0x7f41f8100ee0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009930 tx=0x7f41e802a420 comp rx=0 tx=0).stop 2024-09-19T09:26:28.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 -- 172.21.15.16:0/799298660 shutdown_connections 2024-09-19T09:26:28.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 --2- 172.21.15.16:0/799298660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f8101420 0x7f41f810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 --2- 172.21.15.16:0/799298660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f8100a80 0x7f41f8100ee0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 --2- 172.21.15.16:0/799298660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f8106a80 0x7f41f8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 -- 172.21.15.16:0/799298660 >> 172.21.15.16:0/799298660 conn(0x7f41f80fc7a0 msgr2=0x7f41f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 -- 172.21.15.16:0/799298660 shutdown_connections 2024-09-19T09:26:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 -- 172.21.15.16:0/799298660 wait complete. 2024-09-19T09:26:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 Processor -- start 2024-09-19T09:26:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 -- start start 2024-09-19T09:26:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f8100a80 0x7f41f80fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f8101420 0x7f41f80fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f8106a80 0x7f41f80fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41f8110e80 con 0x7f41f8100a80 2024-09-19T09:26:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41f8110d00 con 0x7f41f8106a80 2024-09-19T09:26:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.661+0000 7f4200046640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41f8111000 con 0x7f41f8101420 2024-09-19T09:26:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41fdde2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f8100a80 0x7f41f80fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41fdde2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f8100a80 0x7f41f80fcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55386/0 (socket says 172.21.15.16:55386) 2024-09-19T09:26:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41fdde2640 1 -- 172.21.15.16:0/1495699025 learned_addr learned my addr 172.21.15.16:0/1495699025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41fe5e3640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f8106a80 0x7f41f80fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41fd5e1640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f8101420 0x7f41f80fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41fdde2640 1 -- 172.21.15.16:0/1495699025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f8101420 msgr2=0x7f41f80fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41fdde2640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f8101420 0x7f41f80fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41fdde2640 1 -- 172.21.15.16:0/1495699025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f8106a80 msgr2=0x7f41f80fd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41fdde2640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f8106a80 0x7f41f80fd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41fdde2640 1 -- 172.21.15.16:0/1495699025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f81a6980 con 0x7f41f8100a80 2024-09-19T09:26:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41fe5e3640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f8106a80 0x7f41f80fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:28.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41fdde2640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f8100a80 0x7f41f80fcd40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f41f400c970 tx=0x7f41f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41eeffd640 1 -- 172.21.15.16:0/1495699025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4007bb0 con 0x7f41f8100a80 2024-09-19T09:26:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41eeffd640 1 -- 172.21.15.16:0/1495699025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f4007d50 con 0x7f41f8100a80 2024-09-19T09:26:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f4200046640 1 -- 172.21.15.16:0/1495699025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f81a6c70 con 0x7f41f8100a80 2024-09-19T09:26:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f41eeffd640 1 -- 172.21.15.16:0/1495699025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f400f450 con 0x7f41f8100a80 2024-09-19T09:26:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.662+0000 7f4200046640 1 -- 172.21.15.16:0/1495699025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f81a7130 con 0x7f41f8100a80 2024-09-19T09:26:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.663+0000 7f4200046640 1 -- 172.21.15.16:0/1495699025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f8100ee0 con 0x7f41f8100a80 2024-09-19T09:26:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.665+0000 7f41eeffd640 1 -- 172.21.15.16:0/1495699025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f41f4005020 con 0x7f41f8100a80 2024-09-19T09:26:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.665+0000 7f41eeffd640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41e40a32d0 0x7f41e40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.665+0000 7f41fd5e1640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41e40a32d0 0x7f41e40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.665+0000 7f41eeffd640 1 -- 172.21.15.16:0/1495699025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f41f40cf8e0 con 0x7f41f8100a80 2024-09-19T09:26:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.666+0000 7f41fd5e1640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41e40a32d0 0x7f41e40a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f41e8000c00 tx=0x7f41e8031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:28.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.669+0000 7f41eeffd640 1 -- 172.21.15.16:0/1495699025 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f41f4014030 con 0x7f41f8100a80 2024-09-19T09:26:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.846+0000 7f4200046640 1 -- 172.21.15.16:0/1495699025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f41f810a030 con 0x7f41f8100a80 2024-09-19T09:26:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.846+0000 7f41eeffd640 1 -- 172.21.15.16:0/1495699025 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f41f409c4a0 con 0x7f41f8100a80 2024-09-19T09:26:28.848 INFO:teuthology.orchestra.run.smithi016.stdout:103079215168 2024-09-19T09:26:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.849+0000 7f4200046640 1 -- 172.21.15.16:0/1495699025 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41e40a32d0 msgr2=0x7f41e40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.849+0000 7f4200046640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41e40a32d0 0x7f41e40a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f41e8000c00 tx=0x7f41e8031490 comp rx=0 tx=0).stop 2024-09-19T09:26:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.849+0000 7f4200046640 1 -- 172.21.15.16:0/1495699025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f8100a80 msgr2=0x7f41f80fcd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.849+0000 7f4200046640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f8100a80 0x7f41f80fcd40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f41f400c970 tx=0x7f41f400ce30 comp rx=0 tx=0).stop 2024-09-19T09:26:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.850+0000 7f4200046640 1 -- 172.21.15.16:0/1495699025 shutdown_connections 2024-09-19T09:26:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.850+0000 7f4200046640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41e40a32d0 0x7f41e40a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.850+0000 7f4200046640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41f8101420 0x7f41f80fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.850+0000 7f4200046640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41f8106a80 0x7f41f80fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.850+0000 7f4200046640 1 --2- 172.21.15.16:0/1495699025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41f8100a80 0x7f41f80fcd40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.850+0000 7f4200046640 1 -- 172.21.15.16:0/1495699025 >> 172.21.15.16:0/1495699025 conn(0x7f41f80fc7a0 msgr2=0x7f41f806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:28.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.850+0000 7f4200046640 1 -- 172.21.15.16:0/1495699025 shutdown_connections 2024-09-19T09:26:28.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.850+0000 7f4200046640 1 -- 172.21.15.16:0/1495699025 wait complete. 2024-09-19T09:26:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.874+0000 7f9302854640 1 -- 172.21.15.16:0/1449789965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc10c180 msgr2=0x7f92fc10e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.874+0000 7f9302854640 1 --2- 172.21.15.16:0/1449789965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc10c180 0x7f92fc10e620 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009ca0 tx=0x7f92f402a950 comp rx=0 tx=0).stop 2024-09-19T09:26:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.874+0000 7f9302854640 1 -- 172.21.15.16:0/1449789965 shutdown_connections 2024-09-19T09:26:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.874+0000 7f9302854640 1 --2- 172.21.15.16:0/1449789965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92fc103670 0x7f92fc10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.874+0000 7f9302854640 1 --2- 172.21.15.16:0/1449789965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc10c180 0x7f92fc10e620 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.874+0000 7f9302854640 1 --2- 172.21.15.16:0/1449789965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc102d50 0x7f92fc103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.874+0000 7f9302854640 1 -- 172.21.15.16:0/1449789965 >> 172.21.15.16:0/1449789965 conn(0x7f92fc0fc7d0 msgr2=0x7f92fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.874+0000 7f9302854640 1 -- 172.21.15.16:0/1449789965 shutdown_connections 2024-09-19T09:26:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.874+0000 7f9302854640 1 -- 172.21.15.16:0/1449789965 wait complete. 2024-09-19T09:26:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.875+0000 7f9302854640 1 Processor -- start 2024-09-19T09:26:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.875+0000 7f9302854640 1 -- start start 2024-09-19T09:26:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.875+0000 7f9302854640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc102d50 0x7f92fc1009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.875+0000 7f9302854640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92fc103670 0x7f92fc100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.875+0000 7f9302854640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc10c180 0x7f92fc101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.875+0000 7f9302854640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc111fb0 con 0x7f92fc102d50 2024-09-19T09:26:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.875+0000 7f9302854640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc111e30 con 0x7f92fc10c180 2024-09-19T09:26:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.875+0000 7f9302854640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc112130 con 0x7f92fc103670 2024-09-19T09:26:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.875+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc102d50 0x7f92fc1009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.875+0000 7f92fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc102d50 0x7f92fc1009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55400/0 (socket says 172.21.15.16:55400) 2024-09-19T09:26:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.875+0000 7f92fbfff640 1 -- 172.21.15.16:0/2282555487 learned_addr learned my addr 172.21.15.16:0/2282555487 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.876+0000 7f92fb7fe640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92fc103670 0x7f92fc100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.876+0000 7f9300df1640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc10c180 0x7f92fc101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.876+0000 7f92fbfff640 1 -- 172.21.15.16:0/2282555487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92fc103670 msgr2=0x7f92fc100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.876+0000 7f92fbfff640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92fc103670 0x7f92fc100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.876+0000 7f92fbfff640 1 -- 172.21.15.16:0/2282555487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc10c180 msgr2=0x7f92fc101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.876+0000 7f92fbfff640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc10c180 0x7f92fc101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.876+0000 7f92fbfff640 1 -- 172.21.15.16:0/2282555487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc1a6970 con 0x7f92fc102d50 2024-09-19T09:26:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.876+0000 7f92fbfff640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc102d50 0x7f92fc1009b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00d950 tx=0x7f92ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.876+0000 7f92f97fa640 1 -- 172.21.15.16:0/2282555487 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec017030 con 0x7f92fc102d50 2024-09-19T09:26:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.876+0000 7f92f97fa640 1 -- 172.21.15.16:0/2282555487 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92ec004020 con 0x7f92fc102d50 2024-09-19T09:26:28.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.876+0000 7f92f97fa640 1 -- 172.21.15.16:0/2282555487 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec0047a0 con 0x7f92fc102d50 2024-09-19T09:26:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.876+0000 7f9302854640 1 -- 172.21.15.16:0/2282555487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc1a6c60 con 0x7f92fc102d50 2024-09-19T09:26:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.876+0000 7f9302854640 1 -- 172.21.15.16:0/2282555487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc1a71a0 con 0x7f92fc102d50 2024-09-19T09:26:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.878+0000 7f9302854640 1 -- 172.21.15.16:0/2282555487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92fc103130 con 0x7f92fc102d50 2024-09-19T09:26:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.879+0000 7f92f97fa640 1 -- 172.21.15.16:0/2282555487 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f92ec0041d0 con 0x7f92fc102d50 2024-09-19T09:26:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.880+0000 7f92f97fa640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f92e40a33d0 0x7f92e40a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.880+0000 7f92f97fa640 1 -- 172.21.15.16:0/2282555487 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f92ec0cf460 con 0x7f92fc102d50 2024-09-19T09:26:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.880+0000 7f92fb7fe640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f92e40a33d0 0x7f92e40a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.881+0000 7f92fb7fe640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f92e40a33d0 0x7f92e40a5890 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f92fc102440 tx=0x7f92f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:28.884+0000 7f92f97fa640 1 -- 172.21.15.16:0/2282555487 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f92ec09bfe0 con 0x7f92fc102d50 2024-09-19T09:26:29.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.059+0000 7f9302854640 1 -- 172.21.15.16:0/2282555487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f92fc107220 con 0x7f92fc102d50 2024-09-19T09:26:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.059+0000 7f92f97fa640 1 -- 172.21.15.16:0/2282555487 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f92ec09b8e0 con 0x7f92fc102d50 2024-09-19T09:26:29.061 INFO:teuthology.orchestra.run.smithi016.stdout:231928234023 2024-09-19T09:26:29.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.061+0000 7f9302854640 1 -- 172.21.15.16:0/2282555487 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f92e40a33d0 msgr2=0x7f92e40a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.061+0000 7f9302854640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f92e40a33d0 0x7f92e40a5890 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f92fc102440 tx=0x7f92f0005f90 comp rx=0 tx=0).stop 2024-09-19T09:26:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.061+0000 7f9302854640 1 -- 172.21.15.16:0/2282555487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc102d50 msgr2=0x7f92fc1009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.061+0000 7f9302854640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc102d50 0x7f92fc1009b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00d950 tx=0x7f92ec00de10 comp rx=0 tx=0).stop 2024-09-19T09:26:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.062+0000 7f9302854640 1 -- 172.21.15.16:0/2282555487 shutdown_connections 2024-09-19T09:26:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.062+0000 7f9302854640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f92e40a33d0 0x7f92e40a5890 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.062+0000 7f9302854640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92fc103670 0x7f92fc100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.062+0000 7f9302854640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92fc10c180 0x7f92fc101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.062+0000 7f9302854640 1 --2- 172.21.15.16:0/2282555487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc102d50 0x7f92fc1009b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.062+0000 7f9302854640 1 -- 172.21.15.16:0/2282555487 >> 172.21.15.16:0/2282555487 conn(0x7f92fc0fc7d0 msgr2=0x7f92fc0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.062+0000 7f9302854640 1 -- 172.21.15.16:0/2282555487 shutdown_connections 2024-09-19T09:26:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:29.062+0000 7f9302854640 1 -- 172.21.15.16:0/2282555487 wait complete. 2024-09-19T09:26:29.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:29 smithi142 ceph-mon[35032]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:26:29.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:29 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1495699025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T09:26:29.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:29 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2282555487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T09:26:29.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:29 smithi097 ceph-mon[34864]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:26:29.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:29 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1495699025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T09:26:29.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:29 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2282555487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T09:26:29.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:29 smithi016 ceph-mon[30979]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:26:29.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:29 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1495699025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T09:26:29.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:29 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2282555487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T09:26:29.696 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215164 got 103079215168 for osd.3 2024-09-19T09:26:29.696 DEBUG:teuthology.parallel:result is None 2024-09-19T09:26:30.771 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462954 got 201863462956 for osd.7 2024-09-19T09:26:30.771 DEBUG:teuthology.parallel:result is None 2024-09-19T09:26:31.198 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234019 got 231928234023 for osd.8 2024-09-19T09:26:31.198 DEBUG:teuthology.parallel:result is None 2024-09-19T09:26:31.198 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T09:26:31.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:26:31.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:31 smithi142 ceph-mon[35032]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:26:31.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:31 smithi097 ceph-mon[34864]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:26:31.492 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:31.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:31 smithi016 ceph-mon[30979]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:26:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.908+0000 7f4a701b9640 1 -- 172.21.15.16:0/810375841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68106f10 msgr2=0x7f4a6810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.908+0000 7f4a701b9640 1 --2- 172.21.15.16:0/810375841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68106f10 0x7f4a6810ea90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009900 tx=0x7f4a6002a410 comp rx=0 tx=0).stop 2024-09-19T09:26:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.908+0000 7f4a701b9640 1 -- 172.21.15.16:0/810375841 shutdown_connections 2024-09-19T09:26:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.908+0000 7f4a701b9640 1 --2- 172.21.15.16:0/810375841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68106f10 0x7f4a6810ea90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.908+0000 7f4a701b9640 1 --2- 172.21.15.16:0/810375841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68102790 0x7f4a681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.908+0000 7f4a701b9640 1 --2- 172.21.15.16:0/810375841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a68101de0 0x7f4a681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.908+0000 7f4a701b9640 1 -- 172.21.15.16:0/810375841 >> 172.21.15.16:0/810375841 conn(0x7f4a680fd8c0 msgr2=0x7f4a680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.909+0000 7f4a701b9640 1 -- 172.21.15.16:0/810375841 shutdown_connections 2024-09-19T09:26:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.909+0000 7f4a701b9640 1 -- 172.21.15.16:0/810375841 wait complete. 2024-09-19T09:26:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.909+0000 7f4a701b9640 1 Processor -- start 2024-09-19T09:26:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.909+0000 7f4a701b9640 1 -- start start 2024-09-19T09:26:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a701b9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a68101de0 0x7f4a6819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a701b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68102790 0x7f4a6819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a701b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68106f10 0x7f4a6819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a701b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68111fb0 con 0x7f4a68102790 2024-09-19T09:26:31.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a701b9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68111e30 con 0x7f4a68106f10 2024-09-19T09:26:31.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a701b9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68112130 con 0x7f4a68101de0 2024-09-19T09:26:31.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a6d754640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68102790 0x7f4a6819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:31.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a6d754640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68102790 0x7f4a6819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55422/0 (socket says 172.21.15.16:55422) 2024-09-19T09:26:31.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a6d754640 1 -- 172.21.15.16:0/1368866461 learned_addr learned my addr 172.21.15.16:0/1368866461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:31.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a6df55640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a68101de0 0x7f4a6819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:31.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a6e756640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68106f10 0x7f4a6819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:31.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a6d754640 1 -- 172.21.15.16:0/1368866461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a68101de0 msgr2=0x7f4a6819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:31.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a6d754640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a68101de0 0x7f4a6819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:31.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a6d754640 1 -- 172.21.15.16:0/1368866461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68106f10 msgr2=0x7f4a6819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:31.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a6d754640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68106f10 0x7f4a6819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:31.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a6d754640 1 -- 172.21.15.16:0/1368866461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6819f6c0 con 0x7f4a68102790 2024-09-19T09:26:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a6d754640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68102790 0x7f4a6819ac50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00b4a0 tx=0x7f4a5c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.910+0000 7f4a5affd640 1 -- 172.21.15.16:0/1368866461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c014070 con 0x7f4a68102790 2024-09-19T09:26:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.911+0000 7f4a5affd640 1 -- 172.21.15.16:0/1368866461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a5c010af0 con 0x7f4a68102790 2024-09-19T09:26:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.911+0000 7f4a701b9640 1 -- 172.21.15.16:0/1368866461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6819f9b0 con 0x7f4a68102790 2024-09-19T09:26:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.911+0000 7f4a5affd640 1 -- 172.21.15.16:0/1368866461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c017d10 con 0x7f4a68102790 2024-09-19T09:26:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.911+0000 7f4a701b9640 1 -- 172.21.15.16:0/1368866461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a681a7290 con 0x7f4a68102790 2024-09-19T09:26:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.912+0000 7f4a701b9640 1 -- 172.21.15.16:0/1368866461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a681021c0 con 0x7f4a68102790 2024-09-19T09:26:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.913+0000 7f4a5affd640 1 -- 172.21.15.16:0/1368866461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4a5c017070 con 0x7f4a68102790 2024-09-19T09:26:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.914+0000 7f4a5affd640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4a540a32d0 0x7f4a540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.914+0000 7f4a6df55640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4a540a32d0 0x7f4a540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.914+0000 7f4a5affd640 1 -- 172.21.15.16:0/1368866461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4a5c0cf000 con 0x7f4a68102790 2024-09-19T09:26:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.914+0000 7f4a6df55640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4a540a32d0 0x7f4a540a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4a64005fd0 tx=0x7f4a64009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:31.917+0000 7f4a5affd640 1 -- 172.21.15.16:0/1368866461 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4a5c09b6c0 con 0x7f4a68102790 2024-09-19T09:26:32.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.093+0000 7f4a701b9640 1 -- 172.21.15.16:0/1368866461 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4a68075430 con 0x7f4a540a32d0 2024-09-19T09:26:32.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.099+0000 7f4a5affd640 1 -- 172.21.15.16:0/1368866461 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258765 (secure 0 0 0) 0x7f4a68075430 con 0x7f4a540a32d0 2024-09-19T09:26:32.103 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:26:32.104 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T09:26:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.102+0000 7f4a701b9640 1 -- 172.21.15.16:0/1368866461 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4a540a32d0 msgr2=0x7f4a540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.102+0000 7f4a701b9640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4a540a32d0 0x7f4a540a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4a64005fd0 tx=0x7f4a64009680 comp rx=0 tx=0).stop 2024-09-19T09:26:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.102+0000 7f4a701b9640 1 -- 172.21.15.16:0/1368866461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68102790 msgr2=0x7f4a6819ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.102+0000 7f4a701b9640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68102790 0x7f4a6819ac50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00b4a0 tx=0x7f4a5c00b960 comp rx=0 tx=0).stop 2024-09-19T09:26:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.102+0000 7f4a701b9640 1 -- 172.21.15.16:0/1368866461 shutdown_connections 2024-09-19T09:26:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.102+0000 7f4a701b9640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a68106f10 0x7f4a6819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.102+0000 7f4a701b9640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68102790 0x7f4a6819ac50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.102+0000 7f4a701b9640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4a540a32d0 0x7f4a540a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.102+0000 7f4a701b9640 1 --2- 172.21.15.16:0/1368866461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a68101de0 0x7f4a6819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.102+0000 7f4a701b9640 1 -- 172.21.15.16:0/1368866461 >> 172.21.15.16:0/1368866461 conn(0x7f4a680fd8c0 msgr2=0x7f4a680feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:32.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.102+0000 7f4a701b9640 1 -- 172.21.15.16:0/1368866461 shutdown_connections 2024-09-19T09:26:32.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:32.102+0000 7f4a701b9640 1 -- 172.21.15.16:0/1368866461 wait complete. 2024-09-19T09:26:32.795 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-19T09:26:31.347947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658360,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":644,"num_read_kb":441,"num_write":428,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1182578,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386864,"kb_used_data":12636,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554960,"statfs":{"total":1151940427776,"available":1151544279040,"internally_reserved":0,"allocated":12939264,"data_stored":7120680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002824"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988927+0000","last_change":"2024-09-19T09:25:29.294965+0000","last_active":"2024-09-19T09:25:42.988927+0000","last_peered":"2024-09-19T09:25:42.988927+0000","last_clean":"2024-09-19T09:25:42.988927+0000","last_became_active":"2024-09-19T09:25:29.294157+0000","last_became_peered":"2024-09-19T09:25:29.294157+0000","last_unstale":"2024-09-19T09:25:42.988927+0000","last_undegraded":"2024-09-19T09:25:42.988927+0000","last_fullsized":"2024-09-19T09:25:42.988927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:24.643835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992123+0000","last_change":"2024-09-19T09:25:29.298888+0000","last_active":"2024-09-19T09:25:42.992123+0000","last_peered":"2024-09-19T09:25:42.992123+0000","last_clean":"2024-09-19T09:25:42.992123+0000","last_became_active":"2024-09-19T09:25:29.298245+0000","last_became_peered":"2024-09-19T09:25:29.298245+0000","last_unstale":"2024-09-19T09:25:42.992123+0000","last_undegraded":"2024-09-19T09:25:42.992123+0000","last_fullsized":"2024-09-19T09:25:42.992123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:03.162342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992284+0000","last_change":"2024-09-19T09:25:29.292300+0000","last_active":"2024-09-19T09:25:42.992284+0000","last_peered":"2024-09-19T09:25:42.992284+0000","last_clean":"2024-09-19T09:25:42.992284+0000","last_became_active":"2024-09-19T09:25:29.291973+0000","last_became_peered":"2024-09-19T09:25:29.291973+0000","last_unstale":"2024-09-19T09:25:42.992284+0000","last_undegraded":"2024-09-19T09:25:42.992284+0000","last_fullsized":"2024-09-19T09:25:42.992284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:19.672778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990140+0000","last_change":"2024-09-19T09:25:29.305695+0000","last_active":"2024-09-19T09:25:42.990140+0000","last_peered":"2024-09-19T09:25:42.990140+0000","last_clean":"2024-09-19T09:25:42.990140+0000","last_became_active":"2024-09-19T09:25:29.305244+0000","last_became_peered":"2024-09-19T09:25:29.305244+0000","last_unstale":"2024-09-19T09:25:42.990140+0000","last_undegraded":"2024-09-19T09:25:42.990140+0000","last_fullsized":"2024-09-19T09:25:42.990140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:38.054624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991017+0000","last_change":"2024-09-19T09:25:29.295259+0000","last_active":"2024-09-19T09:25:42.991017+0000","last_peered":"2024-09-19T09:25:42.991017+0000","last_clean":"2024-09-19T09:25:42.991017+0000","last_became_active":"2024-09-19T09:25:29.294936+0000","last_became_peered":"2024-09-19T09:25:29.294936+0000","last_unstale":"2024-09-19T09:25:42.991017+0000","last_undegraded":"2024-09-19T09:25:42.991017+0000","last_fullsized":"2024-09-19T09:25:42.991017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:59.193504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986511+0000","last_change":"2024-09-19T09:25:29.299650+0000","last_active":"2024-09-19T09:25:42.986511+0000","last_peered":"2024-09-19T09:25:42.986511+0000","last_clean":"2024-09-19T09:25:42.986511+0000","last_became_active":"2024-09-19T09:25:29.299182+0000","last_became_peered":"2024-09-19T09:25:29.299182+0000","last_unstale":"2024-09-19T09:25:42.986511+0000","last_undegraded":"2024-09-19T09:25:42.986511+0000","last_fullsized":"2024-09-19T09:25:42.986511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:44.637771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989644+0000","last_change":"2024-09-19T09:25:29.302501+0000","last_active":"2024-09-19T09:25:42.989644+0000","last_peered":"2024-09-19T09:25:42.989644+0000","last_clean":"2024-09-19T09:25:42.989644+0000","last_became_active":"2024-09-19T09:25:29.301419+0000","last_became_peered":"2024-09-19T09:25:29.301419+0000","last_unstale":"2024-09-19T09:25:42.989644+0000","last_undegraded":"2024-09-19T09:25:42.989644+0000","last_fullsized":"2024-09-19T09:25:42.989644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:50.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990840+0000","last_change":"2024-09-19T09:25:29.300212+0000","last_active":"2024-09-19T09:25:42.990840+0000","last_peered":"2024-09-19T09:25:42.990840+0000","last_clean":"2024-09-19T09:25:42.990840+0000","last_became_active":"2024-09-19T09:25:29.299873+0000","last_became_peered":"2024-09-19T09:25:29.299873+0000","last_unstale":"2024-09-19T09:25:42.990840+0000","last_undegraded":"2024-09-19T09:25:42.990840+0000","last_fullsized":"2024-09-19T09:25:42.990840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:34.620314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.004117+0000","last_change":"2024-09-19T09:25:29.290214+0000","last_active":"2024-09-19T09:25:43.004117+0000","last_peered":"2024-09-19T09:25:43.004117+0000","last_clean":"2024-09-19T09:25:43.004117+0000","last_became_active":"2024-09-19T09:25:29.289707+0000","last_became_peered":"2024-09-19T09:25:29.289707+0000","last_unstale":"2024-09-19T09:25:43.004117+0000","last_undegraded":"2024-09-19T09:25:43.004117+0000","last_fullsized":"2024-09-19T09:25:43.004117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:16.341340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.987005+0000","last_change":"2024-09-19T09:25:29.301774+0000","last_active":"2024-09-19T09:25:42.987005+0000","last_peered":"2024-09-19T09:25:42.987005+0000","last_clean":"2024-09-19T09:25:42.987005+0000","last_became_active":"2024-09-19T09:25:29.301398+0000","last_became_peered":"2024-09-19T09:25:29.301398+0000","last_unstale":"2024-09-19T09:25:42.987005+0000","last_undegraded":"2024-09-19T09:25:42.987005+0000","last_fullsized":"2024-09-19T09:25:42.987005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:36.058038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989844+0000","last_change":"2024-09-19T09:25:29.301475+0000","last_active":"2024-09-19T09:25:42.989844+0000","last_peered":"2024-09-19T09:25:42.989844+0000","last_clean":"2024-09-19T09:25:42.989844+0000","last_became_active":"2024-09-19T09:25:29.300908+0000","last_became_peered":"2024-09-19T09:25:29.300908+0000","last_unstale":"2024-09-19T09:25:42.989844+0000","last_undegraded":"2024-09-19T09:25:42.989844+0000","last_fullsized":"2024-09-19T09:25:42.989844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:44.774127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994395+0000","last_change":"2024-09-19T09:25:42.994395+0000","last_active":"2024-09-19T09:25:42.994395+0000","last_peered":"2024-09-19T09:25:42.994395+0000","last_clean":"2024-09-19T09:25:42.994395+0000","last_became_active":"2024-09-19T09:25:42.993992+0000","last_became_peered":"2024-09-19T09:25:42.993992+0000","last_unstale":"2024-09-19T09:25:42.994395+0000","last_undegraded":"2024-09-19T09:25:42.994395+0000","last_fullsized":"2024-09-19T09:25:42.994395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.486069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989979+0000","last_change":"2024-09-19T09:25:42.989979+0000","last_active":"2024-09-19T09:25:42.989979+0000","last_peered":"2024-09-19T09:25:42.989979+0000","last_clean":"2024-09-19T09:25:42.989979+0000","last_became_active":"2024-09-19T09:25:42.989578+0000","last_became_peered":"2024-09-19T09:25:42.989578+0000","last_unstale":"2024-09-19T09:25:42.989979+0000","last_undegraded":"2024-09-19T09:25:42.989979+0000","last_fullsized":"2024-09-19T09:25:42.989979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:04.583023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992064+0000","last_change":"2024-09-19T09:25:29.299917+0000","last_active":"2024-09-19T09:25:42.992064+0000","last_peered":"2024-09-19T09:25:42.992064+0000","last_clean":"2024-09-19T09:25:42.992064+0000","last_became_active":"2024-09-19T09:25:29.299571+0000","last_became_peered":"2024-09-19T09:25:29.299571+0000","last_unstale":"2024-09-19T09:25:42.992064+0000","last_undegraded":"2024-09-19T09:25:42.992064+0000","last_fullsized":"2024-09-19T09:25:42.992064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:25.142252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991982+0000","last_change":"2024-09-19T09:25:29.299713+0000","last_active":"2024-09-19T09:25:42.991982+0000","last_peered":"2024-09-19T09:25:42.991982+0000","last_clean":"2024-09-19T09:25:42.991982+0000","last_became_active":"2024-09-19T09:25:29.299234+0000","last_became_peered":"2024-09-19T09:25:29.299234+0000","last_unstale":"2024-09-19T09:25:42.991982+0000","last_undegraded":"2024-09-19T09:25:42.991982+0000","last_fullsized":"2024-09-19T09:25:42.991982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:34.714714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003535+0000","last_change":"2024-09-19T09:25:29.293010+0000","last_active":"2024-09-19T09:25:43.003535+0000","last_peered":"2024-09-19T09:25:43.003535+0000","last_clean":"2024-09-19T09:25:43.003535+0000","last_became_active":"2024-09-19T09:25:29.292645+0000","last_became_peered":"2024-09-19T09:25:29.292645+0000","last_unstale":"2024-09-19T09:25:43.003535+0000","last_undegraded":"2024-09-19T09:25:43.003535+0000","last_fullsized":"2024-09-19T09:25:43.003535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:24.749075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989031+0000","last_change":"2024-09-19T09:25:29.294530+0000","last_active":"2024-09-19T09:25:42.989031+0000","last_peered":"2024-09-19T09:25:42.989031+0000","last_clean":"2024-09-19T09:25:42.989031+0000","last_became_active":"2024-09-19T09:25:29.293699+0000","last_became_peered":"2024-09-19T09:25:29.293699+0000","last_unstale":"2024-09-19T09:25:42.989031+0000","last_undegraded":"2024-09-19T09:25:42.989031+0000","last_fullsized":"2024-09-19T09:25:42.989031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:53.491258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988781+0000","last_change":"2024-09-19T09:25:29.290600+0000","last_active":"2024-09-19T09:25:42.988781+0000","last_peered":"2024-09-19T09:25:42.988781+0000","last_clean":"2024-09-19T09:25:42.988781+0000","last_became_active":"2024-09-19T09:25:29.290305+0000","last_became_peered":"2024-09-19T09:25:29.290305+0000","last_unstale":"2024-09-19T09:25:42.988781+0000","last_undegraded":"2024-09-19T09:25:42.988781+0000","last_fullsized":"2024-09-19T09:25:42.988781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:42.809659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.993154+0000","last_change":"2024-09-19T09:25:29.300591+0000","last_active":"2024-09-19T09:25:42.993154+0000","last_peered":"2024-09-19T09:25:42.993154+0000","last_clean":"2024-09-19T09:25:42.993154+0000","last_became_active":"2024-09-19T09:25:29.300251+0000","last_became_peered":"2024-09-19T09:25:29.300251+0000","last_unstale":"2024-09-19T09:25:42.993154+0000","last_undegraded":"2024-09-19T09:25:42.993154+0000","last_fullsized":"2024-09-19T09:25:42.993154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:08.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988507+0000","last_change":"2024-09-19T09:25:29.298452+0000","last_active":"2024-09-19T09:25:42.988507+0000","last_peered":"2024-09-19T09:25:42.988507+0000","last_clean":"2024-09-19T09:25:42.988507+0000","last_became_active":"2024-09-19T09:25:29.298137+0000","last_became_peered":"2024-09-19T09:25:29.298137+0000","last_unstale":"2024-09-19T09:25:42.988507+0000","last_undegraded":"2024-09-19T09:25:42.988507+0000","last_fullsized":"2024-09-19T09:25:42.988507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:11.716228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985825+0000","last_change":"2024-09-19T09:25:29.294278+0000","last_active":"2024-09-19T09:25:42.985825+0000","last_peered":"2024-09-19T09:25:42.985825+0000","last_clean":"2024-09-19T09:25:42.985825+0000","last_became_active":"2024-09-19T09:25:29.293972+0000","last_became_peered":"2024-09-19T09:25:29.293972+0000","last_unstale":"2024-09-19T09:25:42.985825+0000","last_undegraded":"2024-09-19T09:25:42.985825+0000","last_fullsized":"2024-09-19T09:25:42.985825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:27.923839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003737+0000","last_change":"2024-09-19T09:25:29.294000+0000","last_active":"2024-09-19T09:25:43.003737+0000","last_peered":"2024-09-19T09:25:43.003737+0000","last_clean":"2024-09-19T09:25:43.003737+0000","last_became_active":"2024-09-19T09:25:29.293486+0000","last_became_peered":"2024-09-19T09:25:29.293486+0000","last_unstale":"2024-09-19T09:25:43.003737+0000","last_undegraded":"2024-09-19T09:25:43.003737+0000","last_fullsized":"2024-09-19T09:25:43.003737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:44.053620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994951+0000","last_change":"2024-09-19T09:25:42.994951+0000","last_active":"2024-09-19T09:25:42.994951+0000","last_peered":"2024-09-19T09:25:42.994951+0000","last_clean":"2024-09-19T09:25:42.994951+0000","last_became_active":"2024-09-19T09:25:42.994588+0000","last_became_peered":"2024-09-19T09:25:42.994588+0000","last_unstale":"2024-09-19T09:25:42.994951+0000","last_undegraded":"2024-09-19T09:25:42.994951+0000","last_fullsized":"2024-09-19T09:25:42.994951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:10.141169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995128+0000","last_change":"2024-09-19T09:25:29.286395+0000","last_active":"2024-09-19T09:25:42.995128+0000","last_peered":"2024-09-19T09:25:42.995128+0000","last_clean":"2024-09-19T09:25:42.995128+0000","last_became_active":"2024-09-19T09:25:29.286048+0000","last_became_peered":"2024-09-19T09:25:29.286048+0000","last_unstale":"2024-09-19T09:25:42.995128+0000","last_undegraded":"2024-09-19T09:25:42.995128+0000","last_fullsized":"2024-09-19T09:25:42.995128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:43.121166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991295+0000","last_change":"2024-09-19T09:25:29.298458+0000","last_active":"2024-09-19T09:25:42.991295+0000","last_peered":"2024-09-19T09:25:42.991295+0000","last_clean":"2024-09-19T09:25:42.991295+0000","last_became_active":"2024-09-19T09:25:29.297993+0000","last_became_peered":"2024-09-19T09:25:29.297993+0000","last_unstale":"2024-09-19T09:25:42.991295+0000","last_undegraded":"2024-09-19T09:25:42.991295+0000","last_fullsized":"2024-09-19T09:25:42.991295+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:44.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992762+0000","last_change":"2024-09-19T09:25:29.297167+0000","last_active":"2024-09-19T09:25:42.992762+0000","last_peered":"2024-09-19T09:25:42.992762+0000","last_clean":"2024-09-19T09:25:42.992762+0000","last_became_active":"2024-09-19T09:25:29.296777+0000","last_became_peered":"2024-09-19T09:25:29.296777+0000","last_unstale":"2024-09-19T09:25:42.992762+0000","last_undegraded":"2024-09-19T09:25:42.992762+0000","last_fullsized":"2024-09-19T09:25:42.992762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:40.581643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988833+0000","last_change":"2024-09-19T09:25:29.296169+0000","last_active":"2024-09-19T09:25:42.988833+0000","last_peered":"2024-09-19T09:25:42.988833+0000","last_clean":"2024-09-19T09:25:42.988833+0000","last_became_active":"2024-09-19T09:25:29.295806+0000","last_became_peered":"2024-09-19T09:25:29.295806+0000","last_unstale":"2024-09-19T09:25:42.988833+0000","last_undegraded":"2024-09-19T09:25:42.988833+0000","last_fullsized":"2024-09-19T09:25:42.988833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:39.639884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985866+0000","last_change":"2024-09-19T09:25:29.297960+0000","last_active":"2024-09-19T09:25:42.985866+0000","last_peered":"2024-09-19T09:25:42.985866+0000","last_clean":"2024-09-19T09:25:42.985866+0000","last_became_active":"2024-09-19T09:25:29.297577+0000","last_became_peered":"2024-09-19T09:25:29.297577+0000","last_unstale":"2024-09-19T09:25:42.985866+0000","last_undegraded":"2024-09-19T09:25:42.985866+0000","last_fullsized":"2024-09-19T09:25:42.985866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:27.035504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992420+0000","last_change":"2024-09-19T09:25:29.297896+0000","last_active":"2024-09-19T09:25:42.992420+0000","last_peered":"2024-09-19T09:25:42.992420+0000","last_clean":"2024-09-19T09:25:42.992420+0000","last_became_active":"2024-09-19T09:25:29.297594+0000","last_became_peered":"2024-09-19T09:25:29.297594+0000","last_unstale":"2024-09-19T09:25:42.992420+0000","last_undegraded":"2024-09-19T09:25:42.992420+0000","last_fullsized":"2024-09-19T09:25:42.992420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.366325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992205+0000","last_change":"2024-09-19T09:25:29.301851+0000","last_active":"2024-09-19T09:25:42.992205+0000","last_peered":"2024-09-19T09:25:42.992205+0000","last_clean":"2024-09-19T09:25:42.992205+0000","last_became_active":"2024-09-19T09:25:29.299282+0000","last_became_peered":"2024-09-19T09:25:29.299282+0000","last_unstale":"2024-09-19T09:25:42.992205+0000","last_undegraded":"2024-09-19T09:25:42.992205+0000","last_fullsized":"2024-09-19T09:25:42.992205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:00.382114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992658+0000","last_change":"2024-09-19T09:25:29.303316+0000","last_active":"2024-09-19T09:25:42.992658+0000","last_peered":"2024-09-19T09:25:42.992658+0000","last_clean":"2024-09-19T09:25:42.992658+0000","last_became_active":"2024-09-19T09:25:29.302994+0000","last_became_peered":"2024-09-19T09:25:29.302994+0000","last_unstale":"2024-09-19T09:25:42.992658+0000","last_undegraded":"2024-09-19T09:25:42.992658+0000","last_fullsized":"2024-09-19T09:25:42.992658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:46.233991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988501+0000","last_change":"2024-09-19T09:25:29.293004+0000","last_active":"2024-09-19T09:25:42.988501+0000","last_peered":"2024-09-19T09:25:42.988501+0000","last_clean":"2024-09-19T09:25:42.988501+0000","last_became_active":"2024-09-19T09:25:29.292657+0000","last_became_peered":"2024-09-19T09:25:29.292657+0000","last_unstale":"2024-09-19T09:25:42.988501+0000","last_undegraded":"2024-09-19T09:25:42.988501+0000","last_fullsized":"2024-09-19T09:25:42.988501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:50.966477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991936+0000","last_change":"2024-09-19T09:25:29.300434+0000","last_active":"2024-09-19T09:25:42.991936+0000","last_peered":"2024-09-19T09:25:42.991936+0000","last_clean":"2024-09-19T09:25:42.991936+0000","last_became_active":"2024-09-19T09:25:29.300053+0000","last_became_peered":"2024-09-19T09:25:29.300053+0000","last_unstale":"2024-09-19T09:25:42.991936+0000","last_undegraded":"2024-09-19T09:25:42.991936+0000","last_fullsized":"2024-09-19T09:25:42.991936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:13.283298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":81,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:27.031370+0000","last_change":"2024-09-19T09:25:27.280792+0000","last_active":"2024-09-19T09:26:27.031370+0000","last_peered":"2024-09-19T09:26:27.031370+0000","last_clean":"2024-09-19T09:26:27.031370+0000","last_became_active":"2024-09-19T09:25:27.280443+0000","last_became_peered":"2024-09-19T09:25:27.280443+0000","last_unstale":"2024-09-19T09:26:27.031370+0000","last_undegraded":"2024-09-19T09:26:27.031370+0000","last_fullsized":"2024-09-19T09:26:27.031370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.599512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.997114+0000","last_change":"2024-09-19T09:25:29.298975+0000","last_active":"2024-09-19T09:25:42.997114+0000","last_peered":"2024-09-19T09:25:42.997114+0000","last_clean":"2024-09-19T09:25:42.997114+0000","last_became_active":"2024-09-19T09:25:29.298476+0000","last_became_peered":"2024-09-19T09:25:29.298476+0000","last_unstale":"2024-09-19T09:25:42.997114+0000","last_undegraded":"2024-09-19T09:25:42.997114+0000","last_fullsized":"2024-09-19T09:25:42.997114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:09.658894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986623+0000","last_change":"2024-09-19T09:25:27.278257+0000","last_active":"2024-09-19T09:25:42.986623+0000","last_peered":"2024-09-19T09:25:42.986623+0000","last_clean":"2024-09-19T09:25:42.986623+0000","last_became_active":"2024-09-19T09:25:27.277897+0000","last_became_peered":"2024-09-19T09:25:27.277897+0000","last_unstale":"2024-09-19T09:25:42.986623+0000","last_undegraded":"2024-09-19T09:25:42.986623+0000","last_fullsized":"2024-09-19T09:25:42.986623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:47.550314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985617+0000","last_change":"2024-09-19T09:25:27.272215+0000","last_active":"2024-09-19T09:25:42.985617+0000","last_peered":"2024-09-19T09:25:42.985617+0000","last_clean":"2024-09-19T09:25:42.985617+0000","last_became_active":"2024-09-19T09:25:27.271821+0000","last_became_peered":"2024-09-19T09:25:27.271821+0000","last_unstale":"2024-09-19T09:25:42.985617+0000","last_undegraded":"2024-09-19T09:25:42.985617+0000","last_fullsized":"2024-09-19T09:25:42.985617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:30.615730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990922+0000","last_change":"2024-09-19T09:25:29.289347+0000","last_active":"2024-09-19T09:25:42.990922+0000","last_peered":"2024-09-19T09:25:42.990922+0000","last_clean":"2024-09-19T09:25:42.990922+0000","last_became_active":"2024-09-19T09:25:29.289015+0000","last_became_peered":"2024-09-19T09:25:29.289015+0000","last_unstale":"2024-09-19T09:25:42.990922+0000","last_undegraded":"2024-09-19T09:25:42.990922+0000","last_fullsized":"2024-09-19T09:25:42.990922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:18.419334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995432+0000","last_change":"2024-09-19T09:25:27.277342+0000","last_active":"2024-09-19T09:25:42.995432+0000","last_peered":"2024-09-19T09:25:42.995432+0000","last_clean":"2024-09-19T09:25:42.995432+0000","last_became_active":"2024-09-19T09:25:27.277016+0000","last_became_peered":"2024-09-19T09:25:27.277016+0000","last_unstale":"2024-09-19T09:25:42.995432+0000","last_undegraded":"2024-09-19T09:25:42.995432+0000","last_fullsized":"2024-09-19T09:25:42.995432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:56.736156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988740+0000","last_change":"2024-09-19T09:25:29.290924+0000","last_active":"2024-09-19T09:25:42.988740+0000","last_peered":"2024-09-19T09:25:42.988740+0000","last_clean":"2024-09-19T09:25:42.988740+0000","last_became_active":"2024-09-19T09:25:29.290482+0000","last_became_peered":"2024-09-19T09:25:29.290482+0000","last_unstale":"2024-09-19T09:25:42.988740+0000","last_undegraded":"2024-09-19T09:25:42.988740+0000","last_fullsized":"2024-09-19T09:25:42.988740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:16.265040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":120,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:26.029934+0000","last_change":"2024-09-19T09:25:27.277604+0000","last_active":"2024-09-19T09:26:26.029934+0000","last_peered":"2024-09-19T09:26:26.029934+0000","last_clean":"2024-09-19T09:26:26.029934+0000","last_became_active":"2024-09-19T09:25:27.277271+0000","last_became_peered":"2024-09-19T09:25:27.277271+0000","last_unstale":"2024-09-19T09:26:26.029934+0000","last_undegraded":"2024-09-19T09:26:26.029934+0000","last_fullsized":"2024-09-19T09:26:26.029934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:32.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.004044+0000","last_change":"2024-09-19T09:25:29.290537+0000","last_active":"2024-09-19T09:25:43.004044+0000","last_peered":"2024-09-19T09:25:43.004044+0000","last_clean":"2024-09-19T09:25:43.004044+0000","last_became_active":"2024-09-19T09:25:29.287984+0000","last_became_peered":"2024-09-19T09:25:29.287984+0000","last_unstale":"2024-09-19T09:25:43.004044+0000","last_undegraded":"2024-09-19T09:25:43.004044+0000","last_fullsized":"2024-09-19T09:25:43.004044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:26.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990018+0000","last_change":"2024-09-19T09:25:27.278560+0000","last_active":"2024-09-19T09:25:42.990018+0000","last_peered":"2024-09-19T09:25:42.990018+0000","last_clean":"2024-09-19T09:25:42.990018+0000","last_became_active":"2024-09-19T09:25:27.277950+0000","last_became_peered":"2024-09-19T09:25:27.277950+0000","last_unstale":"2024-09-19T09:25:42.990018+0000","last_undegraded":"2024-09-19T09:25:42.990018+0000","last_fullsized":"2024-09-19T09:25:42.990018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:28.387725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992413+0000","last_change":"2024-09-19T09:25:29.298897+0000","last_active":"2024-09-19T09:25:42.992413+0000","last_peered":"2024-09-19T09:25:42.992413+0000","last_clean":"2024-09-19T09:25:42.992413+0000","last_became_active":"2024-09-19T09:25:29.298617+0000","last_became_peered":"2024-09-19T09:25:29.298617+0000","last_unstale":"2024-09-19T09:25:42.992413+0000","last_undegraded":"2024-09-19T09:25:42.992413+0000","last_fullsized":"2024-09-19T09:25:42.992413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:53.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994870+0000","last_change":"2024-09-19T09:25:27.324871+0000","last_active":"2024-09-19T09:25:42.994870+0000","last_peered":"2024-09-19T09:25:42.994870+0000","last_clean":"2024-09-19T09:25:42.994870+0000","last_became_active":"2024-09-19T09:25:27.324471+0000","last_became_peered":"2024-09-19T09:25:27.324471+0000","last_unstale":"2024-09-19T09:25:42.994870+0000","last_undegraded":"2024-09-19T09:25:42.994870+0000","last_fullsized":"2024-09-19T09:25:42.994870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:35.936413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.004242+0000","last_change":"2024-09-19T09:25:29.298332+0000","last_active":"2024-09-19T09:25:43.004242+0000","last_peered":"2024-09-19T09:25:43.004242+0000","last_clean":"2024-09-19T09:25:43.004242+0000","last_became_active":"2024-09-19T09:25:29.298006+0000","last_became_peered":"2024-09-19T09:25:29.298006+0000","last_unstale":"2024-09-19T09:25:43.004242+0000","last_undegraded":"2024-09-19T09:25:43.004242+0000","last_fullsized":"2024-09-19T09:25:43.004242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:02.621821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":157,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:26.038716+0000","last_change":"2024-09-19T09:25:27.279528+0000","last_active":"2024-09-19T09:26:26.038716+0000","last_peered":"2024-09-19T09:26:26.038716+0000","last_clean":"2024-09-19T09:26:26.038716+0000","last_became_active":"2024-09-19T09:25:27.279149+0000","last_became_peered":"2024-09-19T09:25:27.279149+0000","last_unstale":"2024-09-19T09:26:26.038716+0000","last_undegraded":"2024-09-19T09:26:26.038716+0000","last_fullsized":"2024-09-19T09:26:26.038716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:47.030736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":110,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991057+0000","last_change":"2024-09-19T09:25:29.297538+0000","last_active":"2024-09-19T09:25:42.991057+0000","last_peered":"2024-09-19T09:25:42.991057+0000","last_clean":"2024-09-19T09:25:42.991057+0000","last_became_active":"2024-09-19T09:25:29.297181+0000","last_became_peered":"2024-09-19T09:25:29.297181+0000","last_unstale":"2024-09-19T09:25:42.991057+0000","last_undegraded":"2024-09-19T09:25:42.991057+0000","last_fullsized":"2024-09-19T09:25:42.991057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:53.135438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992860+0000","last_change":"2024-09-19T09:25:27.277090+0000","last_active":"2024-09-19T09:25:42.992860+0000","last_peered":"2024-09-19T09:25:42.992860+0000","last_clean":"2024-09-19T09:25:42.992860+0000","last_became_active":"2024-09-19T09:25:27.276718+0000","last_became_peered":"2024-09-19T09:25:27.276718+0000","last_unstale":"2024-09-19T09:25:42.992860+0000","last_undegraded":"2024-09-19T09:25:42.992860+0000","last_fullsized":"2024-09-19T09:25:42.992860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:25.664261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985642+0000","last_change":"2024-09-19T09:25:29.294393+0000","last_active":"2024-09-19T09:25:42.985642+0000","last_peered":"2024-09-19T09:25:42.985642+0000","last_clean":"2024-09-19T09:25:42.985642+0000","last_became_active":"2024-09-19T09:25:29.294099+0000","last_became_peered":"2024-09-19T09:25:29.294099+0000","last_unstale":"2024-09-19T09:25:42.985642+0000","last_undegraded":"2024-09-19T09:25:42.985642+0000","last_fullsized":"2024-09-19T09:25:42.985642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:17.333736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995398+0000","last_change":"2024-09-19T09:25:27.278593+0000","last_active":"2024-09-19T09:25:42.995398+0000","last_peered":"2024-09-19T09:25:42.995398+0000","last_clean":"2024-09-19T09:25:42.995398+0000","last_became_active":"2024-09-19T09:25:27.278232+0000","last_became_peered":"2024-09-19T09:25:27.278232+0000","last_unstale":"2024-09-19T09:25:42.995398+0000","last_undegraded":"2024-09-19T09:25:42.995398+0000","last_fullsized":"2024-09-19T09:25:42.995398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:05.852671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989855+0000","last_change":"2024-09-19T09:25:29.300024+0000","last_active":"2024-09-19T09:25:42.989855+0000","last_peered":"2024-09-19T09:25:42.989855+0000","last_clean":"2024-09-19T09:25:42.989855+0000","last_became_active":"2024-09-19T09:25:29.299683+0000","last_became_peered":"2024-09-19T09:25:29.299683+0000","last_unstale":"2024-09-19T09:25:42.989855+0000","last_undegraded":"2024-09-19T09:25:42.989855+0000","last_fullsized":"2024-09-19T09:25:42.989855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:42.249399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":80,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:26.031594+0000","last_change":"2024-09-19T09:25:27.272739+0000","last_active":"2024-09-19T09:26:26.031594+0000","last_peered":"2024-09-19T09:26:26.031594+0000","last_clean":"2024-09-19T09:26:26.031594+0000","last_became_active":"2024-09-19T09:25:27.272354+0000","last_became_peered":"2024-09-19T09:25:27.272354+0000","last_unstale":"2024-09-19T09:26:26.031594+0000","last_undegraded":"2024-09-19T09:26:26.031594+0000","last_fullsized":"2024-09-19T09:26:26.031594+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:27.863840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989616+0000","last_change":"2024-09-19T09:25:29.303460+0000","last_active":"2024-09-19T09:25:42.989616+0000","last_peered":"2024-09-19T09:25:42.989616+0000","last_clean":"2024-09-19T09:25:42.989616+0000","last_became_active":"2024-09-19T09:25:29.303127+0000","last_became_peered":"2024-09-19T09:25:29.303127+0000","last_unstale":"2024-09-19T09:25:42.989616+0000","last_undegraded":"2024-09-19T09:25:42.989616+0000","last_fullsized":"2024-09-19T09:25:42.989616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:35.313924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990876+0000","last_change":"2024-09-19T09:25:27.323238+0000","last_active":"2024-09-19T09:25:42.990876+0000","last_peered":"2024-09-19T09:25:42.990876+0000","last_clean":"2024-09-19T09:25:42.990876+0000","last_became_active":"2024-09-19T09:25:27.322868+0000","last_became_peered":"2024-09-19T09:25:27.322868+0000","last_unstale":"2024-09-19T09:25:42.990876+0000","last_undegraded":"2024-09-19T09:25:42.990876+0000","last_fullsized":"2024-09-19T09:25:42.990876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:22.890280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.005569+0000","last_change":"2024-09-19T09:25:43.005569+0000","last_active":"2024-09-19T09:25:43.005569+0000","last_peered":"2024-09-19T09:25:43.005569+0000","last_clean":"2024-09-19T09:25:43.005569+0000","last_became_active":"2024-09-19T09:25:43.005143+0000","last_became_peered":"2024-09-19T09:25:43.005143+0000","last_unstale":"2024-09-19T09:25:43.005569+0000","last_undegraded":"2024-09-19T09:25:43.005569+0000","last_fullsized":"2024-09-19T09:25:43.005569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:26.297013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991438+0000","last_change":"2024-09-19T09:25:27.282737+0000","last_active":"2024-09-19T09:25:42.991438+0000","last_peered":"2024-09-19T09:25:42.991438+0000","last_clean":"2024-09-19T09:25:42.991438+0000","last_became_active":"2024-09-19T09:25:27.281785+0000","last_became_peered":"2024-09-19T09:25:27.281785+0000","last_unstale":"2024-09-19T09:25:42.991438+0000","last_undegraded":"2024-09-19T09:25:42.991438+0000","last_fullsized":"2024-09-19T09:25:42.991438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:41.552206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988791+0000","last_change":"2024-09-19T09:25:29.300168+0000","last_active":"2024-09-19T09:25:42.988791+0000","last_peered":"2024-09-19T09:25:42.988791+0000","last_clean":"2024-09-19T09:25:42.988791+0000","last_became_active":"2024-09-19T09:25:29.299811+0000","last_became_peered":"2024-09-19T09:25:29.299811+0000","last_unstale":"2024-09-19T09:25:42.988791+0000","last_undegraded":"2024-09-19T09:25:42.988791+0000","last_fullsized":"2024-09-19T09:25:42.988791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:06.691259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985783+0000","last_change":"2024-09-19T09:25:27.278433+0000","last_active":"2024-09-19T09:25:42.985783+0000","last_peered":"2024-09-19T09:25:42.985783+0000","last_clean":"2024-09-19T09:25:42.985783+0000","last_became_active":"2024-09-19T09:25:27.278076+0000","last_became_peered":"2024-09-19T09:25:27.278076+0000","last_unstale":"2024-09-19T09:25:42.985783+0000","last_undegraded":"2024-09-19T09:25:42.985783+0000","last_fullsized":"2024-09-19T09:25:42.985783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:53.799265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990338+0000","last_change":"2024-09-19T09:25:29.295650+0000","last_active":"2024-09-19T09:25:42.990338+0000","last_peered":"2024-09-19T09:25:42.990338+0000","last_clean":"2024-09-19T09:25:42.990338+0000","last_became_active":"2024-09-19T09:25:29.295277+0000","last_became_peered":"2024-09-19T09:25:29.295277+0000","last_unstale":"2024-09-19T09:25:42.990338+0000","last_undegraded":"2024-09-19T09:25:42.990338+0000","last_fullsized":"2024-09-19T09:25:42.990338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:57.690699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996123+0000","last_change":"2024-09-19T09:25:27.279239+0000","last_active":"2024-09-19T09:25:42.996123+0000","last_peered":"2024-09-19T09:25:42.996123+0000","last_clean":"2024-09-19T09:25:42.996123+0000","last_became_active":"2024-09-19T09:25:27.278868+0000","last_became_peered":"2024-09-19T09:25:27.278868+0000","last_unstale":"2024-09-19T09:25:42.996123+0000","last_undegraded":"2024-09-19T09:25:42.996123+0000","last_fullsized":"2024-09-19T09:25:42.996123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:41.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990429+0000","last_change":"2024-09-19T09:25:29.300833+0000","last_active":"2024-09-19T09:25:42.990429+0000","last_peered":"2024-09-19T09:25:42.990429+0000","last_clean":"2024-09-19T09:25:42.990429+0000","last_became_active":"2024-09-19T09:25:29.300472+0000","last_became_peered":"2024-09-19T09:25:29.300472+0000","last_unstale":"2024-09-19T09:25:42.990429+0000","last_undegraded":"2024-09-19T09:25:42.990429+0000","last_fullsized":"2024-09-19T09:25:42.990429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:50.591276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:26.070111+0000","last_change":"2024-09-19T09:25:43.001722+0000","last_active":"2024-09-19T09:26:26.070111+0000","last_peered":"2024-09-19T09:26:26.070111+0000","last_clean":"2024-09-19T09:26:26.070111+0000","last_became_active":"2024-09-19T09:25:42.992796+0000","last_became_peered":"2024-09-19T09:25:42.992796+0000","last_unstale":"2024-09-19T09:26:26.070111+0000","last_undegraded":"2024-09-19T09:26:26.070111+0000","last_fullsized":"2024-09-19T09:26:26.070111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:59.639010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990231+0000","last_change":"2024-09-19T09:25:29.295906+0000","last_active":"2024-09-19T09:25:42.990231+0000","last_peered":"2024-09-19T09:25:42.990231+0000","last_clean":"2024-09-19T09:25:42.990231+0000","last_became_active":"2024-09-19T09:25:29.295565+0000","last_became_peered":"2024-09-19T09:25:29.295565+0000","last_unstale":"2024-09-19T09:25:42.990231+0000","last_undegraded":"2024-09-19T09:25:42.990231+0000","last_fullsized":"2024-09-19T09:25:42.990231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:24.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":334,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989806+0000","last_change":"2024-09-19T09:24:19.256300+0000","last_active":"2024-09-19T09:25:42.989806+0000","last_peered":"2024-09-19T09:25:42.989806+0000","last_clean":"2024-09-19T09:25:42.989806+0000","last_became_active":"2024-09-19T09:24:19.255833+0000","last_became_peered":"2024-09-19T09:24:19.255833+0000","last_unstale":"2024-09-19T09:25:42.989806+0000","last_undegraded":"2024-09-19T09:25:42.989806+0000","last_fullsized":"2024-09-19T09:25:42.989806+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_clean_scrub_stamp":"2024-09-19T09:20:57.798838+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:14:33.343748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989637+0000","last_change":"2024-09-19T09:25:27.272090+0000","last_active":"2024-09-19T09:25:42.989637+0000","last_peered":"2024-09-19T09:25:42.989637+0000","last_clean":"2024-09-19T09:25:42.989637+0000","last_became_active":"2024-09-19T09:25:27.271651+0000","last_became_peered":"2024-09-19T09:25:27.271651+0000","last_unstale":"2024-09-19T09:25:42.989637+0000","last_undegraded":"2024-09-19T09:25:42.989637+0000","last_fullsized":"2024-09-19T09:25:42.989637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:45.847082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:29.300718+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:29.299460+0000","last_became_peered":"2024-09-19T09:25:29.299460+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:43.063909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003299+0000","last_change":"2024-09-19T09:25:43.003299+0000","last_active":"2024-09-19T09:25:43.003299+0000","last_peered":"2024-09-19T09:25:43.003299+0000","last_clean":"2024-09-19T09:25:43.003299+0000","last_became_active":"2024-09-19T09:25:42.999686+0000","last_became_peered":"2024-09-19T09:25:42.999686+0000","last_unstale":"2024-09-19T09:25:43.003299+0000","last_undegraded":"2024-09-19T09:25:43.003299+0000","last_fullsized":"2024-09-19T09:25:43.003299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:05.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985606+0000","last_change":"2024-09-19T09:25:29.297833+0000","last_active":"2024-09-19T09:25:42.985606+0000","last_peered":"2024-09-19T09:25:42.985606+0000","last_clean":"2024-09-19T09:25:42.985606+0000","last_became_active":"2024-09-19T09:25:29.297534+0000","last_became_peered":"2024-09-19T09:25:29.297534+0000","last_unstale":"2024-09-19T09:25:42.985606+0000","last_undegraded":"2024-09-19T09:25:42.985606+0000","last_fullsized":"2024-09-19T09:25:42.985606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:47.156253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":78,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:27.032323+0000","last_change":"2024-09-19T09:25:27.322759+0000","last_active":"2024-09-19T09:26:27.032323+0000","last_peered":"2024-09-19T09:26:27.032323+0000","last_clean":"2024-09-19T09:26:27.032323+0000","last_became_active":"2024-09-19T09:25:27.322402+0000","last_became_peered":"2024-09-19T09:25:27.322402+0000","last_unstale":"2024-09-19T09:26:27.032323+0000","last_undegraded":"2024-09-19T09:26:27.032323+0000","last_fullsized":"2024-09-19T09:26:27.032323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:06.276957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.999834+0000","last_change":"2024-09-19T09:25:42.999834+0000","last_active":"2024-09-19T09:25:42.999834+0000","last_peered":"2024-09-19T09:25:42.999834+0000","last_clean":"2024-09-19T09:25:42.999834+0000","last_became_active":"2024-09-19T09:25:42.999450+0000","last_became_peered":"2024-09-19T09:25:42.999450+0000","last_unstale":"2024-09-19T09:25:42.999834+0000","last_undegraded":"2024-09-19T09:25:42.999834+0000","last_fullsized":"2024-09-19T09:25:42.999834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:00.509844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":77,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:26.053065+0000","last_change":"2024-09-19T09:25:27.275212+0000","last_active":"2024-09-19T09:26:26.053065+0000","last_peered":"2024-09-19T09:26:26.053065+0000","last_clean":"2024-09-19T09:26:26.053065+0000","last_became_active":"2024-09-19T09:25:27.274802+0000","last_became_peered":"2024-09-19T09:25:27.274802+0000","last_unstale":"2024-09-19T09:26:26.053065+0000","last_undegraded":"2024-09-19T09:26:26.053065+0000","last_fullsized":"2024-09-19T09:26:26.053065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:48.392026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986730+0000","last_change":"2024-09-19T09:25:29.304747+0000","last_active":"2024-09-19T09:25:42.986730+0000","last_peered":"2024-09-19T09:25:42.986730+0000","last_clean":"2024-09-19T09:25:42.986730+0000","last_became_active":"2024-09-19T09:25:29.304386+0000","last_became_peered":"2024-09-19T09:25:29.304386+0000","last_unstale":"2024-09-19T09:25:42.986730+0000","last_undegraded":"2024-09-19T09:25:42.986730+0000","last_fullsized":"2024-09-19T09:25:42.986730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.788697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988854+0000","last_change":"2024-09-19T09:25:27.323083+0000","last_active":"2024-09-19T09:25:42.988854+0000","last_peered":"2024-09-19T09:25:42.988854+0000","last_clean":"2024-09-19T09:25:42.988854+0000","last_became_active":"2024-09-19T09:25:27.322707+0000","last_became_peered":"2024-09-19T09:25:27.322707+0000","last_unstale":"2024-09-19T09:25:42.988854+0000","last_undegraded":"2024-09-19T09:25:42.988854+0000","last_fullsized":"2024-09-19T09:25:42.988854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:24.415116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986687+0000","last_change":"2024-09-19T09:25:29.292472+0000","last_active":"2024-09-19T09:25:42.986687+0000","last_peered":"2024-09-19T09:25:42.986687+0000","last_clean":"2024-09-19T09:25:42.986687+0000","last_became_active":"2024-09-19T09:25:29.292079+0000","last_became_peered":"2024-09-19T09:25:29.292079+0000","last_unstale":"2024-09-19T09:25:42.986687+0000","last_undegraded":"2024-09-19T09:25:42.986687+0000","last_fullsized":"2024-09-19T09:25:42.986687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:35.801594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991224+0000","last_change":"2024-09-19T09:25:27.324839+0000","last_active":"2024-09-19T09:25:42.991224+0000","last_peered":"2024-09-19T09:25:42.991224+0000","last_clean":"2024-09-19T09:25:42.991224+0000","last_became_active":"2024-09-19T09:25:27.324475+0000","last_became_peered":"2024-09-19T09:25:27.324475+0000","last_unstale":"2024-09-19T09:25:42.991224+0000","last_undegraded":"2024-09-19T09:25:42.991224+0000","last_fullsized":"2024-09-19T09:25:42.991224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:32.251753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985998+0000","last_change":"2024-09-19T09:25:29.293025+0000","last_active":"2024-09-19T09:25:42.985998+0000","last_peered":"2024-09-19T09:25:42.985998+0000","last_clean":"2024-09-19T09:25:42.985998+0000","last_became_active":"2024-09-19T09:25:29.292570+0000","last_became_peered":"2024-09-19T09:25:29.292570+0000","last_unstale":"2024-09-19T09:25:42.985998+0000","last_undegraded":"2024-09-19T09:25:42.985998+0000","last_fullsized":"2024-09-19T09:25:42.985998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:16.394524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:54.295611+0000","last_change":"2024-09-19T09:25:27.278100+0000","last_active":"2024-09-19T09:25:54.295611+0000","last_peered":"2024-09-19T09:25:54.295611+0000","last_clean":"2024-09-19T09:25:54.295611+0000","last_became_active":"2024-09-19T09:25:27.277769+0000","last_became_peered":"2024-09-19T09:25:27.277769+0000","last_unstale":"2024-09-19T09:25:54.295611+0000","last_undegraded":"2024-09-19T09:25:54.295611+0000","last_fullsized":"2024-09-19T09:25:54.295611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:31.885142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991325+0000","last_change":"2024-09-19T09:25:29.293651+0000","last_active":"2024-09-19T09:25:42.991325+0000","last_peered":"2024-09-19T09:25:42.991325+0000","last_clean":"2024-09-19T09:25:42.991325+0000","last_became_active":"2024-09-19T09:25:29.293317+0000","last_became_peered":"2024-09-19T09:25:29.293317+0000","last_unstale":"2024-09-19T09:25:42.991325+0000","last_undegraded":"2024-09-19T09:25:42.991325+0000","last_fullsized":"2024-09-19T09:25:42.991325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:40.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990030+0000","last_change":"2024-09-19T09:25:27.269367+0000","last_active":"2024-09-19T09:25:42.990030+0000","last_peered":"2024-09-19T09:25:42.990030+0000","last_clean":"2024-09-19T09:25:42.990030+0000","last_became_active":"2024-09-19T09:25:27.269011+0000","last_became_peered":"2024-09-19T09:25:27.269011+0000","last_unstale":"2024-09-19T09:25:42.990030+0000","last_undegraded":"2024-09-19T09:25:42.990030+0000","last_fullsized":"2024-09-19T09:25:42.990030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:00.930752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992257+0000","last_change":"2024-09-19T09:25:29.299134+0000","last_active":"2024-09-19T09:25:42.992257+0000","last_peered":"2024-09-19T09:25:42.992257+0000","last_clean":"2024-09-19T09:25:42.992257+0000","last_became_active":"2024-09-19T09:25:29.298648+0000","last_became_peered":"2024-09-19T09:25:29.298648+0000","last_unstale":"2024-09-19T09:25:42.992257+0000","last_undegraded":"2024-09-19T09:25:42.992257+0000","last_fullsized":"2024-09-19T09:25:42.992257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:39.627852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003686+0000","last_change":"2024-09-19T09:25:27.322861+0000","last_active":"2024-09-19T09:25:43.003686+0000","last_peered":"2024-09-19T09:25:43.003686+0000","last_clean":"2024-09-19T09:25:43.003686+0000","last_became_active":"2024-09-19T09:25:27.322560+0000","last_became_peered":"2024-09-19T09:25:27.322560+0000","last_unstale":"2024-09-19T09:25:43.003686+0000","last_undegraded":"2024-09-19T09:25:43.003686+0000","last_fullsized":"2024-09-19T09:25:43.003686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:35.993093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990868+0000","last_change":"2024-09-19T09:25:29.301680+0000","last_active":"2024-09-19T09:25:42.990868+0000","last_peered":"2024-09-19T09:25:42.990868+0000","last_clean":"2024-09-19T09:25:42.990868+0000","last_became_active":"2024-09-19T09:25:29.301331+0000","last_became_peered":"2024-09-19T09:25:29.301331+0000","last_unstale":"2024-09-19T09:25:42.990868+0000","last_undegraded":"2024-09-19T09:25:42.990868+0000","last_fullsized":"2024-09-19T09:25:42.990868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:57.364245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:27.272609+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:27.272169+0000","last_became_peered":"2024-09-19T09:25:27.272169+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.879474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988584+0000","last_change":"2024-09-19T09:25:29.296847+0000","last_active":"2024-09-19T09:25:42.988584+0000","last_peered":"2024-09-19T09:25:42.988584+0000","last_clean":"2024-09-19T09:25:42.988584+0000","last_became_active":"2024-09-19T09:25:29.296465+0000","last_became_peered":"2024-09-19T09:25:29.296465+0000","last_unstale":"2024-09-19T09:25:42.988584+0000","last_undegraded":"2024-09-19T09:25:42.988584+0000","last_fullsized":"2024-09-19T09:25:42.988584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:05:48.553252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.012954+0000","last_change":"2024-09-19T09:25:43.012532+0000","last_active":"2024-09-19T09:25:43.012954+0000","last_peered":"2024-09-19T09:25:43.012954+0000","last_clean":"2024-09-19T09:25:43.012954+0000","last_became_active":"2024-09-19T09:25:42.994996+0000","last_became_peered":"2024-09-19T09:25:42.994996+0000","last_unstale":"2024-09-19T09:25:43.012954+0000","last_undegraded":"2024-09-19T09:25:43.012954+0000","last_fullsized":"2024-09-19T09:25:43.012954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:25.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991072+0000","last_change":"2024-09-19T09:25:29.297633+0000","last_active":"2024-09-19T09:25:42.991072+0000","last_peered":"2024-09-19T09:25:42.991072+0000","last_clean":"2024-09-19T09:25:42.991072+0000","last_became_active":"2024-09-19T09:25:29.297284+0000","last_became_peered":"2024-09-19T09:25:29.297284+0000","last_unstale":"2024-09-19T09:25:42.991072+0000","last_undegraded":"2024-09-19T09:25:42.991072+0000","last_fullsized":"2024-09-19T09:25:42.991072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:50.582546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991172+0000","last_change":"2024-09-19T09:25:27.281778+0000","last_active":"2024-09-19T09:25:42.991172+0000","last_peered":"2024-09-19T09:25:42.991172+0000","last_clean":"2024-09-19T09:25:42.991172+0000","last_became_active":"2024-09-19T09:25:27.281280+0000","last_became_peered":"2024-09-19T09:25:27.281280+0000","last_unstale":"2024-09-19T09:25:42.991172+0000","last_undegraded":"2024-09-19T09:25:42.991172+0000","last_fullsized":"2024-09-19T09:25:42.991172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:46.811968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989808+0000","last_change":"2024-09-19T09:25:29.295663+0000","last_active":"2024-09-19T09:25:42.989808+0000","last_peered":"2024-09-19T09:25:42.989808+0000","last_clean":"2024-09-19T09:25:42.989808+0000","last_became_active":"2024-09-19T09:25:29.295146+0000","last_became_peered":"2024-09-19T09:25:29.295146+0000","last_unstale":"2024-09-19T09:25:42.989808+0000","last_undegraded":"2024-09-19T09:25:42.989808+0000","last_fullsized":"2024-09-19T09:25:42.989808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:36.749339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996010+0000","last_change":"2024-09-19T09:25:29.298004+0000","last_active":"2024-09-19T09:25:42.996010+0000","last_peered":"2024-09-19T09:25:42.996010+0000","last_clean":"2024-09-19T09:25:42.996010+0000","last_became_active":"2024-09-19T09:25:29.297600+0000","last_became_peered":"2024-09-19T09:25:29.297600+0000","last_unstale":"2024-09-19T09:25:42.996010+0000","last_undegraded":"2024-09-19T09:25:42.996010+0000","last_fullsized":"2024-09-19T09:25:42.996010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:09.738615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992323+0000","last_change":"2024-09-19T09:25:27.282876+0000","last_active":"2024-09-19T09:25:42.992323+0000","last_peered":"2024-09-19T09:25:42.992323+0000","last_clean":"2024-09-19T09:25:42.992323+0000","last_became_active":"2024-09-19T09:25:27.282201+0000","last_became_peered":"2024-09-19T09:25:27.282201+0000","last_unstale":"2024-09-19T09:25:42.992323+0000","last_undegraded":"2024-09-19T09:25:42.992323+0000","last_fullsized":"2024-09-19T09:25:42.992323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:08.508662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003650+0000","last_change":"2024-09-19T09:25:29.288379+0000","last_active":"2024-09-19T09:25:43.003650+0000","last_peered":"2024-09-19T09:25:43.003650+0000","last_clean":"2024-09-19T09:25:43.003650+0000","last_became_active":"2024-09-19T09:25:29.287917+0000","last_became_peered":"2024-09-19T09:25:29.287917+0000","last_unstale":"2024-09-19T09:25:43.003650+0000","last_undegraded":"2024-09-19T09:25:43.003650+0000","last_fullsized":"2024-09-19T09:25:43.003650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:03.628233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991076+0000","last_change":"2024-09-19T09:25:27.279551+0000","last_active":"2024-09-19T09:25:42.991076+0000","last_peered":"2024-09-19T09:25:42.991076+0000","last_clean":"2024-09-19T09:25:42.991076+0000","last_became_active":"2024-09-19T09:25:27.278956+0000","last_became_peered":"2024-09-19T09:25:27.278956+0000","last_unstale":"2024-09-19T09:25:42.991076+0000","last_undegraded":"2024-09-19T09:25:42.991076+0000","last_fullsized":"2024-09-19T09:25:42.991076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:17.186453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992345+0000","last_change":"2024-09-19T09:25:29.288657+0000","last_active":"2024-09-19T09:25:42.992345+0000","last_peered":"2024-09-19T09:25:42.992345+0000","last_clean":"2024-09-19T09:25:42.992345+0000","last_became_active":"2024-09-19T09:25:29.288355+0000","last_became_peered":"2024-09-19T09:25:29.288355+0000","last_unstale":"2024-09-19T09:25:42.992345+0000","last_undegraded":"2024-09-19T09:25:42.992345+0000","last_fullsized":"2024-09-19T09:25:42.992345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:46.237558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991315+0000","last_change":"2024-09-19T09:25:27.323312+0000","last_active":"2024-09-19T09:25:42.991315+0000","last_peered":"2024-09-19T09:25:42.991315+0000","last_clean":"2024-09-19T09:25:42.991315+0000","last_became_active":"2024-09-19T09:25:27.322984+0000","last_became_peered":"2024-09-19T09:25:27.322984+0000","last_unstale":"2024-09-19T09:25:42.991315+0000","last_undegraded":"2024-09-19T09:25:42.991315+0000","last_fullsized":"2024-09-19T09:25:42.991315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:48.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991159+0000","last_change":"2024-09-19T09:25:29.301832+0000","last_active":"2024-09-19T09:25:42.991159+0000","last_peered":"2024-09-19T09:25:42.991159+0000","last_clean":"2024-09-19T09:25:42.991159+0000","last_became_active":"2024-09-19T09:25:29.301539+0000","last_became_peered":"2024-09-19T09:25:29.301539+0000","last_unstale":"2024-09-19T09:25:42.991159+0000","last_undegraded":"2024-09-19T09:25:42.991159+0000","last_fullsized":"2024-09-19T09:25:42.991159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:00.244948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:26.052621+0000","last_change":"2024-09-19T09:25:43.009668+0000","last_active":"2024-09-19T09:26:26.052621+0000","last_peered":"2024-09-19T09:26:26.052621+0000","last_clean":"2024-09-19T09:26:26.052621+0000","last_became_active":"2024-09-19T09:25:42.994966+0000","last_became_peered":"2024-09-19T09:25:42.994966+0000","last_unstale":"2024-09-19T09:26:26.052621+0000","last_undegraded":"2024-09-19T09:26:26.052621+0000","last_fullsized":"2024-09-19T09:26:26.052621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:04:05.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67758,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":518,"num_read_kb":332,"num_write":245,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1842,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":203274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612634,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":431057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841566,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1593344,"data_stored":1081178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005090,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234023,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":430300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462957,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1022934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":901120,"data_stored":430032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1022498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":909312,"data_stored":431456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":489759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":490709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":921600,"data_stored":431371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:26:32.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:26:33.060 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:33.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:32 smithi142 ceph-mon[35032]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:33.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:32 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:33.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:32 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:33.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:32 smithi142 ceph-mon[35032]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:26:33.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:32 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:33.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:32 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:33.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:32 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:33.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:32 smithi097 ceph-mon[34864]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:33.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:32 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:33.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:32 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:33.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:32 smithi097 ceph-mon[34864]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:26:33.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:32 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:33.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:32 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:33.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:32 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:33.182 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:32 smithi016 ceph-mon[30979]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:33.182 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:32 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:33.182 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:32 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:33.182 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:32 smithi016 ceph-mon[30979]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:26:33.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:32 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:26:33.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:32 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:26:33.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:32 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:26:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.549+0000 7f9060e69640 1 -- 172.21.15.16:0/4281164208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f905c103660 msgr2=0x7f905c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.549+0000 7f9060e69640 1 --2- 172.21.15.16:0/4281164208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f905c103660 0x7f905c10bb50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9050009900 tx=0x7f905002a480 comp rx=0 tx=0).stop 2024-09-19T09:26:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.549+0000 7f9060e69640 1 -- 172.21.15.16:0/4281164208 shutdown_connections 2024-09-19T09:26:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.549+0000 7f9060e69640 1 --2- 172.21.15.16:0/4281164208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f905c10c170 0x7f905c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.549+0000 7f9060e69640 1 --2- 172.21.15.16:0/4281164208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f905c103660 0x7f905c10bb50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:33.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.549+0000 7f9060e69640 1 --2- 172.21.15.16:0/4281164208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f905c102d40 0x7f905c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:33.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.549+0000 7f9060e69640 1 -- 172.21.15.16:0/4281164208 >> 172.21.15.16:0/4281164208 conn(0x7f905c0fc7a0 msgr2=0x7f905c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:33.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.549+0000 7f9060e69640 1 -- 172.21.15.16:0/4281164208 shutdown_connections 2024-09-19T09:26:33.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.549+0000 7f9060e69640 1 -- 172.21.15.16:0/4281164208 wait complete. 2024-09-19T09:26:33.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.550+0000 7f9060e69640 1 Processor -- start 2024-09-19T09:26:33.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.550+0000 7f9060e69640 1 -- start start 2024-09-19T09:26:33.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.550+0000 7f9060e69640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f905c102d40 0x7f905c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:33.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.550+0000 7f9060e69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f905c103660 0x7f905c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:33.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.550+0000 7f9060e69640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f905c10c170 0x7f905c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:33.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.550+0000 7f9060e69640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f905c112030 con 0x7f905c103660 2024-09-19T09:26:33.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.550+0000 7f9060e69640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f905c111eb0 con 0x7f905c10c170 2024-09-19T09:26:33.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.550+0000 7f9060e69640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f905c1121b0 con 0x7f905c102d40 2024-09-19T09:26:33.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.550+0000 7f9059d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f905c103660 0x7f905c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.550+0000 7f9059d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f905c103660 0x7f905c198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55428/0 (socket says 172.21.15.16:55428) 2024-09-19T09:26:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.550+0000 7f9059d9b640 1 -- 172.21.15.16:0/1478371525 learned_addr learned my addr 172.21.15.16:0/1478371525 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.550+0000 7f905a59c640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f905c102d40 0x7f905c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.550+0000 7f905ad9d640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f905c10c170 0x7f905c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.551+0000 7f9059d9b640 1 -- 172.21.15.16:0/1478371525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f905c102d40 msgr2=0x7f905c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.551+0000 7f9059d9b640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f905c102d40 0x7f905c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.551+0000 7f9059d9b640 1 -- 172.21.15.16:0/1478371525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f905c10c170 msgr2=0x7f905c19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.551+0000 7f9059d9b640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f905c10c170 0x7f905c19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.551+0000 7f9059d9b640 1 -- 172.21.15.16:0/1478371525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905c19d640 con 0x7f905c103660 2024-09-19T09:26:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.551+0000 7f905a59c640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f905c102d40 0x7f905c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.551+0000 7f905ad9d640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f905c10c170 0x7f905c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:33.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.551+0000 7f9059d9b640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f905c103660 0x7f905c198bd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9050005ab0 tx=0x7f90500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.551+0000 7f904b7fe640 1 -- 172.21.15.16:0/1478371525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905003f030 con 0x7f905c103660 2024-09-19T09:26:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.551+0000 7f9060e69640 1 -- 172.21.15.16:0/1478371525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905c19d8d0 con 0x7f905c103660 2024-09-19T09:26:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.551+0000 7f904b7fe640 1 -- 172.21.15.16:0/1478371525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9050003ce0 con 0x7f905c103660 2024-09-19T09:26:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.551+0000 7f904b7fe640 1 -- 172.21.15.16:0/1478371525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050035070 con 0x7f905c103660 2024-09-19T09:26:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.551+0000 7f9060e69640 1 -- 172.21.15.16:0/1478371525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905c1a5210 con 0x7f905c103660 2024-09-19T09:26:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.552+0000 7f9060e69640 1 -- 172.21.15.16:0/1478371525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905c06b070 con 0x7f905c103660 2024-09-19T09:26:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.553+0000 7f904b7fe640 1 -- 172.21.15.16:0/1478371525 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9050042030 con 0x7f905c103660 2024-09-19T09:26:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.554+0000 7f904b7fe640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f90440a32d0 0x7f90440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.554+0000 7f905a59c640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f90440a32d0 0x7f90440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:33.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.555+0000 7f904b7fe640 1 -- 172.21.15.16:0/1478371525 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f90500f03b0 con 0x7f905c103660 2024-09-19T09:26:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.555+0000 7f905a59c640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f90440a32d0 0x7f90440a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f904c009930 tx=0x7f904c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:33.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.559+0000 7f904b7fe640 1 -- 172.21.15.16:0/1478371525 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f90500bcf70 con 0x7f905c103660 2024-09-19T09:26:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.733+0000 7f9060e69640 1 -- 172.21.15.16:0/1478371525 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f905c062630 con 0x7f90440a32d0 2024-09-19T09:26:33.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.740+0000 7f904b7fe640 1 -- 172.21.15.16:0/1478371525 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258767 (secure 0 0 0) 0x7f905c062630 con 0x7f90440a32d0 2024-09-19T09:26:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:26:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T09:26:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.743+0000 7f9060e69640 1 -- 172.21.15.16:0/1478371525 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f90440a32d0 msgr2=0x7f90440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.743+0000 7f9060e69640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f90440a32d0 0x7f90440a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f904c009930 tx=0x7f904c008040 comp rx=0 tx=0).stop 2024-09-19T09:26:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.743+0000 7f9060e69640 1 -- 172.21.15.16:0/1478371525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f905c103660 msgr2=0x7f905c198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.743+0000 7f9060e69640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f905c103660 0x7f905c198bd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9050005ab0 tx=0x7f90500042f0 comp rx=0 tx=0).stop 2024-09-19T09:26:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.744+0000 7f9060e69640 1 -- 172.21.15.16:0/1478371525 shutdown_connections 2024-09-19T09:26:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.744+0000 7f9060e69640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f905c10c170 0x7f905c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.744+0000 7f9060e69640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f905c103660 0x7f905c198bd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.744+0000 7f9060e69640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f90440a32d0 0x7f90440a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.744+0000 7f9060e69640 1 --2- 172.21.15.16:0/1478371525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f905c102d40 0x7f905c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.744+0000 7f9060e69640 1 -- 172.21.15.16:0/1478371525 >> 172.21.15.16:0/1478371525 conn(0x7f905c0fc7a0 msgr2=0x7f905c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.744+0000 7f9060e69640 1 -- 172.21.15.16:0/1478371525 shutdown_connections 2024-09-19T09:26:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:33.744+0000 7f9060e69640 1 -- 172.21.15.16:0/1478371525 wait complete. 2024-09-19T09:26:34.395 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-19T09:26:33.353982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658360,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":694,"num_read_kb":466,"num_write":428,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1182578,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386864,"kb_used_data":12636,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554960,"statfs":{"total":1151940427776,"available":1151544279040,"internally_reserved":0,"allocated":12939264,"data_stored":7120680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":150,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008340"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988927+0000","last_change":"2024-09-19T09:25:29.294965+0000","last_active":"2024-09-19T09:25:42.988927+0000","last_peered":"2024-09-19T09:25:42.988927+0000","last_clean":"2024-09-19T09:25:42.988927+0000","last_became_active":"2024-09-19T09:25:29.294157+0000","last_became_peered":"2024-09-19T09:25:29.294157+0000","last_unstale":"2024-09-19T09:25:42.988927+0000","last_undegraded":"2024-09-19T09:25:42.988927+0000","last_fullsized":"2024-09-19T09:25:42.988927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:24.643835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992123+0000","last_change":"2024-09-19T09:25:29.298888+0000","last_active":"2024-09-19T09:25:42.992123+0000","last_peered":"2024-09-19T09:25:42.992123+0000","last_clean":"2024-09-19T09:25:42.992123+0000","last_became_active":"2024-09-19T09:25:29.298245+0000","last_became_peered":"2024-09-19T09:25:29.298245+0000","last_unstale":"2024-09-19T09:25:42.992123+0000","last_undegraded":"2024-09-19T09:25:42.992123+0000","last_fullsized":"2024-09-19T09:25:42.992123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:03.162342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992284+0000","last_change":"2024-09-19T09:25:29.292300+0000","last_active":"2024-09-19T09:25:42.992284+0000","last_peered":"2024-09-19T09:25:42.992284+0000","last_clean":"2024-09-19T09:25:42.992284+0000","last_became_active":"2024-09-19T09:25:29.291973+0000","last_became_peered":"2024-09-19T09:25:29.291973+0000","last_unstale":"2024-09-19T09:25:42.992284+0000","last_undegraded":"2024-09-19T09:25:42.992284+0000","last_fullsized":"2024-09-19T09:25:42.992284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:19.672778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990140+0000","last_change":"2024-09-19T09:25:29.305695+0000","last_active":"2024-09-19T09:25:42.990140+0000","last_peered":"2024-09-19T09:25:42.990140+0000","last_clean":"2024-09-19T09:25:42.990140+0000","last_became_active":"2024-09-19T09:25:29.305244+0000","last_became_peered":"2024-09-19T09:25:29.305244+0000","last_unstale":"2024-09-19T09:25:42.990140+0000","last_undegraded":"2024-09-19T09:25:42.990140+0000","last_fullsized":"2024-09-19T09:25:42.990140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:38.054624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991017+0000","last_change":"2024-09-19T09:25:29.295259+0000","last_active":"2024-09-19T09:25:42.991017+0000","last_peered":"2024-09-19T09:25:42.991017+0000","last_clean":"2024-09-19T09:25:42.991017+0000","last_became_active":"2024-09-19T09:25:29.294936+0000","last_became_peered":"2024-09-19T09:25:29.294936+0000","last_unstale":"2024-09-19T09:25:42.991017+0000","last_undegraded":"2024-09-19T09:25:42.991017+0000","last_fullsized":"2024-09-19T09:25:42.991017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:59.193504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986511+0000","last_change":"2024-09-19T09:25:29.299650+0000","last_active":"2024-09-19T09:25:42.986511+0000","last_peered":"2024-09-19T09:25:42.986511+0000","last_clean":"2024-09-19T09:25:42.986511+0000","last_became_active":"2024-09-19T09:25:29.299182+0000","last_became_peered":"2024-09-19T09:25:29.299182+0000","last_unstale":"2024-09-19T09:25:42.986511+0000","last_undegraded":"2024-09-19T09:25:42.986511+0000","last_fullsized":"2024-09-19T09:25:42.986511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:44.637771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989644+0000","last_change":"2024-09-19T09:25:29.302501+0000","last_active":"2024-09-19T09:25:42.989644+0000","last_peered":"2024-09-19T09:25:42.989644+0000","last_clean":"2024-09-19T09:25:42.989644+0000","last_became_active":"2024-09-19T09:25:29.301419+0000","last_became_peered":"2024-09-19T09:25:29.301419+0000","last_unstale":"2024-09-19T09:25:42.989644+0000","last_undegraded":"2024-09-19T09:25:42.989644+0000","last_fullsized":"2024-09-19T09:25:42.989644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:50.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990840+0000","last_change":"2024-09-19T09:25:29.300212+0000","last_active":"2024-09-19T09:25:42.990840+0000","last_peered":"2024-09-19T09:25:42.990840+0000","last_clean":"2024-09-19T09:25:42.990840+0000","last_became_active":"2024-09-19T09:25:29.299873+0000","last_became_peered":"2024-09-19T09:25:29.299873+0000","last_unstale":"2024-09-19T09:25:42.990840+0000","last_undegraded":"2024-09-19T09:25:42.990840+0000","last_fullsized":"2024-09-19T09:25:42.990840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:34.620314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.004117+0000","last_change":"2024-09-19T09:25:29.290214+0000","last_active":"2024-09-19T09:25:43.004117+0000","last_peered":"2024-09-19T09:25:43.004117+0000","last_clean":"2024-09-19T09:25:43.004117+0000","last_became_active":"2024-09-19T09:25:29.289707+0000","last_became_peered":"2024-09-19T09:25:29.289707+0000","last_unstale":"2024-09-19T09:25:43.004117+0000","last_undegraded":"2024-09-19T09:25:43.004117+0000","last_fullsized":"2024-09-19T09:25:43.004117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:16.341340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.987005+0000","last_change":"2024-09-19T09:25:29.301774+0000","last_active":"2024-09-19T09:25:42.987005+0000","last_peered":"2024-09-19T09:25:42.987005+0000","last_clean":"2024-09-19T09:25:42.987005+0000","last_became_active":"2024-09-19T09:25:29.301398+0000","last_became_peered":"2024-09-19T09:25:29.301398+0000","last_unstale":"2024-09-19T09:25:42.987005+0000","last_undegraded":"2024-09-19T09:25:42.987005+0000","last_fullsized":"2024-09-19T09:25:42.987005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:36.058038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989844+0000","last_change":"2024-09-19T09:25:29.301475+0000","last_active":"2024-09-19T09:25:42.989844+0000","last_peered":"2024-09-19T09:25:42.989844+0000","last_clean":"2024-09-19T09:25:42.989844+0000","last_became_active":"2024-09-19T09:25:29.300908+0000","last_became_peered":"2024-09-19T09:25:29.300908+0000","last_unstale":"2024-09-19T09:25:42.989844+0000","last_undegraded":"2024-09-19T09:25:42.989844+0000","last_fullsized":"2024-09-19T09:25:42.989844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:44.774127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994395+0000","last_change":"2024-09-19T09:25:42.994395+0000","last_active":"2024-09-19T09:25:42.994395+0000","last_peered":"2024-09-19T09:25:42.994395+0000","last_clean":"2024-09-19T09:25:42.994395+0000","last_became_active":"2024-09-19T09:25:42.993992+0000","last_became_peered":"2024-09-19T09:25:42.993992+0000","last_unstale":"2024-09-19T09:25:42.994395+0000","last_undegraded":"2024-09-19T09:25:42.994395+0000","last_fullsized":"2024-09-19T09:25:42.994395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.486069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989979+0000","last_change":"2024-09-19T09:25:42.989979+0000","last_active":"2024-09-19T09:25:42.989979+0000","last_peered":"2024-09-19T09:25:42.989979+0000","last_clean":"2024-09-19T09:25:42.989979+0000","last_became_active":"2024-09-19T09:25:42.989578+0000","last_became_peered":"2024-09-19T09:25:42.989578+0000","last_unstale":"2024-09-19T09:25:42.989979+0000","last_undegraded":"2024-09-19T09:25:42.989979+0000","last_fullsized":"2024-09-19T09:25:42.989979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:04.583023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992064+0000","last_change":"2024-09-19T09:25:29.299917+0000","last_active":"2024-09-19T09:25:42.992064+0000","last_peered":"2024-09-19T09:25:42.992064+0000","last_clean":"2024-09-19T09:25:42.992064+0000","last_became_active":"2024-09-19T09:25:29.299571+0000","last_became_peered":"2024-09-19T09:25:29.299571+0000","last_unstale":"2024-09-19T09:25:42.992064+0000","last_undegraded":"2024-09-19T09:25:42.992064+0000","last_fullsized":"2024-09-19T09:25:42.992064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:25.142252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991982+0000","last_change":"2024-09-19T09:25:29.299713+0000","last_active":"2024-09-19T09:25:42.991982+0000","last_peered":"2024-09-19T09:25:42.991982+0000","last_clean":"2024-09-19T09:25:42.991982+0000","last_became_active":"2024-09-19T09:25:29.299234+0000","last_became_peered":"2024-09-19T09:25:29.299234+0000","last_unstale":"2024-09-19T09:25:42.991982+0000","last_undegraded":"2024-09-19T09:25:42.991982+0000","last_fullsized":"2024-09-19T09:25:42.991982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:34.714714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003535+0000","last_change":"2024-09-19T09:25:29.293010+0000","last_active":"2024-09-19T09:25:43.003535+0000","last_peered":"2024-09-19T09:25:43.003535+0000","last_clean":"2024-09-19T09:25:43.003535+0000","last_became_active":"2024-09-19T09:25:29.292645+0000","last_became_peered":"2024-09-19T09:25:29.292645+0000","last_unstale":"2024-09-19T09:25:43.003535+0000","last_undegraded":"2024-09-19T09:25:43.003535+0000","last_fullsized":"2024-09-19T09:25:43.003535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:24.749075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989031+0000","last_change":"2024-09-19T09:25:29.294530+0000","last_active":"2024-09-19T09:25:42.989031+0000","last_peered":"2024-09-19T09:25:42.989031+0000","last_clean":"2024-09-19T09:25:42.989031+0000","last_became_active":"2024-09-19T09:25:29.293699+0000","last_became_peered":"2024-09-19T09:25:29.293699+0000","last_unstale":"2024-09-19T09:25:42.989031+0000","last_undegraded":"2024-09-19T09:25:42.989031+0000","last_fullsized":"2024-09-19T09:25:42.989031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:53.491258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988781+0000","last_change":"2024-09-19T09:25:29.290600+0000","last_active":"2024-09-19T09:25:42.988781+0000","last_peered":"2024-09-19T09:25:42.988781+0000","last_clean":"2024-09-19T09:25:42.988781+0000","last_became_active":"2024-09-19T09:25:29.290305+0000","last_became_peered":"2024-09-19T09:25:29.290305+0000","last_unstale":"2024-09-19T09:25:42.988781+0000","last_undegraded":"2024-09-19T09:25:42.988781+0000","last_fullsized":"2024-09-19T09:25:42.988781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:42.809659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.993154+0000","last_change":"2024-09-19T09:25:29.300591+0000","last_active":"2024-09-19T09:25:42.993154+0000","last_peered":"2024-09-19T09:25:42.993154+0000","last_clean":"2024-09-19T09:25:42.993154+0000","last_became_active":"2024-09-19T09:25:29.300251+0000","last_became_peered":"2024-09-19T09:25:29.300251+0000","last_unstale":"2024-09-19T09:25:42.993154+0000","last_undegraded":"2024-09-19T09:25:42.993154+0000","last_fullsized":"2024-09-19T09:25:42.993154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:08.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988507+0000","last_change":"2024-09-19T09:25:29.298452+0000","last_active":"2024-09-19T09:25:42.988507+0000","last_peered":"2024-09-19T09:25:42.988507+0000","last_clean":"2024-09-19T09:25:42.988507+0000","last_became_active":"2024-09-19T09:25:29.298137+0000","last_became_peered":"2024-09-19T09:25:29.298137+0000","last_unstale":"2024-09-19T09:25:42.988507+0000","last_undegraded":"2024-09-19T09:25:42.988507+0000","last_fullsized":"2024-09-19T09:25:42.988507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:11.716228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985825+0000","last_change":"2024-09-19T09:25:29.294278+0000","last_active":"2024-09-19T09:25:42.985825+0000","last_peered":"2024-09-19T09:25:42.985825+0000","last_clean":"2024-09-19T09:25:42.985825+0000","last_became_active":"2024-09-19T09:25:29.293972+0000","last_became_peered":"2024-09-19T09:25:29.293972+0000","last_unstale":"2024-09-19T09:25:42.985825+0000","last_undegraded":"2024-09-19T09:25:42.985825+0000","last_fullsized":"2024-09-19T09:25:42.985825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:27.923839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003737+0000","last_change":"2024-09-19T09:25:29.294000+0000","last_active":"2024-09-19T09:25:43.003737+0000","last_peered":"2024-09-19T09:25:43.003737+0000","last_clean":"2024-09-19T09:25:43.003737+0000","last_became_active":"2024-09-19T09:25:29.293486+0000","last_became_peered":"2024-09-19T09:25:29.293486+0000","last_unstale":"2024-09-19T09:25:43.003737+0000","last_undegraded":"2024-09-19T09:25:43.003737+0000","last_fullsized":"2024-09-19T09:25:43.003737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:44.053620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994951+0000","last_change":"2024-09-19T09:25:42.994951+0000","last_active":"2024-09-19T09:25:42.994951+0000","last_peered":"2024-09-19T09:25:42.994951+0000","last_clean":"2024-09-19T09:25:42.994951+0000","last_became_active":"2024-09-19T09:25:42.994588+0000","last_became_peered":"2024-09-19T09:25:42.994588+0000","last_unstale":"2024-09-19T09:25:42.994951+0000","last_undegraded":"2024-09-19T09:25:42.994951+0000","last_fullsized":"2024-09-19T09:25:42.994951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:10.141169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995128+0000","last_change":"2024-09-19T09:25:29.286395+0000","last_active":"2024-09-19T09:25:42.995128+0000","last_peered":"2024-09-19T09:25:42.995128+0000","last_clean":"2024-09-19T09:25:42.995128+0000","last_became_active":"2024-09-19T09:25:29.286048+0000","last_became_peered":"2024-09-19T09:25:29.286048+0000","last_unstale":"2024-09-19T09:25:42.995128+0000","last_undegraded":"2024-09-19T09:25:42.995128+0000","last_fullsized":"2024-09-19T09:25:42.995128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:43.121166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991295+0000","last_change":"2024-09-19T09:25:29.298458+0000","last_active":"2024-09-19T09:25:42.991295+0000","last_peered":"2024-09-19T09:25:42.991295+0000","last_clean":"2024-09-19T09:25:42.991295+0000","last_became_active":"2024-09-19T09:25:29.297993+0000","last_became_peered":"2024-09-19T09:25:29.297993+0000","last_unstale":"2024-09-19T09:25:42.991295+0000","last_undegraded":"2024-09-19T09:25:42.991295+0000","last_fullsized":"2024-09-19T09:25:42.991295+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:44.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992762+0000","last_change":"2024-09-19T09:25:29.297167+0000","last_active":"2024-09-19T09:25:42.992762+0000","last_peered":"2024-09-19T09:25:42.992762+0000","last_clean":"2024-09-19T09:25:42.992762+0000","last_became_active":"2024-09-19T09:25:29.296777+0000","last_became_peered":"2024-09-19T09:25:29.296777+0000","last_unstale":"2024-09-19T09:25:42.992762+0000","last_undegraded":"2024-09-19T09:25:42.992762+0000","last_fullsized":"2024-09-19T09:25:42.992762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:40.581643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988833+0000","last_change":"2024-09-19T09:25:29.296169+0000","last_active":"2024-09-19T09:25:42.988833+0000","last_peered":"2024-09-19T09:25:42.988833+0000","last_clean":"2024-09-19T09:25:42.988833+0000","last_became_active":"2024-09-19T09:25:29.295806+0000","last_became_peered":"2024-09-19T09:25:29.295806+0000","last_unstale":"2024-09-19T09:25:42.988833+0000","last_undegraded":"2024-09-19T09:25:42.988833+0000","last_fullsized":"2024-09-19T09:25:42.988833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:39.639884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985866+0000","last_change":"2024-09-19T09:25:29.297960+0000","last_active":"2024-09-19T09:25:42.985866+0000","last_peered":"2024-09-19T09:25:42.985866+0000","last_clean":"2024-09-19T09:25:42.985866+0000","last_became_active":"2024-09-19T09:25:29.297577+0000","last_became_peered":"2024-09-19T09:25:29.297577+0000","last_unstale":"2024-09-19T09:25:42.985866+0000","last_undegraded":"2024-09-19T09:25:42.985866+0000","last_fullsized":"2024-09-19T09:25:42.985866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:27.035504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992420+0000","last_change":"2024-09-19T09:25:29.297896+0000","last_active":"2024-09-19T09:25:42.992420+0000","last_peered":"2024-09-19T09:25:42.992420+0000","last_clean":"2024-09-19T09:25:42.992420+0000","last_became_active":"2024-09-19T09:25:29.297594+0000","last_became_peered":"2024-09-19T09:25:29.297594+0000","last_unstale":"2024-09-19T09:25:42.992420+0000","last_undegraded":"2024-09-19T09:25:42.992420+0000","last_fullsized":"2024-09-19T09:25:42.992420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.366325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992205+0000","last_change":"2024-09-19T09:25:29.301851+0000","last_active":"2024-09-19T09:25:42.992205+0000","last_peered":"2024-09-19T09:25:42.992205+0000","last_clean":"2024-09-19T09:25:42.992205+0000","last_became_active":"2024-09-19T09:25:29.299282+0000","last_became_peered":"2024-09-19T09:25:29.299282+0000","last_unstale":"2024-09-19T09:25:42.992205+0000","last_undegraded":"2024-09-19T09:25:42.992205+0000","last_fullsized":"2024-09-19T09:25:42.992205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:00.382114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992658+0000","last_change":"2024-09-19T09:25:29.303316+0000","last_active":"2024-09-19T09:25:42.992658+0000","last_peered":"2024-09-19T09:25:42.992658+0000","last_clean":"2024-09-19T09:25:42.992658+0000","last_became_active":"2024-09-19T09:25:29.302994+0000","last_became_peered":"2024-09-19T09:25:29.302994+0000","last_unstale":"2024-09-19T09:25:42.992658+0000","last_undegraded":"2024-09-19T09:25:42.992658+0000","last_fullsized":"2024-09-19T09:25:42.992658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:46.233991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988501+0000","last_change":"2024-09-19T09:25:29.293004+0000","last_active":"2024-09-19T09:25:42.988501+0000","last_peered":"2024-09-19T09:25:42.988501+0000","last_clean":"2024-09-19T09:25:42.988501+0000","last_became_active":"2024-09-19T09:25:29.292657+0000","last_became_peered":"2024-09-19T09:25:29.292657+0000","last_unstale":"2024-09-19T09:25:42.988501+0000","last_undegraded":"2024-09-19T09:25:42.988501+0000","last_fullsized":"2024-09-19T09:25:42.988501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:50.966477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991936+0000","last_change":"2024-09-19T09:25:29.300434+0000","last_active":"2024-09-19T09:25:42.991936+0000","last_peered":"2024-09-19T09:25:42.991936+0000","last_clean":"2024-09-19T09:25:42.991936+0000","last_became_active":"2024-09-19T09:25:29.300053+0000","last_became_peered":"2024-09-19T09:25:29.300053+0000","last_unstale":"2024-09-19T09:25:42.991936+0000","last_undegraded":"2024-09-19T09:25:42.991936+0000","last_fullsized":"2024-09-19T09:25:42.991936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:13.283298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":86,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:32.038569+0000","last_change":"2024-09-19T09:25:27.280792+0000","last_active":"2024-09-19T09:26:32.038569+0000","last_peered":"2024-09-19T09:26:32.038569+0000","last_clean":"2024-09-19T09:26:32.038569+0000","last_became_active":"2024-09-19T09:25:27.280443+0000","last_became_peered":"2024-09-19T09:25:27.280443+0000","last_unstale":"2024-09-19T09:26:32.038569+0000","last_undegraded":"2024-09-19T09:26:32.038569+0000","last_fullsized":"2024-09-19T09:26:32.038569+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.599512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.997114+0000","last_change":"2024-09-19T09:25:29.298975+0000","last_active":"2024-09-19T09:25:42.997114+0000","last_peered":"2024-09-19T09:25:42.997114+0000","last_clean":"2024-09-19T09:25:42.997114+0000","last_became_active":"2024-09-19T09:25:29.298476+0000","last_became_peered":"2024-09-19T09:25:29.298476+0000","last_unstale":"2024-09-19T09:25:42.997114+0000","last_undegraded":"2024-09-19T09:25:42.997114+0000","last_fullsized":"2024-09-19T09:25:42.997114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:09.658894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986623+0000","last_change":"2024-09-19T09:25:27.278257+0000","last_active":"2024-09-19T09:25:42.986623+0000","last_peered":"2024-09-19T09:25:42.986623+0000","last_clean":"2024-09-19T09:25:42.986623+0000","last_became_active":"2024-09-19T09:25:27.277897+0000","last_became_peered":"2024-09-19T09:25:27.277897+0000","last_unstale":"2024-09-19T09:25:42.986623+0000","last_undegraded":"2024-09-19T09:25:42.986623+0000","last_fullsized":"2024-09-19T09:25:42.986623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:47.550314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985617+0000","last_change":"2024-09-19T09:25:27.272215+0000","last_active":"2024-09-19T09:25:42.985617+0000","last_peered":"2024-09-19T09:25:42.985617+0000","last_clean":"2024-09-19T09:25:42.985617+0000","last_became_active":"2024-09-19T09:25:27.271821+0000","last_became_peered":"2024-09-19T09:25:27.271821+0000","last_unstale":"2024-09-19T09:25:42.985617+0000","last_undegraded":"2024-09-19T09:25:42.985617+0000","last_fullsized":"2024-09-19T09:25:42.985617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:30.615730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990922+0000","last_change":"2024-09-19T09:25:29.289347+0000","last_active":"2024-09-19T09:25:42.990922+0000","last_peered":"2024-09-19T09:25:42.990922+0000","last_clean":"2024-09-19T09:25:42.990922+0000","last_became_active":"2024-09-19T09:25:29.289015+0000","last_became_peered":"2024-09-19T09:25:29.289015+0000","last_unstale":"2024-09-19T09:25:42.990922+0000","last_undegraded":"2024-09-19T09:25:42.990922+0000","last_fullsized":"2024-09-19T09:25:42.990922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:18.419334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995432+0000","last_change":"2024-09-19T09:25:27.277342+0000","last_active":"2024-09-19T09:25:42.995432+0000","last_peered":"2024-09-19T09:25:42.995432+0000","last_clean":"2024-09-19T09:25:42.995432+0000","last_became_active":"2024-09-19T09:25:27.277016+0000","last_became_peered":"2024-09-19T09:25:27.277016+0000","last_unstale":"2024-09-19T09:25:42.995432+0000","last_undegraded":"2024-09-19T09:25:42.995432+0000","last_fullsized":"2024-09-19T09:25:42.995432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:56.736156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988740+0000","last_change":"2024-09-19T09:25:29.290924+0000","last_active":"2024-09-19T09:25:42.988740+0000","last_peered":"2024-09-19T09:25:42.988740+0000","last_clean":"2024-09-19T09:25:42.988740+0000","last_became_active":"2024-09-19T09:25:29.290482+0000","last_became_peered":"2024-09-19T09:25:29.290482+0000","last_unstale":"2024-09-19T09:25:42.988740+0000","last_undegraded":"2024-09-19T09:25:42.988740+0000","last_fullsized":"2024-09-19T09:25:42.988740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:16.265040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'4","reported_seq":130,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:31.036715+0000","last_change":"2024-09-19T09:25:27.277604+0000","last_active":"2024-09-19T09:26:31.036715+0000","last_peered":"2024-09-19T09:26:31.036715+0000","last_clean":"2024-09-19T09:26:31.036715+0000","last_became_active":"2024-09-19T09:25:27.277271+0000","last_became_peered":"2024-09-19T09:25:27.277271+0000","last_unstale":"2024-09-19T09:26:31.036715+0000","last_undegraded":"2024-09-19T09:26:31.036715+0000","last_fullsized":"2024-09-19T09:26:31.036715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:32.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.004044+0000","last_change":"2024-09-19T09:25:29.290537+0000","last_active":"2024-09-19T09:25:43.004044+0000","last_peered":"2024-09-19T09:25:43.004044+0000","last_clean":"2024-09-19T09:25:43.004044+0000","last_became_active":"2024-09-19T09:25:29.287984+0000","last_became_peered":"2024-09-19T09:25:29.287984+0000","last_unstale":"2024-09-19T09:25:43.004044+0000","last_undegraded":"2024-09-19T09:25:43.004044+0000","last_fullsized":"2024-09-19T09:25:43.004044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:26.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990018+0000","last_change":"2024-09-19T09:25:27.278560+0000","last_active":"2024-09-19T09:25:42.990018+0000","last_peered":"2024-09-19T09:25:42.990018+0000","last_clean":"2024-09-19T09:25:42.990018+0000","last_became_active":"2024-09-19T09:25:27.277950+0000","last_became_peered":"2024-09-19T09:25:27.277950+0000","last_unstale":"2024-09-19T09:25:42.990018+0000","last_undegraded":"2024-09-19T09:25:42.990018+0000","last_fullsized":"2024-09-19T09:25:42.990018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:28.387725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992413+0000","last_change":"2024-09-19T09:25:29.298897+0000","last_active":"2024-09-19T09:25:42.992413+0000","last_peered":"2024-09-19T09:25:42.992413+0000","last_clean":"2024-09-19T09:25:42.992413+0000","last_became_active":"2024-09-19T09:25:29.298617+0000","last_became_peered":"2024-09-19T09:25:29.298617+0000","last_unstale":"2024-09-19T09:25:42.992413+0000","last_undegraded":"2024-09-19T09:25:42.992413+0000","last_fullsized":"2024-09-19T09:25:42.992413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:53.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994870+0000","last_change":"2024-09-19T09:25:27.324871+0000","last_active":"2024-09-19T09:25:42.994870+0000","last_peered":"2024-09-19T09:25:42.994870+0000","last_clean":"2024-09-19T09:25:42.994870+0000","last_became_active":"2024-09-19T09:25:27.324471+0000","last_became_peered":"2024-09-19T09:25:27.324471+0000","last_unstale":"2024-09-19T09:25:42.994870+0000","last_undegraded":"2024-09-19T09:25:42.994870+0000","last_fullsized":"2024-09-19T09:25:42.994870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:35.936413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.004242+0000","last_change":"2024-09-19T09:25:29.298332+0000","last_active":"2024-09-19T09:25:43.004242+0000","last_peered":"2024-09-19T09:25:43.004242+0000","last_clean":"2024-09-19T09:25:43.004242+0000","last_became_active":"2024-09-19T09:25:29.298006+0000","last_became_peered":"2024-09-19T09:25:29.298006+0000","last_unstale":"2024-09-19T09:25:43.004242+0000","last_undegraded":"2024-09-19T09:25:43.004242+0000","last_fullsized":"2024-09-19T09:25:43.004242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:02.621821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":167,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:31.046422+0000","last_change":"2024-09-19T09:25:27.279528+0000","last_active":"2024-09-19T09:26:31.046422+0000","last_peered":"2024-09-19T09:26:31.046422+0000","last_clean":"2024-09-19T09:26:31.046422+0000","last_became_active":"2024-09-19T09:25:27.279149+0000","last_became_peered":"2024-09-19T09:25:27.279149+0000","last_unstale":"2024-09-19T09:26:31.046422+0000","last_undegraded":"2024-09-19T09:26:31.046422+0000","last_fullsized":"2024-09-19T09:26:31.046422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:47.030736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991057+0000","last_change":"2024-09-19T09:25:29.297538+0000","last_active":"2024-09-19T09:25:42.991057+0000","last_peered":"2024-09-19T09:25:42.991057+0000","last_clean":"2024-09-19T09:25:42.991057+0000","last_became_active":"2024-09-19T09:25:29.297181+0000","last_became_peered":"2024-09-19T09:25:29.297181+0000","last_unstale":"2024-09-19T09:25:42.991057+0000","last_undegraded":"2024-09-19T09:25:42.991057+0000","last_fullsized":"2024-09-19T09:25:42.991057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:53.135438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992860+0000","last_change":"2024-09-19T09:25:27.277090+0000","last_active":"2024-09-19T09:25:42.992860+0000","last_peered":"2024-09-19T09:25:42.992860+0000","last_clean":"2024-09-19T09:25:42.992860+0000","last_became_active":"2024-09-19T09:25:27.276718+0000","last_became_peered":"2024-09-19T09:25:27.276718+0000","last_unstale":"2024-09-19T09:25:42.992860+0000","last_undegraded":"2024-09-19T09:25:42.992860+0000","last_fullsized":"2024-09-19T09:25:42.992860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:25.664261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985642+0000","last_change":"2024-09-19T09:25:29.294393+0000","last_active":"2024-09-19T09:25:42.985642+0000","last_peered":"2024-09-19T09:25:42.985642+0000","last_clean":"2024-09-19T09:25:42.985642+0000","last_became_active":"2024-09-19T09:25:29.294099+0000","last_became_peered":"2024-09-19T09:25:29.294099+0000","last_unstale":"2024-09-19T09:25:42.985642+0000","last_undegraded":"2024-09-19T09:25:42.985642+0000","last_fullsized":"2024-09-19T09:25:42.985642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:17.333736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995398+0000","last_change":"2024-09-19T09:25:27.278593+0000","last_active":"2024-09-19T09:25:42.995398+0000","last_peered":"2024-09-19T09:25:42.995398+0000","last_clean":"2024-09-19T09:25:42.995398+0000","last_became_active":"2024-09-19T09:25:27.278232+0000","last_became_peered":"2024-09-19T09:25:27.278232+0000","last_unstale":"2024-09-19T09:25:42.995398+0000","last_undegraded":"2024-09-19T09:25:42.995398+0000","last_fullsized":"2024-09-19T09:25:42.995398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:05.852671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989855+0000","last_change":"2024-09-19T09:25:29.300024+0000","last_active":"2024-09-19T09:25:42.989855+0000","last_peered":"2024-09-19T09:25:42.989855+0000","last_clean":"2024-09-19T09:25:42.989855+0000","last_became_active":"2024-09-19T09:25:29.299683+0000","last_became_peered":"2024-09-19T09:25:29.299683+0000","last_unstale":"2024-09-19T09:25:42.989855+0000","last_undegraded":"2024-09-19T09:25:42.989855+0000","last_fullsized":"2024-09-19T09:25:42.989855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:42.249399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":85,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:31.038917+0000","last_change":"2024-09-19T09:25:27.272739+0000","last_active":"2024-09-19T09:26:31.038917+0000","last_peered":"2024-09-19T09:26:31.038917+0000","last_clean":"2024-09-19T09:26:31.038917+0000","last_became_active":"2024-09-19T09:25:27.272354+0000","last_became_peered":"2024-09-19T09:25:27.272354+0000","last_unstale":"2024-09-19T09:26:31.038917+0000","last_undegraded":"2024-09-19T09:26:31.038917+0000","last_fullsized":"2024-09-19T09:26:31.038917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:27.863840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989616+0000","last_change":"2024-09-19T09:25:29.303460+0000","last_active":"2024-09-19T09:25:42.989616+0000","last_peered":"2024-09-19T09:25:42.989616+0000","last_clean":"2024-09-19T09:25:42.989616+0000","last_became_active":"2024-09-19T09:25:29.303127+0000","last_became_peered":"2024-09-19T09:25:29.303127+0000","last_unstale":"2024-09-19T09:25:42.989616+0000","last_undegraded":"2024-09-19T09:25:42.989616+0000","last_fullsized":"2024-09-19T09:25:42.989616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:35.313924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990876+0000","last_change":"2024-09-19T09:25:27.323238+0000","last_active":"2024-09-19T09:25:42.990876+0000","last_peered":"2024-09-19T09:25:42.990876+0000","last_clean":"2024-09-19T09:25:42.990876+0000","last_became_active":"2024-09-19T09:25:27.322868+0000","last_became_peered":"2024-09-19T09:25:27.322868+0000","last_unstale":"2024-09-19T09:25:42.990876+0000","last_undegraded":"2024-09-19T09:25:42.990876+0000","last_fullsized":"2024-09-19T09:25:42.990876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:22.890280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.005569+0000","last_change":"2024-09-19T09:25:43.005569+0000","last_active":"2024-09-19T09:25:43.005569+0000","last_peered":"2024-09-19T09:25:43.005569+0000","last_clean":"2024-09-19T09:25:43.005569+0000","last_became_active":"2024-09-19T09:25:43.005143+0000","last_became_peered":"2024-09-19T09:25:43.005143+0000","last_unstale":"2024-09-19T09:25:43.005569+0000","last_undegraded":"2024-09-19T09:25:43.005569+0000","last_fullsized":"2024-09-19T09:25:43.005569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:26.297013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991438+0000","last_change":"2024-09-19T09:25:27.282737+0000","last_active":"2024-09-19T09:25:42.991438+0000","last_peered":"2024-09-19T09:25:42.991438+0000","last_clean":"2024-09-19T09:25:42.991438+0000","last_became_active":"2024-09-19T09:25:27.281785+0000","last_became_peered":"2024-09-19T09:25:27.281785+0000","last_unstale":"2024-09-19T09:25:42.991438+0000","last_undegraded":"2024-09-19T09:25:42.991438+0000","last_fullsized":"2024-09-19T09:25:42.991438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:41.552206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988791+0000","last_change":"2024-09-19T09:25:29.300168+0000","last_active":"2024-09-19T09:25:42.988791+0000","last_peered":"2024-09-19T09:25:42.988791+0000","last_clean":"2024-09-19T09:25:42.988791+0000","last_became_active":"2024-09-19T09:25:29.299811+0000","last_became_peered":"2024-09-19T09:25:29.299811+0000","last_unstale":"2024-09-19T09:25:42.988791+0000","last_undegraded":"2024-09-19T09:25:42.988791+0000","last_fullsized":"2024-09-19T09:25:42.988791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:06.691259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985783+0000","last_change":"2024-09-19T09:25:27.278433+0000","last_active":"2024-09-19T09:25:42.985783+0000","last_peered":"2024-09-19T09:25:42.985783+0000","last_clean":"2024-09-19T09:25:42.985783+0000","last_became_active":"2024-09-19T09:25:27.278076+0000","last_became_peered":"2024-09-19T09:25:27.278076+0000","last_unstale":"2024-09-19T09:25:42.985783+0000","last_undegraded":"2024-09-19T09:25:42.985783+0000","last_fullsized":"2024-09-19T09:25:42.985783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:53.799265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990338+0000","last_change":"2024-09-19T09:25:29.295650+0000","last_active":"2024-09-19T09:25:42.990338+0000","last_peered":"2024-09-19T09:25:42.990338+0000","last_clean":"2024-09-19T09:25:42.990338+0000","last_became_active":"2024-09-19T09:25:29.295277+0000","last_became_peered":"2024-09-19T09:25:29.295277+0000","last_unstale":"2024-09-19T09:25:42.990338+0000","last_undegraded":"2024-09-19T09:25:42.990338+0000","last_fullsized":"2024-09-19T09:25:42.990338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:57.690699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996123+0000","last_change":"2024-09-19T09:25:27.279239+0000","last_active":"2024-09-19T09:25:42.996123+0000","last_peered":"2024-09-19T09:25:42.996123+0000","last_clean":"2024-09-19T09:25:42.996123+0000","last_became_active":"2024-09-19T09:25:27.278868+0000","last_became_peered":"2024-09-19T09:25:27.278868+0000","last_unstale":"2024-09-19T09:25:42.996123+0000","last_undegraded":"2024-09-19T09:25:42.996123+0000","last_fullsized":"2024-09-19T09:25:42.996123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:41.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990429+0000","last_change":"2024-09-19T09:25:29.300833+0000","last_active":"2024-09-19T09:25:42.990429+0000","last_peered":"2024-09-19T09:25:42.990429+0000","last_clean":"2024-09-19T09:25:42.990429+0000","last_became_active":"2024-09-19T09:25:29.300472+0000","last_became_peered":"2024-09-19T09:25:29.300472+0000","last_unstale":"2024-09-19T09:25:42.990429+0000","last_undegraded":"2024-09-19T09:25:42.990429+0000","last_fullsized":"2024-09-19T09:25:42.990429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:50.591276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":71,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:31.077335+0000","last_change":"2024-09-19T09:25:43.001722+0000","last_active":"2024-09-19T09:26:31.077335+0000","last_peered":"2024-09-19T09:26:31.077335+0000","last_clean":"2024-09-19T09:26:31.077335+0000","last_became_active":"2024-09-19T09:25:42.992796+0000","last_became_peered":"2024-09-19T09:25:42.992796+0000","last_unstale":"2024-09-19T09:26:31.077335+0000","last_undegraded":"2024-09-19T09:26:31.077335+0000","last_fullsized":"2024-09-19T09:26:31.077335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:59.639010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990231+0000","last_change":"2024-09-19T09:25:29.295906+0000","last_active":"2024-09-19T09:25:42.990231+0000","last_peered":"2024-09-19T09:25:42.990231+0000","last_clean":"2024-09-19T09:25:42.990231+0000","last_became_active":"2024-09-19T09:25:29.295565+0000","last_became_peered":"2024-09-19T09:25:29.295565+0000","last_unstale":"2024-09-19T09:25:42.990231+0000","last_undegraded":"2024-09-19T09:25:42.990231+0000","last_fullsized":"2024-09-19T09:25:42.990231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:24.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":334,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989806+0000","last_change":"2024-09-19T09:24:19.256300+0000","last_active":"2024-09-19T09:25:42.989806+0000","last_peered":"2024-09-19T09:25:42.989806+0000","last_clean":"2024-09-19T09:25:42.989806+0000","last_became_active":"2024-09-19T09:24:19.255833+0000","last_became_peered":"2024-09-19T09:24:19.255833+0000","last_unstale":"2024-09-19T09:25:42.989806+0000","last_undegraded":"2024-09-19T09:25:42.989806+0000","last_fullsized":"2024-09-19T09:25:42.989806+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_clean_scrub_stamp":"2024-09-19T09:20:57.798838+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:14:33.343748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989637+0000","last_change":"2024-09-19T09:25:27.272090+0000","last_active":"2024-09-19T09:25:42.989637+0000","last_peered":"2024-09-19T09:25:42.989637+0000","last_clean":"2024-09-19T09:25:42.989637+0000","last_became_active":"2024-09-19T09:25:27.271651+0000","last_became_peered":"2024-09-19T09:25:27.271651+0000","last_unstale":"2024-09-19T09:25:42.989637+0000","last_undegraded":"2024-09-19T09:25:42.989637+0000","last_fullsized":"2024-09-19T09:25:42.989637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:45.847082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:29.300718+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:29.299460+0000","last_became_peered":"2024-09-19T09:25:29.299460+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:43.063909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003299+0000","last_change":"2024-09-19T09:25:43.003299+0000","last_active":"2024-09-19T09:25:43.003299+0000","last_peered":"2024-09-19T09:25:43.003299+0000","last_clean":"2024-09-19T09:25:43.003299+0000","last_became_active":"2024-09-19T09:25:42.999686+0000","last_became_peered":"2024-09-19T09:25:42.999686+0000","last_unstale":"2024-09-19T09:25:43.003299+0000","last_undegraded":"2024-09-19T09:25:43.003299+0000","last_fullsized":"2024-09-19T09:25:43.003299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:05.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985606+0000","last_change":"2024-09-19T09:25:29.297833+0000","last_active":"2024-09-19T09:25:42.985606+0000","last_peered":"2024-09-19T09:25:42.985606+0000","last_clean":"2024-09-19T09:25:42.985606+0000","last_became_active":"2024-09-19T09:25:29.297534+0000","last_became_peered":"2024-09-19T09:25:29.297534+0000","last_unstale":"2024-09-19T09:25:42.985606+0000","last_undegraded":"2024-09-19T09:25:42.985606+0000","last_fullsized":"2024-09-19T09:25:42.985606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:47.156253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'5","reported_seq":83,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:32.039754+0000","last_change":"2024-09-19T09:25:27.322759+0000","last_active":"2024-09-19T09:26:32.039754+0000","last_peered":"2024-09-19T09:26:32.039754+0000","last_clean":"2024-09-19T09:26:32.039754+0000","last_became_active":"2024-09-19T09:25:27.322402+0000","last_became_peered":"2024-09-19T09:25:27.322402+0000","last_unstale":"2024-09-19T09:26:32.039754+0000","last_undegraded":"2024-09-19T09:26:32.039754+0000","last_fullsized":"2024-09-19T09:26:32.039754+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:06.276957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.999834+0000","last_change":"2024-09-19T09:25:42.999834+0000","last_active":"2024-09-19T09:25:42.999834+0000","last_peered":"2024-09-19T09:25:42.999834+0000","last_clean":"2024-09-19T09:25:42.999834+0000","last_became_active":"2024-09-19T09:25:42.999450+0000","last_became_peered":"2024-09-19T09:25:42.999450+0000","last_unstale":"2024-09-19T09:25:42.999834+0000","last_undegraded":"2024-09-19T09:25:42.999834+0000","last_fullsized":"2024-09-19T09:25:42.999834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:00.509844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":82,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:31.058970+0000","last_change":"2024-09-19T09:25:27.275212+0000","last_active":"2024-09-19T09:26:31.058970+0000","last_peered":"2024-09-19T09:26:31.058970+0000","last_clean":"2024-09-19T09:26:31.058970+0000","last_became_active":"2024-09-19T09:25:27.274802+0000","last_became_peered":"2024-09-19T09:25:27.274802+0000","last_unstale":"2024-09-19T09:26:31.058970+0000","last_undegraded":"2024-09-19T09:26:31.058970+0000","last_fullsized":"2024-09-19T09:26:31.058970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:48.392026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986730+0000","last_change":"2024-09-19T09:25:29.304747+0000","last_active":"2024-09-19T09:25:42.986730+0000","last_peered":"2024-09-19T09:25:42.986730+0000","last_clean":"2024-09-19T09:25:42.986730+0000","last_became_active":"2024-09-19T09:25:29.304386+0000","last_became_peered":"2024-09-19T09:25:29.304386+0000","last_unstale":"2024-09-19T09:25:42.986730+0000","last_undegraded":"2024-09-19T09:25:42.986730+0000","last_fullsized":"2024-09-19T09:25:42.986730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.788697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988854+0000","last_change":"2024-09-19T09:25:27.323083+0000","last_active":"2024-09-19T09:25:42.988854+0000","last_peered":"2024-09-19T09:25:42.988854+0000","last_clean":"2024-09-19T09:25:42.988854+0000","last_became_active":"2024-09-19T09:25:27.322707+0000","last_became_peered":"2024-09-19T09:25:27.322707+0000","last_unstale":"2024-09-19T09:25:42.988854+0000","last_undegraded":"2024-09-19T09:25:42.988854+0000","last_fullsized":"2024-09-19T09:25:42.988854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:24.415116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986687+0000","last_change":"2024-09-19T09:25:29.292472+0000","last_active":"2024-09-19T09:25:42.986687+0000","last_peered":"2024-09-19T09:25:42.986687+0000","last_clean":"2024-09-19T09:25:42.986687+0000","last_became_active":"2024-09-19T09:25:29.292079+0000","last_became_peered":"2024-09-19T09:25:29.292079+0000","last_unstale":"2024-09-19T09:25:42.986687+0000","last_undegraded":"2024-09-19T09:25:42.986687+0000","last_fullsized":"2024-09-19T09:25:42.986687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:35.801594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991224+0000","last_change":"2024-09-19T09:25:27.324839+0000","last_active":"2024-09-19T09:25:42.991224+0000","last_peered":"2024-09-19T09:25:42.991224+0000","last_clean":"2024-09-19T09:25:42.991224+0000","last_became_active":"2024-09-19T09:25:27.324475+0000","last_became_peered":"2024-09-19T09:25:27.324475+0000","last_unstale":"2024-09-19T09:25:42.991224+0000","last_undegraded":"2024-09-19T09:25:42.991224+0000","last_fullsized":"2024-09-19T09:25:42.991224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:32.251753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985998+0000","last_change":"2024-09-19T09:25:29.293025+0000","last_active":"2024-09-19T09:25:42.985998+0000","last_peered":"2024-09-19T09:25:42.985998+0000","last_clean":"2024-09-19T09:25:42.985998+0000","last_became_active":"2024-09-19T09:25:29.292570+0000","last_became_peered":"2024-09-19T09:25:29.292570+0000","last_unstale":"2024-09-19T09:25:42.985998+0000","last_undegraded":"2024-09-19T09:25:42.985998+0000","last_fullsized":"2024-09-19T09:25:42.985998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:16.394524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:54.295611+0000","last_change":"2024-09-19T09:25:27.278100+0000","last_active":"2024-09-19T09:25:54.295611+0000","last_peered":"2024-09-19T09:25:54.295611+0000","last_clean":"2024-09-19T09:25:54.295611+0000","last_became_active":"2024-09-19T09:25:27.277769+0000","last_became_peered":"2024-09-19T09:25:27.277769+0000","last_unstale":"2024-09-19T09:25:54.295611+0000","last_undegraded":"2024-09-19T09:25:54.295611+0000","last_fullsized":"2024-09-19T09:25:54.295611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:31.885142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991325+0000","last_change":"2024-09-19T09:25:29.293651+0000","last_active":"2024-09-19T09:25:42.991325+0000","last_peered":"2024-09-19T09:25:42.991325+0000","last_clean":"2024-09-19T09:25:42.991325+0000","last_became_active":"2024-09-19T09:25:29.293317+0000","last_became_peered":"2024-09-19T09:25:29.293317+0000","last_unstale":"2024-09-19T09:25:42.991325+0000","last_undegraded":"2024-09-19T09:25:42.991325+0000","last_fullsized":"2024-09-19T09:25:42.991325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:40.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990030+0000","last_change":"2024-09-19T09:25:27.269367+0000","last_active":"2024-09-19T09:25:42.990030+0000","last_peered":"2024-09-19T09:25:42.990030+0000","last_clean":"2024-09-19T09:25:42.990030+0000","last_became_active":"2024-09-19T09:25:27.269011+0000","last_became_peered":"2024-09-19T09:25:27.269011+0000","last_unstale":"2024-09-19T09:25:42.990030+0000","last_undegraded":"2024-09-19T09:25:42.990030+0000","last_fullsized":"2024-09-19T09:25:42.990030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:00.930752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992257+0000","last_change":"2024-09-19T09:25:29.299134+0000","last_active":"2024-09-19T09:25:42.992257+0000","last_peered":"2024-09-19T09:25:42.992257+0000","last_clean":"2024-09-19T09:25:42.992257+0000","last_became_active":"2024-09-19T09:25:29.298648+0000","last_became_peered":"2024-09-19T09:25:29.298648+0000","last_unstale":"2024-09-19T09:25:42.992257+0000","last_undegraded":"2024-09-19T09:25:42.992257+0000","last_fullsized":"2024-09-19T09:25:42.992257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:39.627852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003686+0000","last_change":"2024-09-19T09:25:27.322861+0000","last_active":"2024-09-19T09:25:43.003686+0000","last_peered":"2024-09-19T09:25:43.003686+0000","last_clean":"2024-09-19T09:25:43.003686+0000","last_became_active":"2024-09-19T09:25:27.322560+0000","last_became_peered":"2024-09-19T09:25:27.322560+0000","last_unstale":"2024-09-19T09:25:43.003686+0000","last_undegraded":"2024-09-19T09:25:43.003686+0000","last_fullsized":"2024-09-19T09:25:43.003686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:35.993093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990868+0000","last_change":"2024-09-19T09:25:29.301680+0000","last_active":"2024-09-19T09:25:42.990868+0000","last_peered":"2024-09-19T09:25:42.990868+0000","last_clean":"2024-09-19T09:25:42.990868+0000","last_became_active":"2024-09-19T09:25:29.301331+0000","last_became_peered":"2024-09-19T09:25:29.301331+0000","last_unstale":"2024-09-19T09:25:42.990868+0000","last_undegraded":"2024-09-19T09:25:42.990868+0000","last_fullsized":"2024-09-19T09:25:42.990868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:57.364245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:27.272609+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:27.272169+0000","last_became_peered":"2024-09-19T09:25:27.272169+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.879474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988584+0000","last_change":"2024-09-19T09:25:29.296847+0000","last_active":"2024-09-19T09:25:42.988584+0000","last_peered":"2024-09-19T09:25:42.988584+0000","last_clean":"2024-09-19T09:25:42.988584+0000","last_became_active":"2024-09-19T09:25:29.296465+0000","last_became_peered":"2024-09-19T09:25:29.296465+0000","last_unstale":"2024-09-19T09:25:42.988584+0000","last_undegraded":"2024-09-19T09:25:42.988584+0000","last_fullsized":"2024-09-19T09:25:42.988584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:05:48.553252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.012954+0000","last_change":"2024-09-19T09:25:43.012532+0000","last_active":"2024-09-19T09:25:43.012954+0000","last_peered":"2024-09-19T09:25:43.012954+0000","last_clean":"2024-09-19T09:25:43.012954+0000","last_became_active":"2024-09-19T09:25:42.994996+0000","last_became_peered":"2024-09-19T09:25:42.994996+0000","last_unstale":"2024-09-19T09:25:43.012954+0000","last_undegraded":"2024-09-19T09:25:43.012954+0000","last_fullsized":"2024-09-19T09:25:43.012954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:25.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991072+0000","last_change":"2024-09-19T09:25:29.297633+0000","last_active":"2024-09-19T09:25:42.991072+0000","last_peered":"2024-09-19T09:25:42.991072+0000","last_clean":"2024-09-19T09:25:42.991072+0000","last_became_active":"2024-09-19T09:25:29.297284+0000","last_became_peered":"2024-09-19T09:25:29.297284+0000","last_unstale":"2024-09-19T09:25:42.991072+0000","last_undegraded":"2024-09-19T09:25:42.991072+0000","last_fullsized":"2024-09-19T09:25:42.991072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:50.582546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991172+0000","last_change":"2024-09-19T09:25:27.281778+0000","last_active":"2024-09-19T09:25:42.991172+0000","last_peered":"2024-09-19T09:25:42.991172+0000","last_clean":"2024-09-19T09:25:42.991172+0000","last_became_active":"2024-09-19T09:25:27.281280+0000","last_became_peered":"2024-09-19T09:25:27.281280+0000","last_unstale":"2024-09-19T09:25:42.991172+0000","last_undegraded":"2024-09-19T09:25:42.991172+0000","last_fullsized":"2024-09-19T09:25:42.991172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:46.811968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989808+0000","last_change":"2024-09-19T09:25:29.295663+0000","last_active":"2024-09-19T09:25:42.989808+0000","last_peered":"2024-09-19T09:25:42.989808+0000","last_clean":"2024-09-19T09:25:42.989808+0000","last_became_active":"2024-09-19T09:25:29.295146+0000","last_became_peered":"2024-09-19T09:25:29.295146+0000","last_unstale":"2024-09-19T09:25:42.989808+0000","last_undegraded":"2024-09-19T09:25:42.989808+0000","last_fullsized":"2024-09-19T09:25:42.989808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:36.749339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996010+0000","last_change":"2024-09-19T09:25:29.298004+0000","last_active":"2024-09-19T09:25:42.996010+0000","last_peered":"2024-09-19T09:25:42.996010+0000","last_clean":"2024-09-19T09:25:42.996010+0000","last_became_active":"2024-09-19T09:25:29.297600+0000","last_became_peered":"2024-09-19T09:25:29.297600+0000","last_unstale":"2024-09-19T09:25:42.996010+0000","last_undegraded":"2024-09-19T09:25:42.996010+0000","last_fullsized":"2024-09-19T09:25:42.996010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:09.738615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992323+0000","last_change":"2024-09-19T09:25:27.282876+0000","last_active":"2024-09-19T09:25:42.992323+0000","last_peered":"2024-09-19T09:25:42.992323+0000","last_clean":"2024-09-19T09:25:42.992323+0000","last_became_active":"2024-09-19T09:25:27.282201+0000","last_became_peered":"2024-09-19T09:25:27.282201+0000","last_unstale":"2024-09-19T09:25:42.992323+0000","last_undegraded":"2024-09-19T09:25:42.992323+0000","last_fullsized":"2024-09-19T09:25:42.992323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:08.508662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003650+0000","last_change":"2024-09-19T09:25:29.288379+0000","last_active":"2024-09-19T09:25:43.003650+0000","last_peered":"2024-09-19T09:25:43.003650+0000","last_clean":"2024-09-19T09:25:43.003650+0000","last_became_active":"2024-09-19T09:25:29.287917+0000","last_became_peered":"2024-09-19T09:25:29.287917+0000","last_unstale":"2024-09-19T09:25:43.003650+0000","last_undegraded":"2024-09-19T09:25:43.003650+0000","last_fullsized":"2024-09-19T09:25:43.003650+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:03.628233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991076+0000","last_change":"2024-09-19T09:25:27.279551+0000","last_active":"2024-09-19T09:25:42.991076+0000","last_peered":"2024-09-19T09:25:42.991076+0000","last_clean":"2024-09-19T09:25:42.991076+0000","last_became_active":"2024-09-19T09:25:27.278956+0000","last_became_peered":"2024-09-19T09:25:27.278956+0000","last_unstale":"2024-09-19T09:25:42.991076+0000","last_undegraded":"2024-09-19T09:25:42.991076+0000","last_fullsized":"2024-09-19T09:25:42.991076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:17.186453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992345+0000","last_change":"2024-09-19T09:25:29.288657+0000","last_active":"2024-09-19T09:25:42.992345+0000","last_peered":"2024-09-19T09:25:42.992345+0000","last_clean":"2024-09-19T09:25:42.992345+0000","last_became_active":"2024-09-19T09:25:29.288355+0000","last_became_peered":"2024-09-19T09:25:29.288355+0000","last_unstale":"2024-09-19T09:25:42.992345+0000","last_undegraded":"2024-09-19T09:25:42.992345+0000","last_fullsized":"2024-09-19T09:25:42.992345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:46.237558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991315+0000","last_change":"2024-09-19T09:25:27.323312+0000","last_active":"2024-09-19T09:25:42.991315+0000","last_peered":"2024-09-19T09:25:42.991315+0000","last_clean":"2024-09-19T09:25:42.991315+0000","last_became_active":"2024-09-19T09:25:27.322984+0000","last_became_peered":"2024-09-19T09:25:27.322984+0000","last_unstale":"2024-09-19T09:25:42.991315+0000","last_undegraded":"2024-09-19T09:25:42.991315+0000","last_fullsized":"2024-09-19T09:25:42.991315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:48.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991159+0000","last_change":"2024-09-19T09:25:29.301832+0000","last_active":"2024-09-19T09:25:42.991159+0000","last_peered":"2024-09-19T09:25:42.991159+0000","last_clean":"2024-09-19T09:25:42.991159+0000","last_became_active":"2024-09-19T09:25:29.301539+0000","last_became_peered":"2024-09-19T09:25:29.301539+0000","last_unstale":"2024-09-19T09:25:42.991159+0000","last_undegraded":"2024-09-19T09:25:42.991159+0000","last_fullsized":"2024-09-19T09:25:42.991159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:00.244948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'13","reported_seq":108,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:26:31.058386+0000","last_change":"2024-09-19T09:25:43.009668+0000","last_active":"2024-09-19T09:26:31.058386+0000","last_peered":"2024-09-19T09:26:31.058386+0000","last_clean":"2024-09-19T09:26:31.058386+0000","last_became_active":"2024-09-19T09:25:42.994966+0000","last_became_peered":"2024-09-19T09:25:42.994966+0000","last_unstale":"2024-09-19T09:26:31.058386+0000","last_undegraded":"2024-09-19T09:26:31.058386+0000","last_fullsized":"2024-09-19T09:26:31.058386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:04:05.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67758,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":568,"num_read_kb":357,"num_write":245,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1842,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":203274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612634,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":431057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841567,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1593344,"data_stored":1081178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005091,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234024,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":430300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462958,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1022934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593779,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":901120,"data_stored":430032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1022498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":909312,"data_stored":431456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":489759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":490709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":921600,"data_stored":431371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:26:34.402 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T09:26:34.402 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T09:26:34.403 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T09:26:34.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T09:26:34.664 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:35.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:34 smithi142 ceph-mon[35032]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:26:35.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:34 smithi142 ceph-mon[35032]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:26:35.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:34 smithi097 ceph-mon[34864]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:26:35.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:34 smithi097 ceph-mon[34864]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:26:35.175 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:34 smithi016 ceph-mon[30979]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:26:35.175 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:34 smithi016 ceph-mon[30979]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:26:35.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.256+0000 7f82c14b2640 1 -- 172.21.15.16:0/1506750394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc100a80 msgr2=0x7f82bc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.256+0000 7f82c14b2640 1 --2- 172.21.15.16:0/1506750394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc100a80 0x7f82bc100ee0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009930 tx=0x7f82b002a420 comp rx=0 tx=0).stop 2024-09-19T09:26:35.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.256+0000 7f82c14b2640 1 -- 172.21.15.16:0/1506750394 shutdown_connections 2024-09-19T09:26:35.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.256+0000 7f82c14b2640 1 --2- 172.21.15.16:0/1506750394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82bc101420 0x7f82bc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.256+0000 7f82c14b2640 1 --2- 172.21.15.16:0/1506750394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc100a80 0x7f82bc100ee0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.256+0000 7f82c14b2640 1 --2- 172.21.15.16:0/1506750394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc106a80 0x7f82bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.256+0000 7f82c14b2640 1 -- 172.21.15.16:0/1506750394 >> 172.21.15.16:0/1506750394 conn(0x7f82bc0fc7a0 msgr2=0x7f82bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:35.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.256+0000 7f82c14b2640 1 -- 172.21.15.16:0/1506750394 shutdown_connections 2024-09-19T09:26:35.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.256+0000 7f82c14b2640 1 -- 172.21.15.16:0/1506750394 wait complete. 2024-09-19T09:26:35.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.257+0000 7f82c14b2640 1 Processor -- start 2024-09-19T09:26:35.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.257+0000 7f82c14b2640 1 -- start start 2024-09-19T09:26:35.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.257+0000 7f82c14b2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82bc100a80 0x7f82bc19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.257+0000 7f82c14b2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc101420 0x7f82bc19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.257+0000 7f82c14b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc106a80 0x7f82bc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.257+0000 7f82c14b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc110e80 con 0x7f82bc106a80 2024-09-19T09:26:35.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.257+0000 7f82c14b2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc110d00 con 0x7f82bc100a80 2024-09-19T09:26:35.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.257+0000 7f82c14b2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc111000 con 0x7f82bc101420 2024-09-19T09:26:35.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc106a80 0x7f82bc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc106a80 0x7f82bc19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55446/0 (socket says 172.21.15.16:55446) 2024-09-19T09:26:35.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82bb7fe640 1 -- 172.21.15.16:0/794161014 learned_addr learned my addr 172.21.15.16:0/794161014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:35.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82baffd640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82bc100a80 0x7f82bc19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82bb7fe640 1 -- 172.21.15.16:0/794161014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc101420 msgr2=0x7f82bc19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82ba7fc640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc101420 0x7f82bc19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82bb7fe640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc101420 0x7f82bc19abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82bb7fe640 1 -- 172.21.15.16:0/794161014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82bc100a80 msgr2=0x7f82bc19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82bb7fe640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82bc100a80 0x7f82bc19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82bb7fe640 1 -- 172.21.15.16:0/794161014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc19f630 con 0x7f82bc106a80 2024-09-19T09:26:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82baffd640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82bc100a80 0x7f82bc19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82bb7fe640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc106a80 0x7f82bc19ef50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f82b400eaa0 tx=0x7f82b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82a3fff640 1 -- 172.21.15.16:0/794161014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b400cdb0 con 0x7f82bc106a80 2024-09-19T09:26:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82a3fff640 1 -- 172.21.15.16:0/794161014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82b4005020 con 0x7f82bc106a80 2024-09-19T09:26:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82c14b2640 1 -- 172.21.15.16:0/794161014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82bc19f920 con 0x7f82bc106a80 2024-09-19T09:26:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82a3fff640 1 -- 172.21.15.16:0/794161014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b40116b0 con 0x7f82bc106a80 2024-09-19T09:26:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.258+0000 7f82c14b2640 1 -- 172.21.15.16:0/794161014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82bc1a7200 con 0x7f82bc106a80 2024-09-19T09:26:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.259+0000 7f82c14b2640 1 -- 172.21.15.16:0/794161014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82bc100ee0 con 0x7f82bc106a80 2024-09-19T09:26:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.260+0000 7f82a3fff640 1 -- 172.21.15.16:0/794161014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f82b4005560 con 0x7f82bc106a80 2024-09-19T09:26:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.261+0000 7f82a3fff640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f82a40a3350 0x7f82a40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.262+0000 7f82baffd640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f82a40a3350 0x7f82a40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.262+0000 7f82a3fff640 1 -- 172.21.15.16:0/794161014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f82b40d0130 con 0x7f82bc106a80 2024-09-19T09:26:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.262+0000 7f82baffd640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f82a40a3350 0x7f82a40a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0058c0 tx=0x7f82ac005cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.265+0000 7f82a3fff640 1 -- 172.21.15.16:0/794161014 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f82b4016030 con 0x7f82bc106a80 2024-09-19T09:26:35.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.478+0000 7f82c14b2640 1 -- 172.21.15.16:0/794161014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f82bc19b9e0 con 0x7f82bc106a80 2024-09-19T09:26:35.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.479+0000 7f82a3fff640 1 -- 172.21.15.16:0/794161014 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f82bc19b9e0 con 0x7f82bc106a80 2024-09-19T09:26:35.480 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:26:35.480 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T09:26:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.480+0000 7f82c14b2640 1 -- 172.21.15.16:0/794161014 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f82a40a3350 msgr2=0x7f82a40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.481+0000 7f82c14b2640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f82a40a3350 0x7f82a40a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0058c0 tx=0x7f82ac005cb0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.481+0000 7f82c14b2640 1 -- 172.21.15.16:0/794161014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc106a80 msgr2=0x7f82bc19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.481+0000 7f82c14b2640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc106a80 0x7f82bc19ef50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f82b400eaa0 tx=0x7f82b400ef60 comp rx=0 tx=0).stop 2024-09-19T09:26:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.481+0000 7f82c14b2640 1 -- 172.21.15.16:0/794161014 shutdown_connections 2024-09-19T09:26:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.481+0000 7f82c14b2640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f82a40a3350 0x7f82a40a5810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.481+0000 7f82c14b2640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc101420 0x7f82bc19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.481+0000 7f82c14b2640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82bc106a80 0x7f82bc19ef50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.481+0000 7f82c14b2640 1 --2- 172.21.15.16:0/794161014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82bc100a80 0x7f82bc19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.481+0000 7f82c14b2640 1 -- 172.21.15.16:0/794161014 >> 172.21.15.16:0/794161014 conn(0x7f82bc0fc7a0 msgr2=0x7f82bc06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:35.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.481+0000 7f82c14b2640 1 -- 172.21.15.16:0/794161014 shutdown_connections 2024-09-19T09:26:35.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:35.481+0000 7f82c14b2640 1 -- 172.21.15.16:0/794161014 wait complete. 2024-09-19T09:26:36.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:35 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/794161014' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T09:26:36.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:35 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/794161014' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T09:26:36.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:35 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/794161014' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T09:26:36.171 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T09:26:36.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:26:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:36.824 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:36 smithi016 ceph-mon[30979]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:36.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.905+0000 7f3f559b9640 1 -- 172.21.15.16:0/2798602601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f5010c170 msgr2=0x7f3f5010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:36.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.905+0000 7f3f559b9640 1 --2- 172.21.15.16:0/2798602601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f5010c170 0x7f3f5010e610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009ca0 tx=0x7f3f4802a950 comp rx=0 tx=0).stop 2024-09-19T09:26:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.905+0000 7f3f559b9640 1 -- 172.21.15.16:0/2798602601 shutdown_connections 2024-09-19T09:26:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.905+0000 7f3f559b9640 1 --2- 172.21.15.16:0/2798602601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f50103660 0x7f3f5010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.905+0000 7f3f559b9640 1 --2- 172.21.15.16:0/2798602601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f5010c170 0x7f3f5010e610 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.905+0000 7f3f559b9640 1 --2- 172.21.15.16:0/2798602601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f50102d40 0x7f3f50103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.905+0000 7f3f559b9640 1 -- 172.21.15.16:0/2798602601 >> 172.21.15.16:0/2798602601 conn(0x7f3f500fc7a0 msgr2=0x7f3f500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.905+0000 7f3f559b9640 1 -- 172.21.15.16:0/2798602601 shutdown_connections 2024-09-19T09:26:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.905+0000 7f3f559b9640 1 -- 172.21.15.16:0/2798602601 wait complete. 2024-09-19T09:26:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.906+0000 7f3f559b9640 1 Processor -- start 2024-09-19T09:26:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.906+0000 7f3f559b9640 1 -- start start 2024-09-19T09:26:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.906+0000 7f3f559b9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f50102d40 0x7f3f50198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.906+0000 7f3f559b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f50103660 0x7f3f501989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.906+0000 7f3f559b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f5010c170 0x7f3f5019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.906+0000 7f3f559b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f50111fd0 con 0x7f3f5010c170 2024-09-19T09:26:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.906+0000 7f3f559b9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3f50111e50 con 0x7f3f50103660 2024-09-19T09:26:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.906+0000 7f3f559b9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3f50112150 con 0x7f3f50102d40 2024-09-19T09:26:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.906+0000 7f3f4f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f5010c170 0x7f3f5019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.906+0000 7f3f4f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f5010c170 0x7f3f5019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52944/0 (socket says 172.21.15.16:52944) 2024-09-19T09:26:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.906+0000 7f3f4f7fe640 1 -- 172.21.15.16:0/957753042 learned_addr learned my addr 172.21.15.16:0/957753042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.906+0000 7f3f4effd640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f50102d40 0x7f3f50198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.906+0000 7f3f4e7fc640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f50103660 0x7f3f501989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.907+0000 7f3f4f7fe640 1 -- 172.21.15.16:0/957753042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f50102d40 msgr2=0x7f3f50198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.907+0000 7f3f4f7fe640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f50102d40 0x7f3f50198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.907+0000 7f3f4f7fe640 1 -- 172.21.15.16:0/957753042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f50103660 msgr2=0x7f3f501989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.907+0000 7f3f4f7fe640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f50103660 0x7f3f501989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.907+0000 7f3f4f7fe640 1 -- 172.21.15.16:0/957753042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5019d420 con 0x7f3f5010c170 2024-09-19T09:26:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.907+0000 7f3f4f7fe640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f5010c170 0x7f3f5019cd40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3f48004af0 tx=0x7f3f4802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.907+0000 7f3f549b7640 1 -- 172.21.15.16:0/957753042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f48034040 con 0x7f3f5010c170 2024-09-19T09:26:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.908+0000 7f3f549b7640 1 -- 172.21.15.16:0/957753042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f48030920 con 0x7f3f5010c170 2024-09-19T09:26:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.908+0000 7f3f549b7640 1 -- 172.21.15.16:0/957753042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f48004740 con 0x7f3f5010c170 2024-09-19T09:26:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.908+0000 7f3f559b9640 1 -- 172.21.15.16:0/957753042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5019d6b0 con 0x7f3f5010c170 2024-09-19T09:26:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.908+0000 7f3f559b9640 1 -- 172.21.15.16:0/957753042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5019db40 con 0x7f3f5010c170 2024-09-19T09:26:36.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.909+0000 7f3f559b9640 1 -- 172.21.15.16:0/957753042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f50103120 con 0x7f3f5010c170 2024-09-19T09:26:36.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.910+0000 7f3f549b7640 1 -- 172.21.15.16:0/957753042 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3f48037020 con 0x7f3f5010c170 2024-09-19T09:26:36.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.911+0000 7f3f549b7640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3f3c0a3420 0x7f3f3c0a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:36.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.911+0000 7f3f4effd640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3f3c0a3420 0x7f3f3c0a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:36.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.911+0000 7f3f549b7640 1 -- 172.21.15.16:0/957753042 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3f480f0640 con 0x7f3f5010c170 2024-09-19T09:26:36.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.912+0000 7f3f4effd640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3f3c0a3420 0x7f3f3c0a58e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009990 tx=0x7f3f40009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:36.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:36.915+0000 7f3f549b7640 1 -- 172.21.15.16:0/957753042 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3f480bd2d0 con 0x7f3f5010c170 2024-09-19T09:26:37.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:36 smithi142 ceph-mon[35032]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.129+0000 7f3f559b9640 1 -- 172.21.15.16:0/957753042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f5002d7c0 con 0x7f3f5010c170 2024-09-19T09:26:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.129+0000 7f3f549b7640 1 -- 172.21.15.16:0/957753042 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11967 (secure 0 0 0) 0x7f3f480bcbd0 con 0x7f3f5010c170 2024-09-19T09:26:37.131 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:26:37.131 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"btime":"2024-09-19T09:25:40:972080+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:25:40.972058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24484,24470,24428,24424]},"id":1}]} 2024-09-19T09:26:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-19T09:26:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.131+0000 7f3f559b9640 1 -- 172.21.15.16:0/957753042 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3f3c0a3420 msgr2=0x7f3f3c0a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.131+0000 7f3f559b9640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3f3c0a3420 0x7f3f3c0a58e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009990 tx=0x7f3f40009110 comp rx=0 tx=0).stop 2024-09-19T09:26:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.131+0000 7f3f559b9640 1 -- 172.21.15.16:0/957753042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f5010c170 msgr2=0x7f3f5019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.131+0000 7f3f559b9640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f5010c170 0x7f3f5019cd40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3f48004af0 tx=0x7f3f4802f5b0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.132+0000 7f3f559b9640 1 -- 172.21.15.16:0/957753042 shutdown_connections 2024-09-19T09:26:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.132+0000 7f3f559b9640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f5010c170 0x7f3f5019cd40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.132+0000 7f3f559b9640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f50103660 0x7f3f501989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.132+0000 7f3f559b9640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3f3c0a3420 0x7f3f3c0a58e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.132+0000 7f3f559b9640 1 --2- 172.21.15.16:0/957753042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f50102d40 0x7f3f50198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.132+0000 7f3f559b9640 1 -- 172.21.15.16:0/957753042 >> 172.21.15.16:0/957753042 conn(0x7f3f500fc7a0 msgr2=0x7f3f500fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.132+0000 7f3f559b9640 1 -- 172.21.15.16:0/957753042 shutdown_connections 2024-09-19T09:26:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:37.132+0000 7f3f559b9640 1 -- 172.21.15.16:0/957753042 wait complete. 2024-09-19T09:26:37.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:36 smithi097 ceph-mon[34864]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:37.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:26:38.056 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:38.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:37 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/957753042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:38.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:37 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/957753042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:38.172 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:37 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/957753042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.522+0000 7f8785854640 1 -- 172.21.15.16:0/3803904742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8780100480 msgr2=0x7f8780100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.522+0000 7f8785854640 1 --2- 172.21.15.16:0/3803904742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8780100480 0x7f8780100860 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8770009900 tx=0x7f877002a410 comp rx=0 tx=0).stop 2024-09-19T09:26:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.522+0000 7f8785854640 1 -- 172.21.15.16:0/3803904742 shutdown_connections 2024-09-19T09:26:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.522+0000 7f8785854640 1 --2- 172.21.15.16:0/3803904742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8780100e30 0x7f8780104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.522+0000 7f8785854640 1 --2- 172.21.15.16:0/3803904742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87801055b0 0x7f878010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.522+0000 7f8785854640 1 --2- 172.21.15.16:0/3803904742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8780100480 0x7f8780100860 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.522+0000 7f8785854640 1 -- 172.21.15.16:0/3803904742 >> 172.21.15.16:0/3803904742 conn(0x7f8780077210 msgr2=0x7f87800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.523+0000 7f8785854640 1 -- 172.21.15.16:0/3803904742 shutdown_connections 2024-09-19T09:26:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.523+0000 7f8785854640 1 -- 172.21.15.16:0/3803904742 wait complete. 2024-09-19T09:26:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.523+0000 7f8785854640 1 Processor -- start 2024-09-19T09:26:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.523+0000 7f8785854640 1 -- start start 2024-09-19T09:26:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.523+0000 7f8785854640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8780100480 0x7f8780198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.523+0000 7f8785854640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8780100e30 0x7f8780198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.523+0000 7f8785854640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87801055b0 0x7f878019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.523+0000 7f8785854640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f878010fd90 con 0x7f8780100480 2024-09-19T09:26:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.523+0000 7f8785854640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f878010fc10 con 0x7f8780100e30 2024-09-19T09:26:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.523+0000 7f8785854640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f878010ff10 con 0x7f87801055b0 2024-09-19T09:26:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.523+0000 7f877effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8780100480 0x7f8780198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f877effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8780100480 0x7f8780198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52972/0 (socket says 172.21.15.16:52972) 2024-09-19T09:26:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f877effd640 1 -- 172.21.15.16:0/4056354996 learned_addr learned my addr 172.21.15.16:0/4056354996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f8777fff640 1 --2- 172.21.15.16:0/4056354996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8780100e30 0x7f8780198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f877effd640 1 -- 172.21.15.16:0/4056354996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87801055b0 msgr2=0x7f878019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f877effd640 1 --2- 172.21.15.16:0/4056354996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87801055b0 0x7f878019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f877effd640 1 -- 172.21.15.16:0/4056354996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8780100e30 msgr2=0x7f8780198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f877effd640 1 --2- 172.21.15.16:0/4056354996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8780100e30 0x7f8780198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f877effd640 1 -- 172.21.15.16:0/4056354996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878019d3f0 con 0x7f8780100480 2024-09-19T09:26:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f877effd640 1 --2- 172.21.15.16:0/4056354996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8780100480 0x7f8780198440 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8770006010 tx=0x7f87700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f8784852640 1 -- 172.21.15.16:0/4056354996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877003f030 con 0x7f8780100480 2024-09-19T09:26:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f8784852640 1 -- 172.21.15.16:0/4056354996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8770003ce0 con 0x7f8780100480 2024-09-19T09:26:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f8785854640 1 -- 172.21.15.16:0/4056354996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878019d680 con 0x7f8780100480 2024-09-19T09:26:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f8784852640 1 -- 172.21.15.16:0/4056354996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770035070 con 0x7f8780100480 2024-09-19T09:26:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.524+0000 7f8785854640 1 -- 172.21.15.16:0/4056354996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87801a4fc0 con 0x7f8780100480 2024-09-19T09:26:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.525+0000 7f8785854640 1 -- 172.21.15.16:0/4056354996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8780101960 con 0x7f8780100480 2024-09-19T09:26:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.526+0000 7f8784852640 1 -- 172.21.15.16:0/4056354996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8770042030 con 0x7f8780100480 2024-09-19T09:26:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.527+0000 7f8784852640 1 --2- 172.21.15.16:0/4056354996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f876c0a32d0 0x7f876c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.527+0000 7f8777fff640 1 --2- 172.21.15.16:0/4056354996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f876c0a32d0 0x7f876c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.528+0000 7f8784852640 1 -- 172.21.15.16:0/4056354996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f87700f04f0 con 0x7f8780100480 2024-09-19T09:26:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.528+0000 7f8777fff640 1 --2- 172.21.15.16:0/4056354996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f876c0a32d0 0x7f876c0a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8768005fd0 tx=0x7f8768005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:38.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.530+0000 7f8784852640 1 -- 172.21.15.16:0/4056354996 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f87700bd130 con 0x7f8780100480 2024-09-19T09:26:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.742+0000 7f8785854640 1 -- 172.21.15.16:0/4056354996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8780100860 con 0x7f8780100480 2024-09-19T09:26:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.742+0000 7f8784852640 1 -- 172.21.15.16:0/4056354996 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11967 (secure 0 0 0) 0x7f8780100860 con 0x7f8780100480 2024-09-19T09:26:38.744 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:26:38.744 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"btime":"2024-09-19T09:25:40:972080+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:25:40.972058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24484,24470,24428,24424]},"id":1}]} 2024-09-19T09:26:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-19T09:26:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.745+0000 7f8785854640 1 -- 172.21.15.16:0/4056354996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f876c0a32d0 msgr2=0x7f876c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.745+0000 7f8785854640 1 --2- 172.21.15.16:0/4056354996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f876c0a32d0 0x7f876c0a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8768005fd0 tx=0x7f8768005850 comp rx=0 tx=0).stop 2024-09-19T09:26:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.745+0000 7f8785854640 1 -- 172.21.15.16:0/4056354996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8780100480 msgr2=0x7f8780198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.745+0000 7f8785854640 1 --2- 172.21.15.16:0/4056354996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8780100480 0x7f8780198440 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8770006010 tx=0x7f87700042f0 comp rx=0 tx=0).stop 2024-09-19T09:26:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.745+0000 7f8785854640 1 -- 172.21.15.16:0/4056354996 shutdown_connections 2024-09-19T09:26:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.745+0000 7f8785854640 1 --2- 172.21.15.16:0/4056354996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f876c0a32d0 0x7f876c0a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.745+0000 7f8785854640 1 --2- 172.21.15.16:0/4056354996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87801055b0 0x7f878019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.745+0000 7f8785854640 1 --2- 172.21.15.16:0/4056354996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8780100e30 0x7f8780198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.745+0000 7f8785854640 1 --2- 172.21.15.16:0/4056354996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8780100480 0x7f8780198440 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.745+0000 7f8785854640 1 -- 172.21.15.16:0/4056354996 >> 172.21.15.16:0/4056354996 conn(0x7f8780077210 msgr2=0x7f87800febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.745+0000 7f8785854640 1 -- 172.21.15.16:0/4056354996 shutdown_connections 2024-09-19T09:26:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:38.745+0000 7f8785854640 1 -- 172.21.15.16:0/4056354996 wait complete. 2024-09-19T09:26:39.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:38 smithi016 ceph-mon[30979]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:26:39.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:38 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/4056354996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:39.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:38 smithi142 ceph-mon[35032]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:26:39.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:38 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/4056354996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:39.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:38 smithi097 ceph-mon[34864]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:26:39.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:38 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/4056354996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:39.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:26:39.632 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.039+0000 7f3a08c09640 1 -- 172.21.15.16:0/1791169566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a04101de0 msgr2=0x7f3a041021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.039+0000 7f3a08c09640 1 --2- 172.21.15.16:0/1791169566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a04101de0 0x7f3a041021c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009900 tx=0x7f39f402a410 comp rx=0 tx=0).stop 2024-09-19T09:26:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.039+0000 7f3a08c09640 1 -- 172.21.15.16:0/1791169566 shutdown_connections 2024-09-19T09:26:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.039+0000 7f3a08c09640 1 --2- 172.21.15.16:0/1791169566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a04102790 0x7f3a041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.039+0000 7f3a08c09640 1 --2- 172.21.15.16:0/1791169566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a04106f10 0x7f3a0410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.039+0000 7f3a08c09640 1 --2- 172.21.15.16:0/1791169566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a04101de0 0x7f3a041021c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.039+0000 7f3a08c09640 1 -- 172.21.15.16:0/1791169566 >> 172.21.15.16:0/1791169566 conn(0x7f3a040fd8c0 msgr2=0x7f3a040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.039+0000 7f3a08c09640 1 -- 172.21.15.16:0/1791169566 shutdown_connections 2024-09-19T09:26:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.039+0000 7f3a08c09640 1 -- 172.21.15.16:0/1791169566 wait complete. 2024-09-19T09:26:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.040+0000 7f3a08c09640 1 Processor -- start 2024-09-19T09:26:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.040+0000 7f3a08c09640 1 -- start start 2024-09-19T09:26:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a08c09640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a04101de0 0x7f3a040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a08c09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a04102790 0x7f3a04074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a08c09640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a04106f10 0x7f3a04074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a08c09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04112010 con 0x7f3a04102790 2024-09-19T09:26:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a08c09640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04111e90 con 0x7f3a04106f10 2024-09-19T09:26:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a08c09640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04112190 con 0x7f3a04101de0 2024-09-19T09:26:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a01d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a04102790 0x7f3a04074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a01d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a04102790 0x7f3a04074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52986/0 (socket says 172.21.15.16:52986) 2024-09-19T09:26:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a01d9b640 1 -- 172.21.15.16:0/955401148 learned_addr learned my addr 172.21.15.16:0/955401148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a02d9d640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a04106f10 0x7f3a04074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a0259c640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a04101de0 0x7f3a040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a01d9b640 1 -- 172.21.15.16:0/955401148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a04101de0 msgr2=0x7f3a040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a01d9b640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a04101de0 0x7f3a040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a01d9b640 1 -- 172.21.15.16:0/955401148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a04106f10 msgr2=0x7f3a04074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a01d9b640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a04106f10 0x7f3a04074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:40.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a01d9b640 1 -- 172.21.15.16:0/955401148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a041a6a70 con 0x7f3a04102790 2024-09-19T09:26:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.041+0000 7f3a01d9b640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a04102790 0x7f3a04074a30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f39f800b4a0 tx=0x7f39f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.042+0000 7f39f37fe640 1 -- 172.21.15.16:0/955401148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8014070 con 0x7f3a04102790 2024-09-19T09:26:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.042+0000 7f3a08c09640 1 -- 172.21.15.16:0/955401148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a041a6d60 con 0x7f3a04102790 2024-09-19T09:26:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.042+0000 7f39f37fe640 1 -- 172.21.15.16:0/955401148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f8010af0 con 0x7f3a04102790 2024-09-19T09:26:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.042+0000 7f39f37fe640 1 -- 172.21.15.16:0/955401148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8017d10 con 0x7f3a04102790 2024-09-19T09:26:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.042+0000 7f3a08c09640 1 -- 172.21.15.16:0/955401148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a041a71f0 con 0x7f3a04102790 2024-09-19T09:26:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.043+0000 7f3a08c09640 1 -- 172.21.15.16:0/955401148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a04102240 con 0x7f3a04102790 2024-09-19T09:26:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.044+0000 7f39f37fe640 1 -- 172.21.15.16:0/955401148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f39f8017070 con 0x7f3a04102790 2024-09-19T09:26:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.045+0000 7f39f37fe640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f39ec0a32d0 0x7f39ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.045+0000 7f3a0259c640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f39ec0a32d0 0x7f39ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.045+0000 7f39f37fe640 1 -- 172.21.15.16:0/955401148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f39f80cecb0 con 0x7f3a04102790 2024-09-19T09:26:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.045+0000 7f3a0259c640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f39ec0a32d0 0x7f39ec0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f39f4000c00 tx=0x7f39f40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.049+0000 7f39f37fe640 1 -- 172.21.15.16:0/955401148 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f39f809b330 con 0x7f3a04102790 2024-09-19T09:26:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.230+0000 7f3a08c09640 1 -- 172.21.15.16:0/955401148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a0402d7c0 con 0x7f3a04102790 2024-09-19T09:26:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.232+0000 7f39f37fe640 1 -- 172.21.15.16:0/955401148 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22665 (secure 0 0 0) 0x7f39f809f630 con 0x7f3a04102790 2024-09-19T09:26:40.234 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:26:40.234 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","created":"2024-09-19T09:17:48.242238+0000","modified":"2024-09-19T09:25:42.982671+0000","last_up_change":"2024-09-19T09:24:40.190289+0000","last_in_change":"2024-09-19T09:24:17.190380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:20:57.748686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:25:25.694451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:25:28.262997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbf3b28b-2282-4057-9e65-19ceb78ff19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6803","nonce":1578957623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6805","nonce":1578957623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6809","nonce":1578957623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6807","nonce":1578957623}]},"public_addr":"172.21.15.16:6803/1578957623","cluster_addr":"172.21.15.16:6805/1578957623","heartbeat_back_addr":"172.21.15.16:6809/1578957623","heartbeat_front_addr":"172.21.15.16:6807/1578957623","state":["exists","up"]},{"osd":1,"uuid":"c2517aeb-1d79-41ad-a180-369eecb9732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6801","nonce":3804152349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6803","nonce":3804152349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6807","nonce":3804152349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6805","nonce":3804152349}]},"public_addr":"172.21.15.97:6801/3804152349","cluster_addr":"172.21.15.97:6803/3804152349","heartbeat_back_addr":"172.21.15.97:6807/3804152349","heartbeat_front_addr":"172.21.15.97:6805/3804152349","state":["exists","up"]},{"osd":2,"uuid":"3c367518-5085-471a-98cb-ec5bf5facdf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6801","nonce":2692613951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6803","nonce":2692613951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6807","nonce":2692613951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6805","nonce":2692613951}]},"public_addr":"172.21.15.142:6801/2692613951","cluster_addr":"172.21.15.142:6803/2692613951","heartbeat_back_addr":"172.21.15.142:6807/2692613951","heartbeat_front_addr":"172.21.15.142:6805/2692613951","state":["exists","up"]},{"osd":3,"uuid":"e7118925-f0fa-46b3-8552-3e3bb76a6489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6811","nonce":3169290004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6813","nonce":3169290004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6817","nonce":3169290004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6815","nonce":3169290004}]},"public_addr":"172.21.15.16:6811/3169290004","cluster_addr":"172.21.15.16:6813/3169290004","heartbeat_back_addr":"172.21.15.16:6817/3169290004","heartbeat_front_addr":"172.21.15.16:6815/3169290004","state":["exists","up"]},{"osd":4,"uuid":"dc1964f7-af9c-4853-843b-9818f87c7768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6809","nonce":3491353825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6811","nonce":3491353825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6815","nonce":3491353825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6813","nonce":3491353825}]},"public_addr":"172.21.15.97:6809/3491353825","cluster_addr":"172.21.15.97:6811/3491353825","heartbeat_back_addr":"172.21.15.97:6815/3491353825","heartbeat_front_addr":"172.21.15.97:6813/3491353825","state":["exists","up"]},{"osd":5,"uuid":"4d7fbafc-d9ba-4036-a89d-aa295b8a23a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6809","nonce":577099986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6811","nonce":577099986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6815","nonce":577099986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6813","nonce":577099986}]},"public_addr":"172.21.15.142:6809/577099986","cluster_addr":"172.21.15.142:6811/577099986","heartbeat_back_addr":"172.21.15.142:6815/577099986","heartbeat_front_addr":"172.21.15.142:6813/577099986","state":["exists","up"]},{"osd":6,"uuid":"e3f346fc-c027-44f8-bd37-bccf6ec09e3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6819","nonce":4053539536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6821","nonce":4053539536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6825","nonce":4053539536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6823","nonce":4053539536}]},"public_addr":"172.21.15.16:6819/4053539536","cluster_addr":"172.21.15.16:6821/4053539536","heartbeat_back_addr":"172.21.15.16:6825/4053539536","heartbeat_front_addr":"172.21.15.16:6823/4053539536","state":["exists","up"]},{"osd":7,"uuid":"d390b547-8949-4d4b-8481-c49d6ae7e424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6817","nonce":1157122670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6819","nonce":1157122670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6823","nonce":1157122670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6821","nonce":1157122670}]},"public_addr":"172.21.15.97:6817/1157122670","cluster_addr":"172.21.15.97:6819/1157122670","heartbeat_back_addr":"172.21.15.97:6823/1157122670","heartbeat_front_addr":"172.21.15.97:6821/1157122670","state":["exists","up"]},{"osd":8,"uuid":"78a476c4-5deb-4f9e-a266-db4493850732","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6817","nonce":643795004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6819","nonce":643795004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6823","nonce":643795004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6821","nonce":643795004}]},"public_addr":"172.21.15.142:6817/643795004","cluster_addr":"172.21.15.142:6819/643795004","heartbeat_back_addr":"172.21.15.142:6823/643795004","heartbeat_front_addr":"172.21.15.142:6821/643795004","state":["exists","up"]},{"osd":9,"uuid":"0415d187-4a77-42f7-b8bb-2d0b52e9cb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6827","nonce":2651132360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6829","nonce":2651132360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2651132360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6831","nonce":2651132360}]},"public_addr":"172.21.15.16:6827/2651132360","cluster_addr":"172.21.15.16:6829/2651132360","heartbeat_back_addr":"172.21.15.16:6833/2651132360","heartbeat_front_addr":"172.21.15.16:6831/2651132360","state":["exists","up"]},{"osd":10,"uuid":"6fa78f96-cdfe-42a0-8d27-0705a0c7df8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6825","nonce":1848371486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6827","nonce":1848371486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6831","nonce":1848371486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6829","nonce":1848371486}]},"public_addr":"172.21.15.97:6825/1848371486","cluster_addr":"172.21.15.97:6827/1848371486","heartbeat_back_addr":"172.21.15.97:6831/1848371486","heartbeat_front_addr":"172.21.15.97:6829/1848371486","state":["exists","up"]},{"osd":11,"uuid":"d27f20e3-84ce-470c-9889-cd86906b158d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6825","nonce":155083878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6827","nonce":155083878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6831","nonce":155083878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6829","nonce":155083878}]},"public_addr":"172.21.15.142:6825/155083878","cluster_addr":"172.21.15.142:6827/155083878","heartbeat_back_addr":"172.21.15.142:6831/155083878","heartbeat_front_addr":"172.21.15.142:6829/155083878","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:05.088764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:29.928529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:55.236930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:20.379471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:44.984151+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:10.410074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:35.233553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:59.928700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:24.503069+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:49.853073+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:14.006841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:38.546467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3710267992":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3740275561":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/2755421444":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/357390846":"2024-09-20T09:18:17.070034+0000","172.21.15.16:6800/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3104087402":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3546947161":"2024-09-20T09:18:17.070034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:26:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.235+0000 7f3a08c09640 1 -- 172.21.15.16:0/955401148 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f39ec0a32d0 msgr2=0x7f39ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.235+0000 7f3a08c09640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f39ec0a32d0 0x7f39ec0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f39f4000c00 tx=0x7f39f40058d0 comp rx=0 tx=0).stop 2024-09-19T09:26:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.235+0000 7f3a08c09640 1 -- 172.21.15.16:0/955401148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a04102790 msgr2=0x7f3a04074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.235+0000 7f3a08c09640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a04102790 0x7f3a04074a30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f39f800b4a0 tx=0x7f39f800b960 comp rx=0 tx=0).stop 2024-09-19T09:26:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.235+0000 7f3a08c09640 1 -- 172.21.15.16:0/955401148 shutdown_connections 2024-09-19T09:26:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.235+0000 7f3a08c09640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a04106f10 0x7f3a04074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.235+0000 7f3a08c09640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a04102790 0x7f3a04074a30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.235+0000 7f3a08c09640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f39ec0a32d0 0x7f39ec0a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.235+0000 7f3a08c09640 1 --2- 172.21.15.16:0/955401148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a04101de0 0x7f3a040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.235+0000 7f3a08c09640 1 -- 172.21.15.16:0/955401148 >> 172.21.15.16:0/955401148 conn(0x7f3a040fd8c0 msgr2=0x7f3a040fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.235+0000 7f3a08c09640 1 -- 172.21.15.16:0/955401148 shutdown_connections 2024-09-19T09:26:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:40.235+0000 7f3a08c09640 1 -- 172.21.15.16:0/955401148 wait complete. 2024-09-19T09:26:40.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:26:41.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:40 smithi142 ceph-mon[35032]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:26:41.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:40 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/955401148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:26:41.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:40 smithi097 ceph-mon[34864]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:26:41.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:40 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/955401148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:26:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:41.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:40 smithi016 ceph-mon[30979]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:26:41.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:40 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/955401148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:26:41.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.658+0000 7f50e87ce640 1 -- 172.21.15.16:0/273922275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e01036e0 msgr2=0x7f50e0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.658+0000 7f50e87ce640 1 --2- 172.21.15.16:0/273922275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e01036e0 0x7f50e0107b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009930 tx=0x7f50d402a420 comp rx=0 tx=0).stop 2024-09-19T09:26:41.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.658+0000 7f50e87ce640 1 -- 172.21.15.16:0/273922275 shutdown_connections 2024-09-19T09:26:41.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.658+0000 7f50e87ce640 1 --2- 172.21.15.16:0/273922275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50e01082a0 0x7f50e010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.658+0000 7f50e87ce640 1 --2- 172.21.15.16:0/273922275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e01036e0 0x7f50e0107b90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.658+0000 7f50e87ce640 1 --2- 172.21.15.16:0/273922275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50e0102d30 0x7f50e0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.658+0000 7f50e87ce640 1 -- 172.21.15.16:0/273922275 >> 172.21.15.16:0/273922275 conn(0x7f50e00fc7d0 msgr2=0x7f50e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:41.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.658+0000 7f50e87ce640 1 -- 172.21.15.16:0/273922275 shutdown_connections 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.658+0000 7f50e87ce640 1 -- 172.21.15.16:0/273922275 wait complete. 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.658+0000 7f50e87ce640 1 Processor -- start 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e87ce640 1 -- start start 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e87ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50e0102d30 0x7f50e0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e87ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e01036e0 0x7f50e0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e87ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50e01082a0 0x7f50e019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e87ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50e01120e0 con 0x7f50e01036e0 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e87ce640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f50e0111f60 con 0x7f50e01082a0 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e87ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f50e0112260 con 0x7f50e0102d30 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e5d69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e01036e0 0x7f50e0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e5d69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e01036e0 0x7f50e0198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53004/0 (socket says 172.21.15.16:53004) 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e5d69640 1 -- 172.21.15.16:0/599785721 learned_addr learned my addr 172.21.15.16:0/599785721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e656a640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50e0102d30 0x7f50e0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e6d6b640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50e01082a0 0x7f50e019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e5d69640 1 -- 172.21.15.16:0/599785721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50e0102d30 msgr2=0x7f50e0198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e5d69640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50e0102d30 0x7f50e0198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e5d69640 1 -- 172.21.15.16:0/599785721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50e01082a0 msgr2=0x7f50e019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e5d69640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50e01082a0 0x7f50e019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e5d69640 1 -- 172.21.15.16:0/599785721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e019d5f0 con 0x7f50e01036e0 2024-09-19T09:26:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e6d6b640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50e01082a0 0x7f50e019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.659+0000 7f50e5d69640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e01036e0 0x7f50e0198b80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f50d4005940 tx=0x7f50d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.660+0000 7f50df7fe640 1 -- 172.21.15.16:0/599785721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d403f030 con 0x7f50e01036e0 2024-09-19T09:26:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.660+0000 7f50df7fe640 1 -- 172.21.15.16:0/599785721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d4003ce0 con 0x7f50e01036e0 2024-09-19T09:26:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.660+0000 7f50e87ce640 1 -- 172.21.15.16:0/599785721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e019d880 con 0x7f50e01036e0 2024-09-19T09:26:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.660+0000 7f50df7fe640 1 -- 172.21.15.16:0/599785721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4035070 con 0x7f50e01036e0 2024-09-19T09:26:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.660+0000 7f50e87ce640 1 -- 172.21.15.16:0/599785721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e01a51c0 con 0x7f50e01036e0 2024-09-19T09:26:41.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.661+0000 7f50e87ce640 1 -- 172.21.15.16:0/599785721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50e0103110 con 0x7f50e01036e0 2024-09-19T09:26:41.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.662+0000 7f50df7fe640 1 -- 172.21.15.16:0/599785721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f50d4042030 con 0x7f50e01036e0 2024-09-19T09:26:41.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.662+0000 7f50df7fe640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f50c80a3350 0x7f50c80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:41.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.663+0000 7f50e656a640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f50c80a3350 0x7f50c80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.663+0000 7f50df7fe640 1 -- 172.21.15.16:0/599785721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f50d40f0110 con 0x7f50e01036e0 2024-09-19T09:26:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.663+0000 7f50e656a640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f50c80a3350 0x7f50c80a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f50d0005fd0 tx=0x7f50d0009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:41.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.666+0000 7f50df7fe640 1 -- 172.21.15.16:0/599785721 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f50d40bccd0 con 0x7f50e01036e0 2024-09-19T09:26:41.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.881+0000 7f50e87ce640 1 -- 172.21.15.16:0/599785721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50e01999a0 con 0x7f50e01036e0 2024-09-19T09:26:41.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.882+0000 7f50df7fe640 1 -- 172.21.15.16:0/599785721 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11967 (secure 0 0 0) 0x7f50e01999a0 con 0x7f50e01036e0 2024-09-19T09:26:41.884 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:26:41.884 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"btime":"2024-09-19T09:25:40:972080+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:25:40.972058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24484,24470,24428,24424]},"id":1}]} 2024-09-19T09:26:41.884 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-19T09:26:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.885+0000 7f50e87ce640 1 -- 172.21.15.16:0/599785721 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f50c80a3350 msgr2=0x7f50c80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.885+0000 7f50e87ce640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f50c80a3350 0x7f50c80a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f50d0005fd0 tx=0x7f50d0009680 comp rx=0 tx=0).stop 2024-09-19T09:26:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.885+0000 7f50e87ce640 1 -- 172.21.15.16:0/599785721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e01036e0 msgr2=0x7f50e0198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.885+0000 7f50e87ce640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e01036e0 0x7f50e0198b80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f50d4005940 tx=0x7f50d40042f0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.885+0000 7f50e87ce640 1 -- 172.21.15.16:0/599785721 shutdown_connections 2024-09-19T09:26:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.885+0000 7f50e87ce640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50e01082a0 0x7f50e019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.885+0000 7f50e87ce640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50e01036e0 0x7f50e0198b80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.885+0000 7f50e87ce640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f50c80a3350 0x7f50c80a5810 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.885+0000 7f50e87ce640 1 --2- 172.21.15.16:0/599785721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50e0102d30 0x7f50e0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:41.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.885+0000 7f50e87ce640 1 -- 172.21.15.16:0/599785721 >> 172.21.15.16:0/599785721 conn(0x7f50e00fc7d0 msgr2=0x7f50e0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:41.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.885+0000 7f50e87ce640 1 -- 172.21.15.16:0/599785721 shutdown_connections 2024-09-19T09:26:41.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:41.885+0000 7f50e87ce640 1 -- 172.21.15.16:0/599785721 wait complete. 2024-09-19T09:26:42.528 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T09:25:29.829112+0000', 'modified': '2024-09-19T09:25:40.972058+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24500, 'mds_1': 24484, 'mds_2': 24470, 'mds_3': 24428, 'mds_4': 24424}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14580': {'gid': 14580, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/3573603820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 3573603820}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 3573603820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14592': {'gid': 14592, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/4017062522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 4017062522}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 4017062522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.142:6833/3906366829', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 3906366829}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 3906366829}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.97:6833/108652274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 108652274}, {'type': 'v1', 'addr': '172.21.15.97:6833', 'nonce': 108652274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/2657443138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 2657443138}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 2657443138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/4135704970', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 4135704970}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 4135704970}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6837/81091251', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 81091251}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 81091251}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.97:6837/2018284957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 2018284957}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 2018284957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.142:6839/3822134597', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3822134597}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3822134597}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6839/3225392797', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3225392797}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3225392797}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24500, 'qdb_cluster': [24500, 24484, 24470, 24428, 24424]} 2024-09-19T09:26:42.528 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T09:26:42.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.l status 2024-09-19T09:26:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:43.054 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:42 smithi016 ceph-mon[30979]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:43.054 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:42 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/599785721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:43.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:42 smithi142 ceph-mon[35032]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:43.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:42 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/599785721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:43.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:42 smithi097 ceph-mon[34864]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:43.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:42 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/599785721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:26:43.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.274+0000 7f00335ce640 1 -- 172.21.15.16:0/2838966977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c101de0 msgr2=0x7f002c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:43.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.274+0000 7f00335ce640 1 --2- 172.21.15.16:0/2838966977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c101de0 0x7f002c1021c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0020009900 tx=0x7f002002a410 comp rx=0 tx=0).stop 2024-09-19T09:26:43.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.274+0000 7f00335ce640 1 -- 172.21.15.16:0/2838966977 shutdown_connections 2024-09-19T09:26:43.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.274+0000 7f00335ce640 1 --2- 172.21.15.16:0/2838966977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f002c106f10 0x7f002c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.274+0000 7f00335ce640 1 --2- 172.21.15.16:0/2838966977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f002c102790 0x7f002c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.274+0000 7f00335ce640 1 --2- 172.21.15.16:0/2838966977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c101de0 0x7f002c1021c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.274+0000 7f00335ce640 1 -- 172.21.15.16:0/2838966977 >> 172.21.15.16:0/2838966977 conn(0x7f002c0fd8c0 msgr2=0x7f002c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:43.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.274+0000 7f00335ce640 1 -- 172.21.15.16:0/2838966977 shutdown_connections 2024-09-19T09:26:43.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.274+0000 7f00335ce640 1 -- 172.21.15.16:0/2838966977 wait complete. 2024-09-19T09:26:43.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.275+0000 7f00335ce640 1 Processor -- start 2024-09-19T09:26:43.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.275+0000 7f00335ce640 1 -- start start 2024-09-19T09:26:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.275+0000 7f00335ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f002c101de0 0x7f002c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.275+0000 7f00335ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c102790 0x7f002c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.275+0000 7f00335ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f002c106f10 0x7f002c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.275+0000 7f00335ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f002c111f90 con 0x7f002c102790 2024-09-19T09:26:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.275+0000 7f00335ce640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f002c111e10 con 0x7f002c101de0 2024-09-19T09:26:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.275+0000 7f00335ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f002c112110 con 0x7f002c106f10 2024-09-19T09:26:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.275+0000 7f0030b69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c102790 0x7f002c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.275+0000 7f0030b69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c102790 0x7f002c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53026/0 (socket says 172.21.15.16:53026) 2024-09-19T09:26:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.275+0000 7f0030b69640 1 -- 172.21.15.16:0/3819664699 learned_addr learned my addr 172.21.15.16:0/3819664699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.275+0000 7f003136a640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f002c101de0 0x7f002c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.276+0000 7f0031b6b640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f002c106f10 0x7f002c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.276+0000 7f0030b69640 1 -- 172.21.15.16:0/3819664699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f002c106f10 msgr2=0x7f002c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.276+0000 7f0030b69640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f002c106f10 0x7f002c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.276+0000 7f0030b69640 1 -- 172.21.15.16:0/3819664699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f002c101de0 msgr2=0x7f002c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.276+0000 7f0030b69640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f002c101de0 0x7f002c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.276+0000 7f0030b69640 1 -- 172.21.15.16:0/3819664699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002c19d380 con 0x7f002c102790 2024-09-19T09:26:43.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.276+0000 7f003136a640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f002c101de0 0x7f002c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:43.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.276+0000 7f0030b69640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c102790 0x7f002c198910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f002800cce0 tx=0x7f002800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:43.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.276+0000 7f001e7fc640 1 -- 172.21.15.16:0/3819664699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002800f650 con 0x7f002c102790 2024-09-19T09:26:43.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.276+0000 7f001e7fc640 1 -- 172.21.15.16:0/3819664699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f002800fcd0 con 0x7f002c102790 2024-09-19T09:26:43.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.276+0000 7f00335ce640 1 -- 172.21.15.16:0/3819664699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002c19d670 con 0x7f002c102790 2024-09-19T09:26:43.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.276+0000 7f001e7fc640 1 -- 172.21.15.16:0/3819664699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028018c70 con 0x7f002c102790 2024-09-19T09:26:43.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.276+0000 7f00335ce640 1 -- 172.21.15.16:0/3819664699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002c1a4f50 con 0x7f002c102790 2024-09-19T09:26:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.279+0000 7f001e7fc640 1 -- 172.21.15.16:0/3819664699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0028004b20 con 0x7f002c102790 2024-09-19T09:26:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.279+0000 7f00359a7740 1 Processor -- start 2024-09-19T09:26:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.279+0000 7f00359a7740 1 -- start start 2024-09-19T09:26:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.279+0000 7f00359a7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92aa8d0 0x559ca92aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.279+0000 7f00359a7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92ab1f0 0x559ca92b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.279+0000 7f00359a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92b9fc0 0x559ca92bc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.279+0000 7f00359a7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559ca9211090 con 0x559ca92ab1f0 2024-09-19T09:26:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.279+0000 7f00359a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559ca91b8a70 con 0x559ca92b9fc0 2024-09-19T09:26:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.279+0000 7f00359a7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559ca920d050 con 0x559ca92aa8d0 2024-09-19T09:26:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.279+0000 7f003136a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92ab1f0 0x559ca92b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.279+0000 7f003136a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92ab1f0 0x559ca92b9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53036/0 (socket says 172.21.15.16:53036) 2024-09-19T09:26:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.279+0000 7f003136a640 1 -- 172.21.15.16:0/3834440573 learned_addr learned my addr 172.21.15.16:0/3834440573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f001e7fc640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f00180a3350 0x7f00180a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f0031b6b640 1 --2- 172.21.15.16:0/3834440573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92aa8d0 0x559ca92aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f0030b69640 1 --2- 172.21.15.16:0/3834440573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92b9fc0 0x559ca92bc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f0031b6b640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f00180a3350 0x7f00180a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f003136a640 1 -- 172.21.15.16:0/3834440573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92aa8d0 msgr2=0x559ca92aacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f003136a640 1 --2- 172.21.15.16:0/3834440573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92aa8d0 0x559ca92aacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f003136a640 1 -- 172.21.15.16:0/3834440573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92b9fc0 msgr2=0x559ca92bc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f003136a640 1 --2- 172.21.15.16:0/3834440573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92b9fc0 0x559ca92bc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f003136a640 1 -- 172.21.15.16:0/3834440573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca91c5130 con 0x559ca92ab1f0 2024-09-19T09:26:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f0030b69640 1 --2- 172.21.15.16:0/3834440573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92b9fc0 0x559ca92bc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f001e7fc640 1 -- 172.21.15.16:0/3819664699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f00280cffd0 con 0x7f002c102790 2024-09-19T09:26:43.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f003136a640 1 --2- 172.21.15.16:0/3834440573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92ab1f0 0x559ca92b9a80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0020009900 tx=0x7f00200316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e72b44a9f78c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:43.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f0031b6b640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f00180a3350 0x7f00180a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0024006aa0 tx=0x7f0024006da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:43.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f00177fe640 1 -- 172.21.15.16:0/3834440573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002003f070 con 0x559ca92ab1f0 2024-09-19T09:26:43.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f00177fe640 1 -- 172.21.15.16:0/3834440573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00200035f0 con 0x559ca92ab1f0 2024-09-19T09:26:43.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.280+0000 7f00177fe640 1 -- 172.21.15.16:0/3834440573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020035070 con 0x559ca92ab1f0 2024-09-19T09:26:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.281+0000 7f00359a7740 1 -- 172.21.15.16:0/3834440573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92ab1f0 msgr2=0x559ca92b9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.281+0000 7f00359a7740 1 --2- 172.21.15.16:0/3834440573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92ab1f0 0x559ca92b9a80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0020009900 tx=0x7f00200316e0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.281+0000 7f00359a7740 1 -- 172.21.15.16:0/3834440573 shutdown_connections 2024-09-19T09:26:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.281+0000 7f00359a7740 1 --2- 172.21.15.16:0/3834440573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92b9fc0 0x559ca92bc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.281+0000 7f00359a7740 1 --2- 172.21.15.16:0/3834440573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92ab1f0 0x559ca92b9a80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.281+0000 7f00359a7740 1 --2- 172.21.15.16:0/3834440573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92aa8d0 0x559ca92aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.281+0000 7f00359a7740 1 -- 172.21.15.16:0/3834440573 >> 172.21.15.16:0/3834440573 conn(0x559ca92a6f60 msgr2=0x559ca92a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.281+0000 7f00359a7740 1 -- 172.21.15.16:0/3834440573 shutdown_connections 2024-09-19T09:26:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.281+0000 7f00359a7740 1 -- 172.21.15.16:0/3834440573 wait complete. 2024-09-19T09:26:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.281+0000 7f00359a7740 1 -- start start 2024-09-19T09:26:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.282+0000 7f00359a7740 1 Processor -- start 2024-09-19T09:26:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.282+0000 7f00359a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92aa8d0 0x559ca92b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.282+0000 7f00359a7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92ab1f0 0x559ca92b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.282+0000 7f00359a7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92b9fc0 0x559ca92b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.282+0000 7f00359a7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559ca92bcb90 con 0x559ca92b9fc0 2024-09-19T09:26:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.282+0000 7f00359a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559ca91cbf20 con 0x559ca92aa8d0 2024-09-19T09:26:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.282+0000 7f00359a7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559ca92bcd10 con 0x559ca92ab1f0 2024-09-19T09:26:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.282+0000 7f00359a7740 10 client.0 _ll_register_callbacks cb 0x559ca9195030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.282+0000 7f0031b6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92b9fc0 0x559ca92b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.282+0000 7f0031b6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92b9fc0 0x559ca92b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53052/0 (socket says 172.21.15.16:53052) 2024-09-19T09:26:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.282+0000 7f0031b6b640 1 -- 172.21.15.16:0/1238993999 learned_addr learned my addr 172.21.15.16:0/1238993999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.282+0000 7f003136a640 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92aa8d0 0x559ca92b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.282+0000 7f0030b69640 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92ab1f0 0x559ca92b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f0031b6b640 1 -- 172.21.15.16:0/1238993999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92ab1f0 msgr2=0x559ca92b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f0031b6b640 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92ab1f0 0x559ca92b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f0031b6b640 1 -- 172.21.15.16:0/1238993999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92aa8d0 msgr2=0x559ca92b98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f0031b6b640 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92aa8d0 0x559ca92b98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f0031b6b640 1 -- 172.21.15.16:0/1238993999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca8f822b0 con 0x559ca92b9fc0 2024-09-19T09:26:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f0030b69640 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92ab1f0 0x559ca92b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f0031b6b640 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92b9fc0 0x559ca92b7180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0024012f40 tx=0x7f00240047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520954b2fd0b6f7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f0014ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:26:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f0014ff9640 1 -- 172.21.15.16:0/1238993999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024026060 con 0x559ca92b9fc0 2024-09-19T09:26:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f0014ff9640 1 -- 172.21.15.16:0/1238993999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0024014590 con 0x559ca92b9fc0 2024-09-19T09:26:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f00359a7740 1 -- 172.21.15.16:0/1238993999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00200035f0 con 0x559ca92b9fc0 2024-09-19T09:26:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f0014ff9640 1 -- 172.21.15.16:0/1238993999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0024014790 con 0x559ca92b9fc0 2024-09-19T09:26:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f00359a7740 10 client.14949 fetch_fsmap learned FSMap version 27 2024-09-19T09:26:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f00359a7740 1 -- 172.21.15.16:0/1238993999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559ca92c8630 con 0x559ca92b9fc0 2024-09-19T09:26:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f0014ff9640 1 -- 172.21.15.16:0/1238993999 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00200035f0 con 0x559ca92b9fc0 2024-09-19T09:26:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.283+0000 7f0014ff9640 1 -- 172.21.15.16:0/1238993999 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f002402b070 con 0x559ca92b9fc0 2024-09-19T09:26:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.284+0000 7f00359a7740 10 client.14949 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:26:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.284+0000 7f00359a7740 10 client.14949 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-19T09:26:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.284+0000 7f00359a7740 20 client.14949 populate_metadata read hostname 'smithi016' 2024-09-19T09:26:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.284+0000 7f00359a7740 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559ca92cac50 0x559ca92cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.284+0000 7f00359a7740 4 client.14949 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.284+0000 7f00359a7740 1 -- 172.21.15.16:0/1238993999 --> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ca921e600 con 0x559ca92cac50 2024-09-19T09:26:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.284+0000 7f003136a640 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559ca92cac50 0x559ca92cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.284+0000 7f003136a640 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559ca92cac50 0x559ca92cc0c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b71ceccf3b8d6b80 server_cookie=78d843ce99380964 in_seq=0 out_seq=0 2024-09-19T09:26:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.284+0000 7f0014ff9640 10 client.14949 ms_handle_connect on v2:172.21.15.142:6832/3906366829 2024-09-19T09:26:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.285+0000 7f0014ff9640 1 -- 172.21.15.16:0/1238993999 <== mds.4 v2:172.21.15.142:6832/3906366829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ca921e600 con 0x559ca92cac50 2024-09-19T09:26:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.285+0000 7f0014ff9640 10 client.14949 handle_command_reply: tid=0 2024-09-19T09:26:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.285+0000 7f00359a7740 1 client.14949 shutdown 2024-09-19T09:26:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.285+0000 7f00359a7740 1 -- 172.21.15.16:0/1238993999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92b9fc0 msgr2=0x559ca92b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.285+0000 7f00359a7740 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92b9fc0 0x559ca92b7180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0024012f40 tx=0x7f00240047d0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.285+0000 7f00359a7740 1 -- 172.21.15.16:0/1238993999 shutdown_connections 2024-09-19T09:26:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.285+0000 7f00359a7740 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559ca92cac50 0x559ca92cc0c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.285+0000 7f0014ff9640 0 client.14949 ms_handle_reset on v2:172.21.15.142:6832/3906366829 2024-09-19T09:26:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.285+0000 7f00359a7740 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92ab1f0 0x559ca92b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.285+0000 7f00359a7740 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92b9fc0 0x559ca92b7180 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.285+0000 7f00359a7740 1 --2- 172.21.15.16:0/1238993999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92aa8d0 0x559ca92b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.285+0000 7f00359a7740 1 -- 172.21.15.16:0/1238993999 >> 172.21.15.16:0/1238993999 conn(0x559ca92a6f60 msgr2=0x559ca92acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.286+0000 7f00359a7740 1 -- 172.21.15.16:0/1238993999 shutdown_connections 2024-09-19T09:26:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.286+0000 7f00359a7740 1 -- 172.21.15.16:0/1238993999 wait complete. 2024-09-19T09:26:43.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.286+0000 7f00359a7740 20 client.14949 trim_cache size 0 max 16384 2024-09-19T09:26:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.303+0000 7f00359a7740 1 Processor -- start 2024-09-19T09:26:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.303+0000 7f00359a7740 1 -- start start 2024-09-19T09:26:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f00359a7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 0x559ca92ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f00359a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92ab1f0 0x559ca92b2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f00359a7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92b9fc0 0x559ca92b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f00359a7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559ca92ca400 con 0x559ca92aa8d0 2024-09-19T09:26:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f00359a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559ca92caad0 con 0x559ca92ab1f0 2024-09-19T09:26:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f00359a7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559ca92cb1a0 con 0x559ca92b9fc0 2024-09-19T09:26:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f003136a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 0x559ca92ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f003136a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 0x559ca92ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53066/0 (socket says 172.21.15.16:53066) 2024-09-19T09:26:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f003136a640 1 -- 172.21.15.16:0/2172522837 learned_addr learned my addr 172.21.15.16:0/2172522837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f0030b69640 1 --2- 172.21.15.16:0/2172522837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92ab1f0 0x559ca92b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f003136a640 1 -- 172.21.15.16:0/2172522837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92b9fc0 msgr2=0x559ca92b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f003136a640 1 --2- 172.21.15.16:0/2172522837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92b9fc0 0x559ca92b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f003136a640 1 -- 172.21.15.16:0/2172522837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92ab1f0 msgr2=0x559ca92b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f003136a640 1 --2- 172.21.15.16:0/2172522837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92ab1f0 0x559ca92b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f003136a640 1 -- 172.21.15.16:0/2172522837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca92ab020 con 0x559ca92aa8d0 2024-09-19T09:26:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.304+0000 7f0030b69640 1 --2- 172.21.15.16:0/2172522837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92ab1f0 0x559ca92b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.305+0000 7f003136a640 1 --2- 172.21.15.16:0/2172522837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 0x559ca92ca580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0020009900 tx=0x7f002002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d587007c4e494501 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.305+0000 7f0016ffd640 1 -- 172.21.15.16:0/2172522837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002003f040 con 0x559ca92aa8d0 2024-09-19T09:26:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.305+0000 7f0016ffd640 1 -- 172.21.15.16:0/2172522837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0020004460 con 0x559ca92aa8d0 2024-09-19T09:26:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.305+0000 7f0016ffd640 1 -- 172.21.15.16:0/2172522837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020035040 con 0x559ca92aa8d0 2024-09-19T09:26:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.305+0000 7f00359a7740 1 -- 172.21.15.16:0/2172522837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 msgr2=0x559ca92ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.305+0000 7f00359a7740 1 --2- 172.21.15.16:0/2172522837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 0x559ca92ca580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0020009900 tx=0x7f002002a410 comp rx=0 tx=0).stop 2024-09-19T09:26:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.305+0000 7f00359a7740 1 -- 172.21.15.16:0/2172522837 shutdown_connections 2024-09-19T09:26:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.305+0000 7f00359a7740 1 --2- 172.21.15.16:0/2172522837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92b9fc0 0x559ca92b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.305+0000 7f00359a7740 1 --2- 172.21.15.16:0/2172522837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92ab1f0 0x559ca92b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.305+0000 7f00359a7740 1 --2- 172.21.15.16:0/2172522837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 0x559ca92ca580 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.306+0000 7f00359a7740 1 -- 172.21.15.16:0/2172522837 >> 172.21.15.16:0/2172522837 conn(0x559ca92a6f60 msgr2=0x559ca92b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.306+0000 7f00359a7740 1 -- 172.21.15.16:0/2172522837 shutdown_connections 2024-09-19T09:26:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.306+0000 7f00359a7740 1 -- 172.21.15.16:0/2172522837 wait complete. 2024-09-19T09:26:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.306+0000 7f00359a7740 1 -- start start 2024-09-19T09:26:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.306+0000 7f00359a7740 1 Processor -- start 2024-09-19T09:26:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.306+0000 7f00359a7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 0x559ca92d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.306+0000 7f00359a7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92ab1f0 0x559ca92d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.306+0000 7f00359a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92b9fc0 0x559ca92cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.306+0000 7f00359a7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559ca92cbf40 con 0x559ca92aa8d0 2024-09-19T09:26:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.306+0000 7f00359a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559ca92cb870 con 0x559ca92b9fc0 2024-09-19T09:26:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.306+0000 7f00359a7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559ca92cc610 con 0x559ca92ab1f0 2024-09-19T09:26:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.306+0000 7f00359a7740 10 client.0 _ll_register_callbacks cb 0x7f002400d640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:43.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f003136a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 0x559ca92d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f003136a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 0x559ca92d1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53072/0 (socket says 172.21.15.16:53072) 2024-09-19T09:26:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f003136a640 1 -- 172.21.15.16:0/3363777212 learned_addr learned my addr 172.21.15.16:0/3363777212 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f0030b69640 1 --2- 172.21.15.16:0/3363777212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92ab1f0 0x559ca92d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f003136a640 1 -- 172.21.15.16:0/3363777212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92ab1f0 msgr2=0x559ca92d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f003136a640 1 --2- 172.21.15.16:0/3363777212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92ab1f0 0x559ca92d15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f003136a640 1 -- 172.21.15.16:0/3363777212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92b9fc0 msgr2=0x559ca92cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f003136a640 1 --2- 172.21.15.16:0/3363777212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92b9fc0 0x559ca92cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f003136a640 1 -- 172.21.15.16:0/3363777212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca92cdd30 con 0x559ca92aa8d0 2024-09-19T09:26:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f0030b69640 1 --2- 172.21.15.16:0/3363777212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92ab1f0 0x559ca92d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f003136a640 1 --2- 172.21.15.16:0/3363777212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 0x559ca92d1080 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0020004be0 tx=0x7f00200047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c356826dbe39ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:43.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f0014ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:26:43.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f0014ff9640 1 -- 172.21.15.16:0/3363777212 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002005b060 con 0x559ca92aa8d0 2024-09-19T09:26:43.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f0014ff9640 1 -- 172.21.15.16:0/3363777212 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0020005090 con 0x559ca92aa8d0 2024-09-19T09:26:43.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.307+0000 7f00359a7740 1 -- 172.21.15.16:0/3363777212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0020004460 con 0x559ca92aa8d0 2024-09-19T09:26:43.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.308+0000 7f0014ff9640 1 -- 172.21.15.16:0/3363777212 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f002003f030 con 0x559ca92aa8d0 2024-09-19T09:26:43.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.308+0000 7f00359a7740 10 client.14955 fetch_fsmap learned FSMap version 27 2024-09-19T09:26:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.308+0000 7f00359a7740 1 -- 172.21.15.16:0/3363777212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559ca92d2840 con 0x559ca92aa8d0 2024-09-19T09:26:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.308+0000 7f0014ff9640 1 -- 172.21.15.16:0/3363777212 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0020004460 con 0x559ca92aa8d0 2024-09-19T09:26:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.308+0000 7f0014ff9640 1 -- 172.21.15.16:0/3363777212 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0020060070 con 0x559ca92aa8d0 2024-09-19T09:26:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.308+0000 7f00359a7740 10 client.14955 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:26:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.308+0000 7f00359a7740 10 client.14955 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-19T09:26:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.308+0000 7f00359a7740 20 client.14955 populate_metadata read hostname 'smithi016' 2024-09-19T09:26:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.308+0000 7f00359a7740 1 --2- 172.21.15.16:0/3363777212 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559ca92cac50 0x559ca9337da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.308+0000 7f00359a7740 4 client.14955 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:26:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.308+0000 7f00359a7740 1 -- 172.21.15.16:0/3363777212 --> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] -- command(tid 0: {"prefix": "status"}) -- 0x559ca9218b10 con 0x559ca92cac50 2024-09-19T09:26:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.308+0000 7f0030b69640 1 --2- 172.21.15.16:0/3363777212 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559ca92cac50 0x559ca9337da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f0030b69640 1 --2- 172.21.15.16:0/3363777212 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559ca92cac50 0x559ca9337da0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2b0cdbb606265411 server_cookie=72988363ab53a30d in_seq=0 out_seq=0 2024-09-19T09:26:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f0014ff9640 10 client.14955 ms_handle_connect on v2:172.21.15.142:6832/3906366829 2024-09-19T09:26:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f0014ff9640 1 -- 172.21.15.16:0/3363777212 <== mds.4 v2:172.21.15.142:6832/3906366829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559ca9218b10 con 0x559ca92cac50 2024-09-19T09:26:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f0014ff9640 10 client.14955 handle_command_reply: tid=0 2024-09-19T09:26:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f00359a7740 1 client.14955 shutdown 2024-09-19T09:26:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f00359a7740 1 -- 172.21.15.16:0/3363777212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 msgr2=0x559ca92d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f00359a7740 1 --2- 172.21.15.16:0/3363777212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 0x559ca92d1080 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0020004be0 tx=0x7f00200047a0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f00359a7740 1 -- 172.21.15.16:0/3363777212 shutdown_connections 2024-09-19T09:26:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f00359a7740 1 --2- 172.21.15.16:0/3363777212 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559ca92cac50 0x559ca9337da0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f00359a7740 1 --2- 172.21.15.16:0/3363777212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ca92ab1f0 0x559ca92d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f00359a7740 1 --2- 172.21.15.16:0/3363777212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ca92b9fc0 0x559ca92cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f00359a7740 1 --2- 172.21.15.16:0/3363777212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ca92aa8d0 0x559ca92d1080 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f00359a7740 1 -- 172.21.15.16:0/3363777212 >> 172.21.15.16:0/3363777212 conn(0x559ca92a6f60 msgr2=0x559ca92b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f0014ff9640 0 client.14955 ms_handle_reset on v2:172.21.15.142:6832/3906366829 2024-09-19T09:26:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f00359a7740 1 -- 172.21.15.16:0/3363777212 shutdown_connections 2024-09-19T09:26:43.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.309+0000 7f00359a7740 1 -- 172.21.15.16:0/3363777212 wait complete. 2024-09-19T09:26:43.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.310+0000 7f00359a7740 20 client.14955 trim_cache size 0 max 16384 2024-09-19T09:26:43.316 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:26:43.316 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:26:43.316 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-09-19T09:26:43.316 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24424, 2024-09-19T09:26:43.316 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:26:43.316 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:26:43.316 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:26:43.316 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 63.342731706999999, 2024-09-19T09:26:43.316 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-19T09:26:43.317 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:26:43.317 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:26:43.317 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 113.76159536500001 2024-09-19T09:26:43.317 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:26:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.310+0000 7f00335ce640 1 -- 172.21.15.16:0/3819664699 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f00180a3350 msgr2=0x7f00180a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:43.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.310+0000 7f00335ce640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f00180a3350 0x7f00180a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0024006aa0 tx=0x7f0024006da0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.310+0000 7f00335ce640 1 -- 172.21.15.16:0/3819664699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c102790 msgr2=0x7f002c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.310+0000 7f00335ce640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c102790 0x7f002c198910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f002800cce0 tx=0x7f002800a620 comp rx=0 tx=0).stop 2024-09-19T09:26:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.310+0000 7f00335ce640 1 -- 172.21.15.16:0/3819664699 shutdown_connections 2024-09-19T09:26:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.310+0000 7f00335ce640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f00180a3350 0x7f00180a5810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.310+0000 7f00335ce640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f002c106f10 0x7f002c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.310+0000 7f00335ce640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c102790 0x7f002c198910 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.310+0000 7f00335ce640 1 --2- 172.21.15.16:0/3819664699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f002c101de0 0x7f002c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.310+0000 7f00335ce640 1 -- 172.21.15.16:0/3819664699 >> 172.21.15.16:0/3819664699 conn(0x7f002c0fd8c0 msgr2=0x7f002c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.310+0000 7f00335ce640 1 -- 172.21.15.16:0/3819664699 shutdown_connections 2024-09-19T09:26:43.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:43.310+0000 7f00335ce640 1 -- 172.21.15.16:0/3819664699 wait complete. 2024-09-19T09:26:43.962 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.k status 2024-09-19T09:26:44.226 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:44.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.705+0000 7f46778b7640 1 -- 172.21.15.16:0/2059183895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4670101280 msgr2=0x7f467010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.705+0000 7f46778b7640 1 --2- 172.21.15.16:0/2059183895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4670101280 0x7f467010df40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4660009900 tx=0x7f466002a410 comp rx=0 tx=0).stop 2024-09-19T09:26:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.705+0000 7f46778b7640 1 -- 172.21.15.16:0/2059183895 shutdown_connections 2024-09-19T09:26:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.705+0000 7f46778b7640 1 --2- 172.21.15.16:0/2059183895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f467010e480 0x7f4670110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.705+0000 7f46778b7640 1 --2- 172.21.15.16:0/2059183895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4670101280 0x7f467010df40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.705+0000 7f46778b7640 1 --2- 172.21.15.16:0/2059183895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46701008d0 0x7f4670100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.705+0000 7f46778b7640 1 -- 172.21.15.16:0/2059183895 >> 172.21.15.16:0/2059183895 conn(0x7f46700fc7a0 msgr2=0x7f46700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.705+0000 7f46778b7640 1 -- 172.21.15.16:0/2059183895 shutdown_connections 2024-09-19T09:26:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.705+0000 7f46778b7640 1 -- 172.21.15.16:0/2059183895 wait complete. 2024-09-19T09:26:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.706+0000 7f46778b7640 1 Processor -- start 2024-09-19T09:26:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.706+0000 7f46778b7640 1 -- start start 2024-09-19T09:26:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.706+0000 7f46778b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46701008d0 0x7f467019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.706+0000 7f46778b7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4670101280 0x7f467019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.706+0000 7f46778b7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467010e480 0x7f467019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.706+0000 7f46778b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4670114200 con 0x7f46701008d0 2024-09-19T09:26:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.706+0000 7f46778b7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4670114080 con 0x7f467010e480 2024-09-19T09:26:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.706+0000 7f46778b7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4670114380 con 0x7f4670101280 2024-09-19T09:26:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.706+0000 7f4675653640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46701008d0 0x7f467019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.706+0000 7f4675653640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46701008d0 0x7f467019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53102/0 (socket says 172.21.15.16:53102) 2024-09-19T09:26:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.706+0000 7f4675653640 1 -- 172.21.15.16:0/3998236518 learned_addr learned my addr 172.21.15.16:0/3998236518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.706+0000 7f4674e52640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4670101280 0x7f467019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.706+0000 7f4675e54640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467010e480 0x7f467019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.707+0000 7f4675653640 1 -- 172.21.15.16:0/3998236518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4670101280 msgr2=0x7f467019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.707+0000 7f4675653640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4670101280 0x7f467019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.707+0000 7f4675653640 1 -- 172.21.15.16:0/3998236518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467010e480 msgr2=0x7f467019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.707+0000 7f4675653640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467010e480 0x7f467019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.707+0000 7f4675653640 1 -- 172.21.15.16:0/3998236518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467019f6f0 con 0x7f46701008d0 2024-09-19T09:26:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.707+0000 7f4675e54640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467010e480 0x7f467019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.707+0000 7f4675653640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46701008d0 0x7f467019a740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f466c00b4a0 tx=0x7f466c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.707+0000 7f46667fc640 1 -- 172.21.15.16:0/3998236518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c014070 con 0x7f46701008d0 2024-09-19T09:26:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.707+0000 7f46667fc640 1 -- 172.21.15.16:0/3998236518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466c010af0 con 0x7f46701008d0 2024-09-19T09:26:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.707+0000 7f46778b7640 1 -- 172.21.15.16:0/3998236518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467019f9e0 con 0x7f46701008d0 2024-09-19T09:26:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.707+0000 7f46667fc640 1 -- 172.21.15.16:0/3998236518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c017d10 con 0x7f46701008d0 2024-09-19T09:26:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.707+0000 7f46778b7640 1 -- 172.21.15.16:0/3998236518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46701a72c0 con 0x7f46701008d0 2024-09-19T09:26:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.709+0000 7f46667fc640 1 -- 172.21.15.16:0/3998236518 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f466c017070 con 0x7f46701008d0 2024-09-19T09:26:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f4679c90740 1 Processor -- start 2024-09-19T09:26:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f4679c90740 1 -- start start 2024-09-19T09:26:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f4679c90740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e719d0 0x55bcc3e71db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f4679c90740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e722f0 0x55bcc3e7c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f4679c90740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e7efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f4679c90740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3dd8090 con 0x55bcc3e719d0 2024-09-19T09:26:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f4679c90740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3d7fa70 con 0x55bcc3e722f0 2024-09-19T09:26:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f4679c90740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3dd4050 con 0x55bcc3e7cbb0 2024-09-19T09:26:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f4675e54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e719d0 0x55bcc3e71db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f4675e54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e719d0 0x55bcc3e71db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53114/0 (socket says 172.21.15.16:53114) 2024-09-19T09:26:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f4675e54640 1 -- 172.21.15.16:0/504943278 learned_addr learned my addr 172.21.15.16:0/504943278 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f4675653640 1 --2- 172.21.15.16:0/504943278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e722f0 0x55bcc3e7c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f46667fc640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f465c0a32d0 0x7f465c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.710+0000 7f4674e52640 1 --2- 172.21.15.16:0/504943278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e7efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.711+0000 7f4675e54640 1 -- 172.21.15.16:0/504943278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 msgr2=0x55bcc3e7efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.711+0000 7f4675e54640 1 --2- 172.21.15.16:0/504943278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e7efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.711+0000 7f4675e54640 1 -- 172.21.15.16:0/504943278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e722f0 msgr2=0x55bcc3e7c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.711+0000 7f4675e54640 1 --2- 172.21.15.16:0/504943278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e722f0 0x55bcc3e7c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.711+0000 7f4675e54640 1 -- 172.21.15.16:0/504943278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc3d8c130 con 0x55bcc3e719d0 2024-09-19T09:26:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.711+0000 7f46667fc640 1 -- 172.21.15.16:0/3998236518 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f466c0ced60 con 0x7f46701008d0 2024-09-19T09:26:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.711+0000 7f4675e54640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f465c0a32d0 0x7f465c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.711+0000 7f4675e54640 1 --2- 172.21.15.16:0/504943278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e719d0 0x55bcc3e71db0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4668008ef0 tx=0x7f466800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece1f48f1979a374 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.711+0000 7f465b7fe640 1 -- 172.21.15.16:0/504943278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466801a8e0 con 0x55bcc3e719d0 2024-09-19T09:26:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.711+0000 7f465b7fe640 1 -- 172.21.15.16:0/504943278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4668019410 con 0x55bcc3e719d0 2024-09-19T09:26:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.711+0000 7f465b7fe640 1 -- 172.21.15.16:0/504943278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466800e5f0 con 0x55bcc3e719d0 2024-09-19T09:26:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.711+0000 7f4675e54640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f465c0a32d0 0x7f465c0a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f466801e000 tx=0x7f466801f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.712+0000 7f4679c90740 1 -- 172.21.15.16:0/504943278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e719d0 msgr2=0x55bcc3e71db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.712+0000 7f4679c90740 1 --2- 172.21.15.16:0/504943278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e719d0 0x55bcc3e71db0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4668008ef0 tx=0x7f466800d710 comp rx=0 tx=0).stop 2024-09-19T09:26:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.712+0000 7f4679c90740 1 -- 172.21.15.16:0/504943278 shutdown_connections 2024-09-19T09:26:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.712+0000 7f4679c90740 1 --2- 172.21.15.16:0/504943278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e7efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.712+0000 7f4679c90740 1 --2- 172.21.15.16:0/504943278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e722f0 0x55bcc3e7c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.712+0000 7f4679c90740 1 --2- 172.21.15.16:0/504943278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e719d0 0x55bcc3e71db0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.712+0000 7f4679c90740 1 -- 172.21.15.16:0/504943278 >> 172.21.15.16:0/504943278 conn(0x55bcc3e6df60 msgr2=0x55bcc3e6e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.712+0000 7f4679c90740 1 -- 172.21.15.16:0/504943278 shutdown_connections 2024-09-19T09:26:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.712+0000 7f4679c90740 1 -- 172.21.15.16:0/504943278 wait complete. 2024-09-19T09:26:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.712+0000 7f4679c90740 1 -- start start 2024-09-19T09:26:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.712+0000 7f4679c90740 1 Processor -- start 2024-09-19T09:26:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4679c90740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e7c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4679c90740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 0x55bcc3e797c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4679c90740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e79d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4679c90740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3e7f780 con 0x55bcc3e722f0 2024-09-19T09:26:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4679c90740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3d92f20 con 0x55bcc3e719d0 2024-09-19T09:26:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4679c90740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3e7f900 con 0x55bcc3e7cbb0 2024-09-19T09:26:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4679c90740 10 client.0 _ll_register_callbacks cb 0x55bcc3d5c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4674e52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 0x55bcc3e797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4674e52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 0x55bcc3e797c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53130/0 (socket says 172.21.15.16:53130) 2024-09-19T09:26:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4674e52640 1 -- 172.21.15.16:0/3666512682 learned_addr learned my addr 172.21.15.16:0/3666512682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4675653640 1 --2- 172.21.15.16:0/3666512682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e7c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4675e54640 1 --2- 172.21.15.16:0/3666512682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e79d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4674e52640 1 -- 172.21.15.16:0/3666512682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 msgr2=0x55bcc3e79d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4674e52640 1 --2- 172.21.15.16:0/3666512682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e79d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4674e52640 1 -- 172.21.15.16:0/3666512682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 msgr2=0x55bcc3e7c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4674e52640 1 --2- 172.21.15.16:0/3666512682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e7c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.713+0000 7f4674e52640 1 -- 172.21.15.16:0/3666512682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc3b492b0 con 0x55bcc3e722f0 2024-09-19T09:26:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4674e52640 1 --2- 172.21.15.16:0/3666512682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 0x55bcc3e797c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4660005ec0 tx=0x7f46600319d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699784670e9204d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4658ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:26:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4658ff9640 1 -- 172.21.15.16:0/3666512682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660035040 con 0x55bcc3e722f0 2024-09-19T09:26:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4658ff9640 1 -- 172.21.15.16:0/3666512682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4660004380 con 0x55bcc3e722f0 2024-09-19T09:26:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4679c90740 1 -- 172.21.15.16:0/3666512682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4668019410 con 0x55bcc3e722f0 2024-09-19T09:26:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4658ff9640 1 -- 172.21.15.16:0/3666512682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4660005020 con 0x55bcc3e722f0 2024-09-19T09:26:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4679c90740 10 client.14967 fetch_fsmap learned FSMap version 27 2024-09-19T09:26:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4679c90740 1 -- 172.21.15.16:0/3666512682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bcc3e8b0d0 con 0x55bcc3e722f0 2024-09-19T09:26:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4658ff9640 1 -- 172.21.15.16:0/3666512682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4668019410 con 0x55bcc3e722f0 2024-09-19T09:26:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4658ff9640 1 -- 172.21.15.16:0/3666512682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f46600035f0 con 0x55bcc3e722f0 2024-09-19T09:26:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4679c90740 10 client.14967 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:26:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4679c90740 10 client.14967 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-19T09:26:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4679c90740 20 client.14967 populate_metadata read hostname 'smithi016' 2024-09-19T09:26:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4679c90740 1 --2- 172.21.15.16:0/3666512682 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55bcc3e8d730 0x55bcc3e8eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4679c90740 4 client.14967 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.714+0000 7f4679c90740 1 -- 172.21.15.16:0/3666512682 --> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcc3de5600 con 0x55bcc3e8d730 2024-09-19T09:26:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.715+0000 7f4675653640 1 --2- 172.21.15.16:0/3666512682 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55bcc3e8d730 0x55bcc3e8eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.715+0000 7f4658ff9640 10 client.14967 ms_handle_connect on v2:172.21.15.97:6832/108652274 2024-09-19T09:26:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.715+0000 7f4675653640 1 --2- 172.21.15.16:0/3666512682 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55bcc3e8d730 0x55bcc3e8eba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8c8f4a9df4a01ce server_cookie=81face82ea2eca17 in_seq=0 out_seq=0 2024-09-19T09:26:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.716+0000 7f4658ff9640 1 -- 172.21.15.16:0/3666512682 <== mds.3 v2:172.21.15.97:6832/108652274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcc3de5600 con 0x55bcc3e8d730 2024-09-19T09:26:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.716+0000 7f4658ff9640 10 client.14967 handle_command_reply: tid=0 2024-09-19T09:26:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.716+0000 7f4679c90740 1 client.14967 shutdown 2024-09-19T09:26:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.716+0000 7f4679c90740 1 -- 172.21.15.16:0/3666512682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 msgr2=0x55bcc3e797c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:44.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.716+0000 7f4679c90740 1 --2- 172.21.15.16:0/3666512682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 0x55bcc3e797c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4660005ec0 tx=0x7f46600319d0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.716+0000 7f4679c90740 1 -- 172.21.15.16:0/3666512682 shutdown_connections 2024-09-19T09:26:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.716+0000 7f4679c90740 1 --2- 172.21.15.16:0/3666512682 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55bcc3e8d730 0x55bcc3e8eba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.716+0000 7f4679c90740 1 --2- 172.21.15.16:0/3666512682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e79d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.716+0000 7f4679c90740 1 --2- 172.21.15.16:0/3666512682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 0x55bcc3e797c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.716+0000 7f4679c90740 1 --2- 172.21.15.16:0/3666512682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e7c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.716+0000 7f4658ff9640 0 client.14967 ms_handle_reset on v2:172.21.15.97:6832/108652274 2024-09-19T09:26:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.716+0000 7f4679c90740 1 -- 172.21.15.16:0/3666512682 >> 172.21.15.16:0/3666512682 conn(0x55bcc3e6df60 msgr2=0x55bcc3e73c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.717+0000 7f4679c90740 1 -- 172.21.15.16:0/3666512682 shutdown_connections 2024-09-19T09:26:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.717+0000 7f4679c90740 1 -- 172.21.15.16:0/3666512682 wait complete. 2024-09-19T09:26:44.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.717+0000 7f4679c90740 20 client.14967 trim_cache size 0 max 16384 2024-09-19T09:26:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4679c90740 1 Processor -- start 2024-09-19T09:26:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4679c90740 1 -- start start 2024-09-19T09:26:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4679c90740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e8de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4679c90740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e722f0 0x55bcc3e787a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4679c90740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e96e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4679c90740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3e8cee0 con 0x55bcc3e7cbb0 2024-09-19T09:26:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4679c90740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3e8d5b0 con 0x55bcc3e719d0 2024-09-19T09:26:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4679c90740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3e8dc80 con 0x55bcc3e722f0 2024-09-19T09:26:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4675e54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e96e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4675e54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e96e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53136/0 (socket says 172.21.15.16:53136) 2024-09-19T09:26:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4675e54640 1 -- 172.21.15.16:0/221463253 learned_addr learned my addr 172.21.15.16:0/221463253 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4674e52640 1 --2- 172.21.15.16:0/221463253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e722f0 0x55bcc3e787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4675653640 1 --2- 172.21.15.16:0/221463253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e8de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4675e54640 1 -- 172.21.15.16:0/221463253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e722f0 msgr2=0x55bcc3e787a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4675e54640 1 --2- 172.21.15.16:0/221463253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e722f0 0x55bcc3e787a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4675e54640 1 -- 172.21.15.16:0/221463253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 msgr2=0x55bcc3e8de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4675e54640 1 --2- 172.21.15.16:0/221463253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e8de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4675e54640 1 -- 172.21.15.16:0/221463253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc3e77d10 con 0x55bcc3e7cbb0 2024-09-19T09:26:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.735+0000 7f4675653640 1 --2- 172.21.15.16:0/221463253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e8de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.736+0000 7f4675e54640 1 --2- 172.21.15.16:0/221463253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e96e80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4668008ef0 tx=0x7f4668008610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6826a5fbe21da1ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:44.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.736+0000 7f465affd640 1 -- 172.21.15.16:0/221463253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668019de0 con 0x55bcc3e7cbb0 2024-09-19T09:26:44.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.736+0000 7f465affd640 1 -- 172.21.15.16:0/221463253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466800e3b0 con 0x55bcc3e7cbb0 2024-09-19T09:26:44.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.736+0000 7f465affd640 1 -- 172.21.15.16:0/221463253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466801a930 con 0x55bcc3e7cbb0 2024-09-19T09:26:44.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.736+0000 7f4679c90740 1 -- 172.21.15.16:0/221463253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e7cbb0 msgr2=0x55bcc3e96e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:44.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.736+0000 7f4679c90740 1 --2- 172.21.15.16:0/221463253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e96e80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4668008ef0 tx=0x7f4668008610 comp rx=0 tx=0).stop 2024-09-19T09:26:44.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.736+0000 7f4679c90740 1 -- 172.21.15.16:0/221463253 shutdown_connections 2024-09-19T09:26:44.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.736+0000 7f4679c90740 1 --2- 172.21.15.16:0/221463253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e722f0 0x55bcc3e787a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.736+0000 7f4679c90740 1 --2- 172.21.15.16:0/221463253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3e96e80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.736+0000 7f4679c90740 1 --2- 172.21.15.16:0/221463253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e8de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.736+0000 7f4679c90740 1 -- 172.21.15.16:0/221463253 >> 172.21.15.16:0/221463253 conn(0x55bcc3e6df60 msgr2=0x55bcc3e78090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.737+0000 7f4679c90740 1 -- 172.21.15.16:0/221463253 shutdown_connections 2024-09-19T09:26:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.737+0000 7f4679c90740 1 -- 172.21.15.16:0/221463253 wait complete. 2024-09-19T09:26:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.737+0000 7f4679c90740 1 -- start start 2024-09-19T09:26:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.737+0000 7f4679c90740 1 Processor -- start 2024-09-19T09:26:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.737+0000 7f4679c90740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e91f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.737+0000 7f4679c90740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 0x55bcc3e92450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.737+0000 7f4679c90740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3ef5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.737+0000 7f4679c90740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3e8ea20 con 0x55bcc3e722f0 2024-09-19T09:26:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.737+0000 7f4679c90740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3e8e350 con 0x55bcc3e719d0 2024-09-19T09:26:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.737+0000 7f4679c90740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bcc3e8f0f0 con 0x55bcc3e7cbb0 2024-09-19T09:26:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.737+0000 7f4679c90740 10 client.0 _ll_register_callbacks cb 0x7f4660005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.737+0000 7f4674e52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 0x55bcc3e92450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4674e52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 0x55bcc3e92450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53146/0 (socket says 172.21.15.16:53146) 2024-09-19T09:26:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4674e52640 1 -- 172.21.15.16:0/2775663149 learned_addr learned my addr 172.21.15.16:0/2775663149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4675653640 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e91f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4675e54640 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3ef5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4674e52640 1 -- 172.21.15.16:0/2775663149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 msgr2=0x55bcc3ef5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4674e52640 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3ef5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4674e52640 1 -- 172.21.15.16:0/2775663149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 msgr2=0x55bcc3e91f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4674e52640 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e91f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4674e52640 1 -- 172.21.15.16:0/2775663149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc3e92d30 con 0x55bcc3e722f0 2024-09-19T09:26:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4675e54640 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3ef5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4675653640 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e91f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4674e52640 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 0x55bcc3e92450 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4660000c00 tx=0x7f4660003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588a440dd8cc3777 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4658ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4658ff9640 1 -- 172.21.15.16:0/2775663149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660035070 con 0x55bcc3e722f0 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4658ff9640 1 -- 172.21.15.16:0/2775663149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4660004180 con 0x55bcc3e722f0 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4679c90740 1 -- 172.21.15.16:0/2775663149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f466800e3b0 con 0x55bcc3e722f0 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4658ff9640 1 -- 172.21.15.16:0/2775663149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4660031a40 con 0x55bcc3e722f0 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4679c90740 10 client.14973 fetch_fsmap learned FSMap version 27 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.738+0000 7f4679c90740 1 -- 172.21.15.16:0/2775663149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bcc3ef7480 con 0x55bcc3e722f0 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.739+0000 7f4658ff9640 1 -- 172.21.15.16:0/2775663149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466800e3b0 con 0x55bcc3e722f0 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.739+0000 7f4658ff9640 1 -- 172.21.15.16:0/2775663149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4660004aa0 con 0x55bcc3e722f0 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.739+0000 7f4679c90740 10 client.14973 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.739+0000 7f4679c90740 10 client.14973 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-19T09:26:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.739+0000 7f4679c90740 20 client.14973 populate_metadata read hostname 'smithi016' 2024-09-19T09:26:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.739+0000 7f4679c90740 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55bcc3efc060 0x55bcc3efad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.739+0000 7f4679c90740 4 client.14973 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:26:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.739+0000 7f4679c90740 1 -- 172.21.15.16:0/2775663149 --> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] -- command(tid 0: {"prefix": "status"}) -- 0x55bcc3ddfb10 con 0x55bcc3efc060 2024-09-19T09:26:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.739+0000 7f4675653640 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55bcc3efc060 0x55bcc3efad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4658ff9640 10 client.14973 ms_handle_connect on v2:172.21.15.97:6832/108652274 2024-09-19T09:26:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4675653640 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55bcc3efc060 0x55bcc3efad70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a8034ff642e2b8a3 server_cookie=345bb69290f076da in_seq=0 out_seq=0 2024-09-19T09:26:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4658ff9640 1 -- 172.21.15.16:0/2775663149 <== mds.3 v2:172.21.15.97:6832/108652274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bcc3ddfb10 con 0x55bcc3efc060 2024-09-19T09:26:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4658ff9640 10 client.14973 handle_command_reply: tid=0 2024-09-19T09:26:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4679c90740 1 client.14973 shutdown 2024-09-19T09:26:44.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4679c90740 1 -- 172.21.15.16:0/2775663149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 msgr2=0x55bcc3e92450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:44.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4679c90740 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 0x55bcc3e92450 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4660000c00 tx=0x7f4660003820 comp rx=0 tx=0).stop 2024-09-19T09:26:44.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4679c90740 1 -- 172.21.15.16:0/2775663149 shutdown_connections 2024-09-19T09:26:44.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4679c90740 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55bcc3efc060 0x55bcc3efad70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4679c90740 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcc3e7cbb0 0x55bcc3ef5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4679c90740 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc3e722f0 0x55bcc3e92450 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4679c90740 1 --2- 172.21.15.16:0/2775663149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bcc3e719d0 0x55bcc3e91f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4658ff9640 0 client.14973 ms_handle_reset on v2:172.21.15.97:6832/108652274 2024-09-19T09:26:44.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4679c90740 1 -- 172.21.15.16:0/2775663149 >> 172.21.15.16:0/2775663149 conn(0x55bcc3e6df60 msgr2=0x55bcc3e788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:44.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4679c90740 1 -- 172.21.15.16:0/2775663149 shutdown_connections 2024-09-19T09:26:44.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4679c90740 1 -- 172.21.15.16:0/2775663149 wait complete. 2024-09-19T09:26:44.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.740+0000 7f4679c90740 20 client.14973 trim_cache size 0 max 16384 2024-09-19T09:26:44.748 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:26:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:26:44.748 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-09-19T09:26:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24428, 2024-09-19T09:26:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:26:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:26:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:26:44.749 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 65.782140737999995, 2024-09-19T09:26:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-19T09:26:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:26:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:26:44.750 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 112.620310719 2024-09-19T09:26:44.750 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:26:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.741+0000 7f46778b7640 1 -- 172.21.15.16:0/3998236518 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f465c0a32d0 msgr2=0x7f465c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.741+0000 7f46778b7640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f465c0a32d0 0x7f465c0a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f466801e000 tx=0x7f466801f000 comp rx=0 tx=0).stop 2024-09-19T09:26:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.741+0000 7f46778b7640 1 -- 172.21.15.16:0/3998236518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46701008d0 msgr2=0x7f467019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.741+0000 7f46778b7640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46701008d0 0x7f467019a740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f466c00b4a0 tx=0x7f466c00b960 comp rx=0 tx=0).stop 2024-09-19T09:26:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.741+0000 7f46778b7640 1 -- 172.21.15.16:0/3998236518 shutdown_connections 2024-09-19T09:26:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.741+0000 7f46778b7640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f465c0a32d0 0x7f465c0a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.741+0000 7f46778b7640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4670101280 0x7f467019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.741+0000 7f46778b7640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f467010e480 0x7f467019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.741+0000 7f46778b7640 1 --2- 172.21.15.16:0/3998236518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46701008d0 0x7f467019a740 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.741+0000 7f46778b7640 1 -- 172.21.15.16:0/3998236518 >> 172.21.15.16:0/3998236518 conn(0x7f46700fc7a0 msgr2=0x7f46700feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.741+0000 7f46778b7640 1 -- 172.21.15.16:0/3998236518 shutdown_connections 2024-09-19T09:26:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:44.741+0000 7f46778b7640 1 -- 172.21.15.16:0/3998236518 wait complete. 2024-09-19T09:26:45.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:44 smithi016 ceph-mon[30979]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:26:45.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:44 smithi142 ceph-mon[35032]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:26:45.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:44 smithi097 ceph-mon[34864]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:26:45.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T09:26:45.700 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.148+0000 7f88a4bf2640 1 -- 172.21.15.16:0/3763085987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a0102790 msgr2=0x7f88a01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.148+0000 7f88a4bf2640 1 --2- 172.21.15.16:0/3763085987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a0102790 0x7f88a01067e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8890009900 tx=0x7f889002a410 comp rx=0 tx=0).stop 2024-09-19T09:26:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.148+0000 7f88a4bf2640 1 -- 172.21.15.16:0/3763085987 shutdown_connections 2024-09-19T09:26:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.148+0000 7f88a4bf2640 1 --2- 172.21.15.16:0/3763085987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88a0106f10 0x7f88a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.148+0000 7f88a4bf2640 1 --2- 172.21.15.16:0/3763085987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a0102790 0x7f88a01067e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.148+0000 7f88a4bf2640 1 --2- 172.21.15.16:0/3763085987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88a0101de0 0x7f88a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.148+0000 7f88a4bf2640 1 -- 172.21.15.16:0/3763085987 >> 172.21.15.16:0/3763085987 conn(0x7f88a00fd8c0 msgr2=0x7f88a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.148+0000 7f88a4bf2640 1 -- 172.21.15.16:0/3763085987 shutdown_connections 2024-09-19T09:26:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.148+0000 7f88a4bf2640 1 -- 172.21.15.16:0/3763085987 wait complete. 2024-09-19T09:26:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.149+0000 7f88a4bf2640 1 Processor -- start 2024-09-19T09:26:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.149+0000 7f88a4bf2640 1 -- start start 2024-09-19T09:26:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.149+0000 7f88a4bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a0101de0 0x7f88a01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.149+0000 7f88a4bf2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88a0102790 0x7f88a0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.149+0000 7f88a4bf2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88a0106f10 0x7f88a019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.149+0000 7f88a4bf2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88a0112010 con 0x7f88a0101de0 2024-09-19T09:26:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.149+0000 7f88a4bf2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88a0111e90 con 0x7f88a0102790 2024-09-19T09:26:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.149+0000 7f88a4bf2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f88a0112190 con 0x7f88a0106f10 2024-09-19T09:26:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.149+0000 7f889e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a0101de0 0x7f88a01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.149+0000 7f889e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a0101de0 0x7f88a01985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46584/0 (socket says 172.21.15.16:46584) 2024-09-19T09:26:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.149+0000 7f889e59c640 1 -- 172.21.15.16:0/3457249049 learned_addr learned my addr 172.21.15.16:0/3457249049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.150+0000 7f889e59c640 1 -- 172.21.15.16:0/3457249049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88a0106f10 msgr2=0x7f88a019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.150+0000 7f889e59c640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88a0106f10 0x7f88a019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.150+0000 7f889e59c640 1 -- 172.21.15.16:0/3457249049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88a0102790 msgr2=0x7f88a0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.150+0000 7f889dd9b640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88a0102790 0x7f88a0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.150+0000 7f889e59c640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88a0102790 0x7f88a0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.150+0000 7f889e59c640 1 -- 172.21.15.16:0/3457249049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a019d560 con 0x7f88a0101de0 2024-09-19T09:26:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.150+0000 7f889dd9b640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88a0102790 0x7f88a0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.150+0000 7f889e59c640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a0101de0 0x7f88a01985b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f888800b4a0 tx=0x7f888800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.150+0000 7f88977fe640 1 -- 172.21.15.16:0/3457249049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888014070 con 0x7f88a0101de0 2024-09-19T09:26:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.150+0000 7f88a4bf2640 1 -- 172.21.15.16:0/3457249049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a019d850 con 0x7f88a0101de0 2024-09-19T09:26:46.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.150+0000 7f88977fe640 1 -- 172.21.15.16:0/3457249049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8888010af0 con 0x7f88a0101de0 2024-09-19T09:26:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.150+0000 7f88977fe640 1 -- 172.21.15.16:0/3457249049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888017d10 con 0x7f88a0101de0 2024-09-19T09:26:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.150+0000 7f88a4bf2640 1 -- 172.21.15.16:0/3457249049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a01a5130 con 0x7f88a0101de0 2024-09-19T09:26:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.152+0000 7f88977fe640 1 -- 172.21.15.16:0/3457249049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8888017070 con 0x7f88a0101de0 2024-09-19T09:26:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.153+0000 7f88a6fcb740 1 Processor -- start 2024-09-19T09:26:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.153+0000 7f88a6fcb740 1 -- start start 2024-09-19T09:26:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.153+0000 7f88a6fcb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae88b9d0 0x55e8ae88bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.153+0000 7f88a6fcb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88c2f0 0x55e8ae896670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.153+0000 7f88a6fcb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae896bb0 0x55e8ae898fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.153+0000 7f88a6fcb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae7f2090 con 0x55e8ae896bb0 2024-09-19T09:26:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.153+0000 7f88a6fcb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae799a70 con 0x55e8ae88b9d0 2024-09-19T09:26:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.153+0000 7f88a6fcb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae7ee050 con 0x55e8ae88c2f0 2024-09-19T09:26:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.153+0000 7f88977fe640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f888c0a32d0 0x7f888c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.153+0000 7f889dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae896bb0 0x55e8ae898fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.153+0000 7f889dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae896bb0 0x55e8ae898fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46586/0 (socket says 172.21.15.16:46586) 2024-09-19T09:26:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.153+0000 7f889dd9b640 1 -- 172.21.15.16:0/1660692558 learned_addr learned my addr 172.21.15.16:0/1660692558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.153+0000 7f889ed9d640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f888c0a32d0 0x7f888c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.154+0000 7f889ed9d640 1 --2- 172.21.15.16:0/1660692558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae88b9d0 0x55e8ae88bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.154+0000 7f889e59c640 1 --2- 172.21.15.16:0/1660692558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88c2f0 0x55e8ae896670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.154+0000 7f88977fe640 1 -- 172.21.15.16:0/3457249049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f88880cee80 con 0x7f88a0101de0 2024-09-19T09:26:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.154+0000 7f889dd9b640 1 -- 172.21.15.16:0/1660692558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88c2f0 msgr2=0x55e8ae896670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.154+0000 7f889dd9b640 1 --2- 172.21.15.16:0/1660692558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88c2f0 0x55e8ae896670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.154+0000 7f889dd9b640 1 -- 172.21.15.16:0/1660692558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae88b9d0 msgr2=0x55e8ae88bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.154+0000 7f889dd9b640 1 --2- 172.21.15.16:0/1660692558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae88b9d0 0x55e8ae88bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.154+0000 7f889dd9b640 1 -- 172.21.15.16:0/1660692558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8ae7a6130 con 0x55e8ae896bb0 2024-09-19T09:26:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.154+0000 7f889ed9d640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f888c0a32d0 0x7f888c0a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8898013460 tx=0x7f8898001ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.154+0000 7f889dd9b640 1 --2- 172.21.15.16:0/1660692558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae896bb0 0x55e8ae898fa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8890005ab0 tx=0x7f8890009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e018b8c2edb4a7b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.154+0000 7f8894ff9640 1 -- 172.21.15.16:0/1660692558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889003f070 con 0x55e8ae896bb0 2024-09-19T09:26:46.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.154+0000 7f8894ff9640 1 -- 172.21.15.16:0/1660692558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88900317b0 con 0x55e8ae896bb0 2024-09-19T09:26:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.154+0000 7f8894ff9640 1 -- 172.21.15.16:0/1660692558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890035070 con 0x55e8ae896bb0 2024-09-19T09:26:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.155+0000 7f88a6fcb740 1 -- 172.21.15.16:0/1660692558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae896bb0 msgr2=0x55e8ae898fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.155+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/1660692558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae896bb0 0x55e8ae898fa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8890005ab0 tx=0x7f8890009680 comp rx=0 tx=0).stop 2024-09-19T09:26:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.155+0000 7f88a6fcb740 1 -- 172.21.15.16:0/1660692558 shutdown_connections 2024-09-19T09:26:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.155+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/1660692558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88c2f0 0x55e8ae896670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.155+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/1660692558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae896bb0 0x55e8ae898fa0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.155+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/1660692558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae88b9d0 0x55e8ae88bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.155+0000 7f88a6fcb740 1 -- 172.21.15.16:0/1660692558 >> 172.21.15.16:0/1660692558 conn(0x55e8ae887f60 msgr2=0x55e8ae888340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.155+0000 7f88a6fcb740 1 -- 172.21.15.16:0/1660692558 shutdown_connections 2024-09-19T09:26:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.155+0000 7f88a6fcb740 1 -- 172.21.15.16:0/1660692558 wait complete. 2024-09-19T09:26:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.155+0000 7f88a6fcb740 1 -- start start 2024-09-19T09:26:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f88a6fcb740 1 Processor -- start 2024-09-19T09:26:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f88a6fcb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88b9d0 0x55e8ae8963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f88a6fcb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88c2f0 0x55e8ae893700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f88a6fcb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae896bb0 0x55e8ae893c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f88a6fcb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae899780 con 0x55e8ae88c2f0 2024-09-19T09:26:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f88a6fcb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae7acf20 con 0x55e8ae896bb0 2024-09-19T09:26:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f88a6fcb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae899900 con 0x55e8ae88b9d0 2024-09-19T09:26:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f88a6fcb740 10 client.0 _ll_register_callbacks cb 0x55e8ae776030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f889dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88c2f0 0x55e8ae893700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f889dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88c2f0 0x55e8ae893700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46592/0 (socket says 172.21.15.16:46592) 2024-09-19T09:26:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f889dd9b640 1 -- 172.21.15.16:0/91966772 learned_addr learned my addr 172.21.15.16:0/91966772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f889e59c640 1 --2- 172.21.15.16:0/91966772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88b9d0 0x55e8ae8963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f889dd9b640 1 -- 172.21.15.16:0/91966772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88b9d0 msgr2=0x55e8ae8963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f889dd9b640 1 --2- 172.21.15.16:0/91966772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88b9d0 0x55e8ae8963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f889dd9b640 1 -- 172.21.15.16:0/91966772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae896bb0 msgr2=0x55e8ae893c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f889ed9d640 1 --2- 172.21.15.16:0/91966772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae896bb0 0x55e8ae893c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f889dd9b640 1 --2- 172.21.15.16:0/91966772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae896bb0 0x55e8ae893c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f889dd9b640 1 -- 172.21.15.16:0/91966772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8ae5632b0 con 0x55e8ae88c2f0 2024-09-19T09:26:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f889dd9b640 1 --2- 172.21.15.16:0/91966772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88c2f0 0x55e8ae893700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8890006010 tx=0x7f889002ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66c14d6a9eb2586 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.156+0000 7f8885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:26:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f8885ffb640 1 -- 172.21.15.16:0/91966772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889003f050 con 0x55e8ae88c2f0 2024-09-19T09:26:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f8885ffb640 1 -- 172.21.15.16:0/91966772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8890004240 con 0x55e8ae88c2f0 2024-09-19T09:26:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f88a6fcb740 1 -- 172.21.15.16:0/91966772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88900317b0 con 0x55e8ae88c2f0 2024-09-19T09:26:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f8885ffb640 1 -- 172.21.15.16:0/91966772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8890035040 con 0x55e8ae88c2f0 2024-09-19T09:26:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f88a6fcb740 10 client.14985 fetch_fsmap learned FSMap version 27 2024-09-19T09:26:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f88a6fcb740 1 -- 172.21.15.16:0/91966772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e8ae8a5010 con 0x55e8ae88c2f0 2024-09-19T09:26:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f8885ffb640 1 -- 172.21.15.16:0/91966772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88900317b0 con 0x55e8ae88c2f0 2024-09-19T09:26:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f8885ffb640 1 -- 172.21.15.16:0/91966772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f88900047e0 con 0x55e8ae88c2f0 2024-09-19T09:26:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f88a6fcb740 10 client.14985 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:26:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f88a6fcb740 10 client.14985 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:26:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f88a6fcb740 20 client.14985 populate_metadata read hostname 'smithi016' 2024-09-19T09:26:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/91966772 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55e8ae8a71b0 0x55e8ae8a8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f88a6fcb740 4 client.14985 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.157+0000 7f88a6fcb740 1 -- 172.21.15.16:0/91966772 --> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8ae7ff600 con 0x55e8ae8a71b0 2024-09-19T09:26:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.158+0000 7f889e59c640 1 --2- 172.21.15.16:0/91966772 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55e8ae8a71b0 0x55e8ae8a8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.158+0000 7f8885ffb640 10 client.14985 ms_handle_connect on v2:172.21.15.97:6836/2018284957 2024-09-19T09:26:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.158+0000 7f889e59c640 1 --2- 172.21.15.16:0/91966772 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55e8ae8a71b0 0x55e8ae8a8620 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=807d11bd243ce56 server_cookie=343033ffc08bc0d5 in_seq=0 out_seq=0 2024-09-19T09:26:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f8885ffb640 1 -- 172.21.15.16:0/91966772 <== mds.2 v2:172.21.15.97:6836/2018284957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8ae7ff600 con 0x55e8ae8a71b0 2024-09-19T09:26:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f8885ffb640 10 client.14985 handle_command_reply: tid=0 2024-09-19T09:26:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f88a6fcb740 1 client.14985 shutdown 2024-09-19T09:26:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f88a6fcb740 1 -- 172.21.15.16:0/91966772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88c2f0 msgr2=0x55e8ae893700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/91966772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88c2f0 0x55e8ae893700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8890006010 tx=0x7f889002ab10 comp rx=0 tx=0).stop 2024-09-19T09:26:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f88a6fcb740 1 -- 172.21.15.16:0/91966772 shutdown_connections 2024-09-19T09:26:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/91966772 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55e8ae8a71b0 0x55e8ae8a8620 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/91966772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae896bb0 0x55e8ae893c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f8885ffb640 0 client.14985 ms_handle_reset on v2:172.21.15.97:6836/2018284957 2024-09-19T09:26:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/91966772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88c2f0 0x55e8ae893700 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/91966772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88b9d0 0x55e8ae8963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f88a6fcb740 1 -- 172.21.15.16:0/91966772 >> 172.21.15.16:0/91966772 conn(0x55e8ae887f60 msgr2=0x55e8ae88dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f88a6fcb740 1 -- 172.21.15.16:0/91966772 shutdown_connections 2024-09-19T09:26:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f88a6fcb740 1 -- 172.21.15.16:0/91966772 wait complete. 2024-09-19T09:26:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.159+0000 7f88a6fcb740 20 client.14985 trim_cache size 0 max 16384 2024-09-19T09:26:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.177+0000 7f88a6fcb740 1 Processor -- start 2024-09-19T09:26:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.177+0000 7f88a6fcb740 1 -- start start 2024-09-19T09:26:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f88a6fcb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 0x55e8ae8a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f88a6fcb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88c2f0 0x55e8ae8963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f88a6fcb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae896bb0 0x55e8ae886ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f88a6fcb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae8a6960 con 0x55e8ae88b9d0 2024-09-19T09:26:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f88a6fcb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae8a7030 con 0x55e8ae896bb0 2024-09-19T09:26:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f88a6fcb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae8a7700 con 0x55e8ae88c2f0 2024-09-19T09:26:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f889e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 0x55e8ae8a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f889e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 0x55e8ae8a6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46596/0 (socket says 172.21.15.16:46596) 2024-09-19T09:26:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f889e59c640 1 -- 172.21.15.16:0/3934713899 learned_addr learned my addr 172.21.15.16:0/3934713899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f889e59c640 1 -- 172.21.15.16:0/3934713899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88c2f0 msgr2=0x55e8ae8963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f889e59c640 1 --2- 172.21.15.16:0/3934713899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88c2f0 0x55e8ae8963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f889e59c640 1 -- 172.21.15.16:0/3934713899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae896bb0 msgr2=0x55e8ae886ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f889e59c640 1 --2- 172.21.15.16:0/3934713899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae896bb0 0x55e8ae886ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f889e59c640 1 -- 172.21.15.16:0/3934713899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8ae88cee0 con 0x55e8ae88b9d0 2024-09-19T09:26:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f889e59c640 1 --2- 172.21.15.16:0/3934713899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 0x55e8ae8a6ae0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f88880093d0 tx=0x7f888801c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69759b09aa7f697 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f8887fff640 1 -- 172.21.15.16:0/3934713899 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888801e040 con 0x55e8ae88b9d0 2024-09-19T09:26:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f8887fff640 1 -- 172.21.15.16:0/3934713899 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88880a4080 con 0x55e8ae88b9d0 2024-09-19T09:26:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.178+0000 7f8887fff640 1 -- 172.21.15.16:0/3934713899 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888010ca0 con 0x55e8ae88b9d0 2024-09-19T09:26:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.179+0000 7f88a6fcb740 1 -- 172.21.15.16:0/3934713899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 msgr2=0x55e8ae8a6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.179+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/3934713899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 0x55e8ae8a6ae0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f88880093d0 tx=0x7f888801c310 comp rx=0 tx=0).stop 2024-09-19T09:26:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.179+0000 7f88a6fcb740 1 -- 172.21.15.16:0/3934713899 shutdown_connections 2024-09-19T09:26:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.179+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/3934713899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae88c2f0 0x55e8ae8963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.179+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/3934713899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae896bb0 0x55e8ae886ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.179+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/3934713899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 0x55e8ae8a6ae0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.179+0000 7f88a6fcb740 1 -- 172.21.15.16:0/3934713899 >> 172.21.15.16:0/3934713899 conn(0x55e8ae887f60 msgr2=0x55e8ae8867e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.179+0000 7f88a6fcb740 1 -- 172.21.15.16:0/3934713899 shutdown_connections 2024-09-19T09:26:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.179+0000 7f88a6fcb740 1 -- 172.21.15.16:0/3934713899 wait complete. 2024-09-19T09:26:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.179+0000 7f88a6fcb740 1 -- start start 2024-09-19T09:26:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.179+0000 7f88a6fcb740 1 Processor -- start 2024-09-19T09:26:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f88a6fcb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 0x55e8ae909880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f88a6fcb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae88c2f0 0x55e8ae909dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f88a6fcb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae896bb0 0x55e8ae90a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f88a6fcb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae8a84a0 con 0x55e8ae88b9d0 2024-09-19T09:26:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f88a6fcb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae8a7dd0 con 0x55e8ae88c2f0 2024-09-19T09:26:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f88a6fcb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae8a8b70 con 0x55e8ae896bb0 2024-09-19T09:26:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f88a6fcb740 10 client.0 _ll_register_callbacks cb 0x7f8888017f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f889e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 0x55e8ae909880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f889e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 0x55e8ae909880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46608/0 (socket says 172.21.15.16:46608) 2024-09-19T09:26:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f889e59c640 1 -- 172.21.15.16:0/2956546265 learned_addr learned my addr 172.21.15.16:0/2956546265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f889dd9b640 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae88c2f0 0x55e8ae909dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f889ed9d640 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae896bb0 0x55e8ae90a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f889e59c640 1 -- 172.21.15.16:0/2956546265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae896bb0 msgr2=0x55e8ae90a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f889e59c640 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae896bb0 0x55e8ae90a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f889e59c640 1 -- 172.21.15.16:0/2956546265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae88c2f0 msgr2=0x55e8ae909dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f889e59c640 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae88c2f0 0x55e8ae909dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f889e59c640 1 -- 172.21.15.16:0/2956546265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8ae914b50 con 0x55e8ae88b9d0 2024-09-19T09:26:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f889dd9b640 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae88c2f0 0x55e8ae909dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f889e59c640 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 0x55e8ae909880 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8888097870 tx=0x7f888800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee0a1a37fa61725 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f8885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:26:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f8885ffb640 1 -- 172.21.15.16:0/2956546265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88880b4060 con 0x55e8ae88b9d0 2024-09-19T09:26:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f8885ffb640 1 -- 172.21.15.16:0/2956546265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8888010ca0 con 0x55e8ae88b9d0 2024-09-19T09:26:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f88a6fcb740 1 -- 172.21.15.16:0/2956546265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88880a4080 con 0x55e8ae88b9d0 2024-09-19T09:26:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f8885ffb640 1 -- 172.21.15.16:0/2956546265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8888099710 con 0x55e8ae88b9d0 2024-09-19T09:26:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.180+0000 7f88a6fcb740 10 client.14991 fetch_fsmap learned FSMap version 27 2024-09-19T09:26:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.181+0000 7f88a6fcb740 1 -- 172.21.15.16:0/2956546265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e8ae915e30 con 0x55e8ae88b9d0 2024-09-19T09:26:46.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.181+0000 7f8885ffb640 1 -- 172.21.15.16:0/2956546265 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88880a4080 con 0x55e8ae88b9d0 2024-09-19T09:26:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.181+0000 7f8885ffb640 1 -- 172.21.15.16:0/2956546265 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f88880b4200 con 0x55e8ae88b9d0 2024-09-19T09:26:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.181+0000 7f88a6fcb740 10 client.14991 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:26:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.181+0000 7f88a6fcb740 10 client.14991 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:26:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.181+0000 7f88a6fcb740 20 client.14991 populate_metadata read hostname 'smithi016' 2024-09-19T09:26:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.181+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55e8ae91a3b0 0x55e8ae9190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.181+0000 7f88a6fcb740 4 client.14991 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:26:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.181+0000 7f88a6fcb740 1 -- 172.21.15.16:0/2956546265 --> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] -- command(tid 0: {"prefix": "status"}) -- 0x55e8ae7f9b10 con 0x55e8ae91a3b0 2024-09-19T09:26:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.181+0000 7f889dd9b640 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55e8ae91a3b0 0x55e8ae9190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f8885ffb640 10 client.14991 ms_handle_connect on v2:172.21.15.97:6836/2018284957 2024-09-19T09:26:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f889dd9b640 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55e8ae91a3b0 0x55e8ae9190c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e77722bedf3e59f server_cookie=93f2cb39022f4cc7 in_seq=0 out_seq=0 2024-09-19T09:26:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f8885ffb640 1 -- 172.21.15.16:0/2956546265 <== mds.2 v2:172.21.15.97:6836/2018284957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55e8ae7f9b10 con 0x55e8ae91a3b0 2024-09-19T09:26:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f8885ffb640 10 client.14991 handle_command_reply: tid=0 2024-09-19T09:26:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f88a6fcb740 1 client.14991 shutdown 2024-09-19T09:26:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f88a6fcb740 1 -- 172.21.15.16:0/2956546265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 msgr2=0x55e8ae909880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 0x55e8ae909880 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8888097870 tx=0x7f888800bf90 comp rx=0 tx=0).stop 2024-09-19T09:26:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f88a6fcb740 1 -- 172.21.15.16:0/2956546265 shutdown_connections 2024-09-19T09:26:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55e8ae91a3b0 0x55e8ae9190c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8ae896bb0 0x55e8ae90a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8ae88c2f0 0x55e8ae909dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f88a6fcb740 1 --2- 172.21.15.16:0/2956546265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ae88b9d0 0x55e8ae909880 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f88a6fcb740 1 -- 172.21.15.16:0/2956546265 >> 172.21.15.16:0/2956546265 conn(0x55e8ae887f60 msgr2=0x55e8ae887000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f8885ffb640 0 client.14991 ms_handle_reset on v2:172.21.15.97:6836/2018284957 2024-09-19T09:26:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f88a6fcb740 1 -- 172.21.15.16:0/2956546265 shutdown_connections 2024-09-19T09:26:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f88a6fcb740 1 -- 172.21.15.16:0/2956546265 wait complete. 2024-09-19T09:26:46.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.182+0000 7f88a6fcb740 20 client.14991 trim_cache size 0 max 16384 2024-09-19T09:26:46.205 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:26:46.205 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:26:46.205 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-19T09:26:46.205 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24470, 2024-09-19T09:26:46.205 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:26:46.205 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:26:46.205 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:26:46.205 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 67.856382312999997, 2024-09-19T09:26:46.206 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-19T09:26:46.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:26:46.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:26:46.206 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 97.705754102 2024-09-19T09:26:46.206 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:26:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.183+0000 7f88a4bf2640 1 -- 172.21.15.16:0/3457249049 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f888c0a32d0 msgr2=0x7f888c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.183+0000 7f88a4bf2640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f888c0a32d0 0x7f888c0a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8898013460 tx=0x7f8898001ba0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.183+0000 7f88a4bf2640 1 -- 172.21.15.16:0/3457249049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a0101de0 msgr2=0x7f88a01985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.183+0000 7f88a4bf2640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a0101de0 0x7f88a01985b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f888800b4a0 tx=0x7f888800b960 comp rx=0 tx=0).stop 2024-09-19T09:26:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.183+0000 7f88a4bf2640 1 -- 172.21.15.16:0/3457249049 shutdown_connections 2024-09-19T09:26:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.183+0000 7f88a4bf2640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f888c0a32d0 0x7f888c0a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.183+0000 7f88a4bf2640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88a0106f10 0x7f88a019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.183+0000 7f88a4bf2640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88a0102790 0x7f88a0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.183+0000 7f88a4bf2640 1 --2- 172.21.15.16:0/3457249049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a0101de0 0x7f88a01985b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.183+0000 7f88a4bf2640 1 -- 172.21.15.16:0/3457249049 >> 172.21.15.16:0/3457249049 conn(0x7f88a00fd8c0 msgr2=0x7f88a00fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.183+0000 7f88a4bf2640 1 -- 172.21.15.16:0/3457249049 shutdown_connections 2024-09-19T09:26:46.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:46.183+0000 7f88a4bf2640 1 -- 172.21.15.16:0/3457249049 wait complete. 2024-09-19T09:26:46.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T09:26:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:47.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:46 smithi142 ceph-mon[35032]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:47.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:46 smithi097 ceph-mon[34864]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:47.213 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:46 smithi016 ceph-mon[30979]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.539+0000 7f0ac6c3b640 1 -- 172.21.15.16:0/808787040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac0102790 msgr2=0x7f0ac01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.539+0000 7f0ac6c3b640 1 --2- 172.21.15.16:0/808787040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac0102790 0x7f0ac01067e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009ca0 tx=0x7f0ab002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.539+0000 7f0ac6c3b640 1 -- 172.21.15.16:0/808787040 shutdown_connections 2024-09-19T09:26:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.539+0000 7f0ac6c3b640 1 --2- 172.21.15.16:0/808787040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac0106f10 0x7f0ac010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.539+0000 7f0ac6c3b640 1 --2- 172.21.15.16:0/808787040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac0102790 0x7f0ac01067e0 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009ca0 tx=0x7f0ab002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.539+0000 7f0ac6c3b640 1 --2- 172.21.15.16:0/808787040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac0101de0 0x7f0ac01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.539+0000 7f0ac6c3b640 1 -- 172.21.15.16:0/808787040 >> 172.21.15.16:0/808787040 conn(0x7f0ac00fd8c0 msgr2=0x7f0ac00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:47.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.540+0000 7f0ac6c3b640 1 -- 172.21.15.16:0/808787040 shutdown_connections 2024-09-19T09:26:47.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.540+0000 7f0ac6c3b640 1 -- 172.21.15.16:0/808787040 wait complete. 2024-09-19T09:26:47.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.540+0000 7f0ac6c3b640 1 Processor -- start 2024-09-19T09:26:47.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0ac6c3b640 1 -- start start 2024-09-19T09:26:47.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0ac6c3b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac0101de0 0x7f0ac01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0ac6c3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac0106f10 0x7f0ac01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0ac6c3b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac010de60 0x7f0ac010e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0ac6c3b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac0111f90 con 0x7f0ac0106f10 2024-09-19T09:26:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0ac6c3b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac0111e10 con 0x7f0ac010de60 2024-09-19T09:26:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0ac6c3b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac0112110 con 0x7f0ac0101de0 2024-09-19T09:26:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0abffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac0106f10 0x7f0ac01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0abffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac0106f10 0x7f0ac01a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46626/0 (socket says 172.21.15.16:46626) 2024-09-19T09:26:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0abffff640 1 -- 172.21.15.16:0/3033825956 learned_addr learned my addr 172.21.15.16:0/3033825956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0ac49d7640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac0101de0 0x7f0ac01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0ac51d8640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac010de60 0x7f0ac010e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0abffff640 1 -- 172.21.15.16:0/3033825956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac0101de0 msgr2=0x7f0ac01a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0abffff640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac0101de0 0x7f0ac01a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0abffff640 1 -- 172.21.15.16:0/3033825956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac010de60 msgr2=0x7f0ac010e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0abffff640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac010de60 0x7f0ac010e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0abffff640 1 -- 172.21.15.16:0/3033825956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac010ea40 con 0x7f0ac0106f10 2024-09-19T09:26:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.541+0000 7f0abffff640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac0106f10 0x7f0ac01a0b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009bf0 tx=0x7f0ab0030f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.542+0000 7f0abdffb640 1 -- 172.21.15.16:0/3033825956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0031960 con 0x7f0ac0106f10 2024-09-19T09:26:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.542+0000 7f0abdffb640 1 -- 172.21.15.16:0/3033825956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ab0046960 con 0x7f0ac0106f10 2024-09-19T09:26:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.542+0000 7f0ac6c3b640 1 -- 172.21.15.16:0/3033825956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac010ecd0 con 0x7f0ac0106f10 2024-09-19T09:26:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.542+0000 7f0abdffb640 1 -- 172.21.15.16:0/3033825956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab003a9d0 con 0x7f0ac0106f10 2024-09-19T09:26:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.542+0000 7f0ac6c3b640 1 -- 172.21.15.16:0/3033825956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac01a7740 con 0x7f0ac0106f10 2024-09-19T09:26:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.544+0000 7f0abdffb640 1 -- 172.21.15.16:0/3033825956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0ab0046b10 con 0x7f0ac0106f10 2024-09-19T09:26:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.544+0000 7f0ac9014740 1 Processor -- start 2024-09-19T09:26:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.544+0000 7f0ac9014740 1 -- start start 2024-09-19T09:26:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac9014740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 0x5598808dadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac9014740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808e5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac9014740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 0x5598808e7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac9014740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559880841090 con 0x5598808db2f0 2024-09-19T09:26:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac9014740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5598807e8a70 con 0x5598808da9d0 2024-09-19T09:26:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac9014740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55988083d050 con 0x5598808e5bb0 2024-09-19T09:26:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0abdffb640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0aac0a3350 0x7f0aac0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac49d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808e5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac49d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808e5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46630/0 (socket says 172.21.15.16:46630) 2024-09-19T09:26:47.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac49d7640 1 -- 172.21.15.16:0/3100682984 learned_addr learned my addr 172.21.15.16:0/3100682984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:47.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac51d8640 1 --2- 172.21.15.16:0/3100682984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 0x5598808dadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac51d8640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0aac0a3350 0x7f0aac0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0abdffb640 1 -- 172.21.15.16:0/3033825956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0ab00f0dd0 con 0x7f0ac0106f10 2024-09-19T09:26:47.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0abffff640 1 --2- 172.21.15.16:0/3100682984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 0x5598808e7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac49d7640 1 -- 172.21.15.16:0/3100682984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 msgr2=0x5598808e7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:47.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac49d7640 1 --2- 172.21.15.16:0/3100682984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 0x5598808e7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac49d7640 1 -- 172.21.15.16:0/3100682984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 msgr2=0x5598808dadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:47.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac49d7640 1 --2- 172.21.15.16:0/3100682984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 0x5598808dadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac49d7640 1 -- 172.21.15.16:0/3100682984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598807f5130 con 0x5598808db2f0 2024-09-19T09:26:47.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac51d8640 1 --2- 172.21.15.16:0/3100682984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 0x5598808dadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:47.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.545+0000 7f0ac49d7640 1 --2- 172.21.15.16:0/3100682984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808e5670 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009a70 tx=0x7f0ab8009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b61e908c4a27d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.546+0000 7f0ab6ffd640 1 -- 172.21.15.16:0/3100682984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8017070 con 0x5598808db2f0 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.546+0000 7f0ac51d8640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0aac0a3350 0x7f0aac0a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80058c0 tx=0x7f0aa800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.546+0000 7f0ab6ffd640 1 -- 172.21.15.16:0/3100682984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ab8013470 con 0x5598808db2f0 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.546+0000 7f0ab6ffd640 1 -- 172.21.15.16:0/3100682984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8012630 con 0x5598808db2f0 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.546+0000 7f0ac9014740 1 -- 172.21.15.16:0/3100682984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 msgr2=0x5598808e5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.546+0000 7f0ac9014740 1 --2- 172.21.15.16:0/3100682984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808e5670 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009a70 tx=0x7f0ab8009f30 comp rx=0 tx=0).stop 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.546+0000 7f0ac9014740 1 -- 172.21.15.16:0/3100682984 shutdown_connections 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.546+0000 7f0ac9014740 1 --2- 172.21.15.16:0/3100682984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 0x5598808e7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.546+0000 7f0ac9014740 1 --2- 172.21.15.16:0/3100682984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808e5670 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.546+0000 7f0ac9014740 1 --2- 172.21.15.16:0/3100682984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 0x5598808dadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.546+0000 7f0ac9014740 1 -- 172.21.15.16:0/3100682984 >> 172.21.15.16:0/3100682984 conn(0x5598808d6f60 msgr2=0x5598808d7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.546+0000 7f0ac9014740 1 -- 172.21.15.16:0/3100682984 shutdown_connections 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.547+0000 7f0ac9014740 1 -- 172.21.15.16:0/3100682984 wait complete. 2024-09-19T09:26:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.547+0000 7f0ac9014740 1 -- start start 2024-09-19T09:26:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.547+0000 7f0ac9014740 1 Processor -- start 2024-09-19T09:26:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.547+0000 7f0ac9014740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 0x5598808e53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.547+0000 7f0ac9014740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808e2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.547+0000 7f0ac9014740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 0x5598808e2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.547+0000 7f0ac9014740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5598808e8780 con 0x5598808db2f0 2024-09-19T09:26:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.547+0000 7f0ac9014740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5598807fbf20 con 0x5598808da9d0 2024-09-19T09:26:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.547+0000 7f0ac9014740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5598808e8900 con 0x5598808e5bb0 2024-09-19T09:26:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.547+0000 7f0ac9014740 10 client.0 _ll_register_callbacks cb 0x5598807c5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.547+0000 7f0abffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808e2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.547+0000 7f0abffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808e2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46640/0 (socket says 172.21.15.16:46640) 2024-09-19T09:26:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0abffff640 1 -- 172.21.15.16:0/1275441732 learned_addr learned my addr 172.21.15.16:0/1275441732 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0ac49d7640 1 --2- 172.21.15.16:0/1275441732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 0x5598808e53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0ac51d8640 1 --2- 172.21.15.16:0/1275441732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 0x5598808e2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0abffff640 1 -- 172.21.15.16:0/1275441732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 msgr2=0x5598808e2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0abffff640 1 --2- 172.21.15.16:0/1275441732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 0x5598808e2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0abffff640 1 -- 172.21.15.16:0/1275441732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 msgr2=0x5598808e53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0abffff640 1 --2- 172.21.15.16:0/1275441732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 0x5598808e53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0abffff640 1 -- 172.21.15.16:0/1275441732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598805b22b0 con 0x5598808db2f0 2024-09-19T09:26:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0abffff640 1 --2- 172.21.15.16:0/1275441732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808e2740 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00b9d70 tx=0x7f0ab00bcdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d57ab78d82be2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0a9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:26:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0a9ffff640 1 -- 172.21.15.16:0/1275441732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab003ad70 con 0x5598808db2f0 2024-09-19T09:26:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0a9ffff640 1 -- 172.21.15.16:0/1275441732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0ab0004b40 con 0x5598808db2f0 2024-09-19T09:26:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0ac9014740 1 -- 172.21.15.16:0/1275441732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab8013470 con 0x5598808db2f0 2024-09-19T09:26:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0a9ffff640 1 -- 172.21.15.16:0/1275441732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ab00bd3a0 con 0x5598808db2f0 2024-09-19T09:26:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0ac9014740 10 client.15003 fetch_fsmap learned FSMap version 27 2024-09-19T09:26:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.548+0000 7f0ac9014740 1 -- 172.21.15.16:0/1275441732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5598808e8780 con 0x5598808db2f0 2024-09-19T09:26:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.549+0000 7f0a9ffff640 1 -- 172.21.15.16:0/1275441732 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ab8013470 con 0x5598808db2f0 2024-09-19T09:26:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.549+0000 7f0a9ffff640 1 -- 172.21.15.16:0/1275441732 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0ab004e030 con 0x5598808db2f0 2024-09-19T09:26:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.549+0000 7f0ac9014740 10 client.15003 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:26:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.549+0000 7f0ac9014740 10 client.15003 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:26:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.549+0000 7f0ac9014740 20 client.15003 populate_metadata read hostname 'smithi016' 2024-09-19T09:26:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.549+0000 7f0ac9014740 1 --2- 172.21.15.16:0/1275441732 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x5598808f6620 0x5598808f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.549+0000 7f0ac9014740 4 client.15003 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.549+0000 7f0ac9014740 1 -- 172.21.15.16:0/1275441732 --> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55988084e600 con 0x5598808f6620 2024-09-19T09:26:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.549+0000 7f0ac49d7640 1 --2- 172.21.15.16:0/1275441732 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x5598808f6620 0x5598808f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.549+0000 7f0ac49d7640 1 --2- 172.21.15.16:0/1275441732 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x5598808f6620 0x5598808f7a90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ecf42c9171671bc0 server_cookie=67ebfd30ccc8b45c in_seq=0 out_seq=0 2024-09-19T09:26:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.549+0000 7f0a9ffff640 10 client.15003 ms_handle_connect on v2:172.21.15.142:6838/3822134597 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.550+0000 7f0a9ffff640 1 -- 172.21.15.16:0/1275441732 <== mds.1 v2:172.21.15.142:6838/3822134597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55988084e600 con 0x5598808f6620 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.550+0000 7f0a9ffff640 10 client.15003 handle_command_reply: tid=0 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.550+0000 7f0ac9014740 1 client.15003 shutdown 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.550+0000 7f0ac9014740 1 -- 172.21.15.16:0/1275441732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 msgr2=0x5598808e2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.550+0000 7f0ac9014740 1 --2- 172.21.15.16:0/1275441732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808e2740 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00b9d70 tx=0x7f0ab00bcdc0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.550+0000 7f0ac9014740 1 -- 172.21.15.16:0/1275441732 shutdown_connections 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.550+0000 7f0ac9014740 1 --2- 172.21.15.16:0/1275441732 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x5598808f6620 0x5598808f7a90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.550+0000 7f0ac9014740 1 --2- 172.21.15.16:0/1275441732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 0x5598808e2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.550+0000 7f0ac9014740 1 --2- 172.21.15.16:0/1275441732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808e2740 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.550+0000 7f0ac9014740 1 --2- 172.21.15.16:0/1275441732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 0x5598808e53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.550+0000 7f0a9ffff640 0 client.15003 ms_handle_reset on v2:172.21.15.142:6838/3822134597 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.550+0000 7f0ac9014740 1 -- 172.21.15.16:0/1275441732 >> 172.21.15.16:0/1275441732 conn(0x5598808d6f60 msgr2=0x5598808dcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.551+0000 7f0ac9014740 1 -- 172.21.15.16:0/1275441732 shutdown_connections 2024-09-19T09:26:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.551+0000 7f0ac9014740 1 -- 172.21.15.16:0/1275441732 wait complete. 2024-09-19T09:26:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.551+0000 7f0ac9014740 20 client.15003 trim_cache size 0 max 16384 2024-09-19T09:26:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.568+0000 7f0ac9014740 1 Processor -- start 2024-09-19T09:26:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.568+0000 7f0ac9014740 1 -- start start 2024-09-19T09:26:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0ac9014740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 0x5598808e2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0ac9014740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808f5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0ac9014740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 0x5598808f5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0ac9014740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5598808f5dd0 con 0x5598808db2f0 2024-09-19T09:26:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0ac9014740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5598808f64a0 con 0x5598808da9d0 2024-09-19T09:26:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0ac9014740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5598808f6b70 con 0x5598808e5bb0 2024-09-19T09:26:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0abffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0abffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808f5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46644/0 (socket says 172.21.15.16:46644) 2024-09-19T09:26:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0abffff640 1 -- 172.21.15.16:0/521682134 learned_addr learned my addr 172.21.15.16:0/521682134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0ac49d7640 1 --2- 172.21.15.16:0/521682134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 0x5598808e2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0abffff640 1 -- 172.21.15.16:0/521682134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 msgr2=0x5598808f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0abffff640 1 --2- 172.21.15.16:0/521682134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 0x5598808f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0abffff640 1 -- 172.21.15.16:0/521682134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 msgr2=0x5598808e2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0abffff640 1 --2- 172.21.15.16:0/521682134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 0x5598808e2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0abffff640 1 -- 172.21.15.16:0/521682134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598808d6d90 con 0x5598808db2f0 2024-09-19T09:26:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0abffff640 1 --2- 172.21.15.16:0/521682134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808f5130 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00b9010 tx=0x7f0ab00bce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966e237757fecce4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0ab67fc640 1 -- 172.21.15.16:0/521682134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00da020 con 0x5598808db2f0 2024-09-19T09:26:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0ab67fc640 1 -- 172.21.15.16:0/521682134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ab0036040 con 0x5598808db2f0 2024-09-19T09:26:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.569+0000 7f0ab67fc640 1 -- 172.21.15.16:0/521682134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0004da0 con 0x5598808db2f0 2024-09-19T09:26:47.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.570+0000 7f0ac9014740 1 -- 172.21.15.16:0/521682134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 msgr2=0x5598808f5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:47.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.570+0000 7f0ac9014740 1 --2- 172.21.15.16:0/521682134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808f5130 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00b9010 tx=0x7f0ab00bce30 comp rx=0 tx=0).stop 2024-09-19T09:26:47.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.570+0000 7f0ac9014740 1 -- 172.21.15.16:0/521682134 shutdown_connections 2024-09-19T09:26:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.570+0000 7f0ac9014740 1 --2- 172.21.15.16:0/521682134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808e5bb0 0x5598808f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.570+0000 7f0ac9014740 1 --2- 172.21.15.16:0/521682134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808db2f0 0x5598808f5130 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.570+0000 7f0ac9014740 1 --2- 172.21.15.16:0/521682134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808da9d0 0x5598808e2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.570+0000 7f0ac9014740 1 -- 172.21.15.16:0/521682134 >> 172.21.15.16:0/521682134 conn(0x5598808d6f60 msgr2=0x5598808e1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.570+0000 7f0ac9014740 1 -- 172.21.15.16:0/521682134 shutdown_connections 2024-09-19T09:26:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.570+0000 7f0ac9014740 1 -- 172.21.15.16:0/521682134 wait complete. 2024-09-19T09:26:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.570+0000 7f0ac9014740 1 -- start start 2024-09-19T09:26:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.570+0000 7f0ac9014740 1 Processor -- start 2024-09-19T09:26:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac9014740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808da9d0 0x5598808f9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac9014740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808db2f0 0x5598808f9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac9014740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808e5bb0 0x55988095ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac9014740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5598808f7910 con 0x5598808da9d0 2024-09-19T09:26:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac9014740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5598808f7240 con 0x5598808e5bb0 2024-09-19T09:26:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac9014740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5598808f7fe0 con 0x5598808db2f0 2024-09-19T09:26:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac9014740 10 client.0 _ll_register_callbacks cb 0x7f0ab003a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac49d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808da9d0 0x5598808f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac49d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808da9d0 0x5598808f9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46652/0 (socket says 172.21.15.16:46652) 2024-09-19T09:26:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac49d7640 1 -- 172.21.15.16:0/357418348 learned_addr learned my addr 172.21.15.16:0/357418348 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0abffff640 1 --2- 172.21.15.16:0/357418348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808db2f0 0x5598808f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac51d8640 1 --2- 172.21.15.16:0/357418348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808e5bb0 0x55988095ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac49d7640 1 -- 172.21.15.16:0/357418348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808db2f0 msgr2=0x5598808f9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac49d7640 1 --2- 172.21.15.16:0/357418348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808db2f0 0x5598808f9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac49d7640 1 -- 172.21.15.16:0/357418348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808e5bb0 msgr2=0x55988095ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac49d7640 1 --2- 172.21.15.16:0/357418348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808e5bb0 0x55988095ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.571+0000 7f0ac49d7640 1 -- 172.21.15.16:0/357418348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988095f310 con 0x5598808da9d0 2024-09-19T09:26:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0ac49d7640 1 --2- 172.21.15.16:0/357418348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808da9d0 0x5598808f9230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8004700 tx=0x7f0ab8013df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355a67aeaf313f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0a9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0a9ffff640 1 -- 172.21.15.16:0/357418348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab802a070 con 0x5598808da9d0 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0a9ffff640 1 -- 172.21.15.16:0/357418348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0ab8013470 con 0x5598808da9d0 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0ac9014740 1 -- 172.21.15.16:0/357418348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab0036040 con 0x5598808da9d0 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0a9ffff640 1 -- 172.21.15.16:0/357418348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ab8017040 con 0x5598808da9d0 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0ac9014740 10 client.15009 fetch_fsmap learned FSMap version 27 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0ac9014740 1 -- 172.21.15.16:0/357418348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559880960610 con 0x5598808da9d0 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0a9ffff640 1 -- 172.21.15.16:0/357418348 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ab0036040 con 0x5598808da9d0 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0a9ffff640 1 -- 172.21.15.16:0/357418348 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0ab802d020 con 0x5598808da9d0 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0ac9014740 10 client.15009 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0ac9014740 10 client.15009 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0ac9014740 20 client.15009 populate_metadata read hostname 'smithi016' 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0ac9014740 1 --2- 172.21.15.16:0/357418348 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x559880964de0 0x559880963af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0ac9014740 4 client.15009 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:26:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.572+0000 7f0ac9014740 1 -- 172.21.15.16:0/357418348 --> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] -- command(tid 0: {"prefix": "status"}) -- 0x559880848b10 con 0x559880964de0 2024-09-19T09:26:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0abffff640 1 --2- 172.21.15.16:0/357418348 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x559880964de0 0x559880963af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0abffff640 1 --2- 172.21.15.16:0/357418348 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x559880964de0 0x559880963af0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95acb3a6a3dc86b3 server_cookie=d32301d47211adfd in_seq=0 out_seq=0 2024-09-19T09:26:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0a9ffff640 10 client.15009 ms_handle_connect on v2:172.21.15.142:6838/3822134597 2024-09-19T09:26:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0a9ffff640 1 -- 172.21.15.16:0/357418348 <== mds.1 v2:172.21.15.142:6838/3822134597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559880848b10 con 0x559880964de0 2024-09-19T09:26:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0a9ffff640 10 client.15009 handle_command_reply: tid=0 2024-09-19T09:26:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0ac9014740 1 client.15009 shutdown 2024-09-19T09:26:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0ac9014740 1 -- 172.21.15.16:0/357418348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808da9d0 msgr2=0x5598808f9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0ac9014740 1 --2- 172.21.15.16:0/357418348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808da9d0 0x5598808f9230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8004700 tx=0x7f0ab8013df0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0ac9014740 1 -- 172.21.15.16:0/357418348 shutdown_connections 2024-09-19T09:26:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0ac9014740 1 --2- 172.21.15.16:0/357418348 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x559880964de0 0x559880963af0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0ac9014740 1 --2- 172.21.15.16:0/357418348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5598808db2f0 0x5598808f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0ac9014740 1 --2- 172.21.15.16:0/357418348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598808e5bb0 0x55988095ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0ac9014740 1 --2- 172.21.15.16:0/357418348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598808da9d0 0x5598808f9230 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0ac9014740 1 -- 172.21.15.16:0/357418348 >> 172.21.15.16:0/357418348 conn(0x5598808d6f60 msgr2=0x5598808f5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.573+0000 7f0a9ffff640 0 client.15009 ms_handle_reset on v2:172.21.15.142:6838/3822134597 2024-09-19T09:26:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac9014740 1 -- 172.21.15.16:0/357418348 shutdown_connections 2024-09-19T09:26:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac9014740 1 -- 172.21.15.16:0/357418348 wait complete. 2024-09-19T09:26:47.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac9014740 20 client.15009 trim_cache size 0 max 16384 2024-09-19T09:26:47.580 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:26:47.580 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:26:47.580 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-19T09:26:47.580 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24484, 2024-09-19T09:26:47.580 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:26:47.581 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:26:47.581 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:26:47.581 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 70.255749168999998, 2024-09-19T09:26:47.581 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-19T09:26:47.581 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:26:47.581 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:26:47.581 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 96.603330052000004 2024-09-19T09:26:47.581 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:26:47.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac6c3b640 1 -- 172.21.15.16:0/3033825956 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0aac0a3350 msgr2=0x7f0aac0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:47.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac6c3b640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0aac0a3350 0x7f0aac0a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80058c0 tx=0x7f0aa800a250 comp rx=0 tx=0).stop 2024-09-19T09:26:47.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac6c3b640 1 -- 172.21.15.16:0/3033825956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac0106f10 msgr2=0x7f0ac01a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:47.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac6c3b640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac0106f10 0x7f0ac01a0b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009bf0 tx=0x7f0ab0030f80 comp rx=0 tx=0).stop 2024-09-19T09:26:47.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac6c3b640 1 -- 172.21.15.16:0/3033825956 shutdown_connections 2024-09-19T09:26:47.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac6c3b640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ac010de60 0x7f0ac010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac6c3b640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ac0106f10 0x7f0ac01a0b50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac6c3b640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0aac0a3350 0x7f0aac0a5810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac6c3b640 1 --2- 172.21.15.16:0/3033825956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac0101de0 0x7f0ac01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:47.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac6c3b640 1 -- 172.21.15.16:0/3033825956 >> 172.21.15.16:0/3033825956 conn(0x7f0ac00fd8c0 msgr2=0x7f0ac0074780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:47.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac6c3b640 1 -- 172.21.15.16:0/3033825956 shutdown_connections 2024-09-19T09:26:47.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:47.574+0000 7f0ac6c3b640 1 -- 172.21.15.16:0/3033825956 wait complete. 2024-09-19T09:26:48.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T09:26:49.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:48 smithi142 ceph-mon[35032]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:26:49.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:48 smithi097 ceph-mon[34864]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:26:49.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:48 smithi016 ceph-mon[30979]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:26:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.730+0000 7f02c3948640 1 -- 172.21.15.16:0/1703788019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc102ef0 msgr2=0x7f02bc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.730+0000 7f02c3948640 1 --2- 172.21.15.16:0/1703788019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc102ef0 0x7f02bc10da40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009ca0 tx=0x7f02b402a950 comp rx=0 tx=0).stop 2024-09-19T09:26:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.730+0000 7f02c3948640 1 -- 172.21.15.16:0/1703788019 shutdown_connections 2024-09-19T09:26:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.730+0000 7f02c3948640 1 --2- 172.21.15.16:0/1703788019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02bc102570 0x7f02bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.730+0000 7f02c3948640 1 --2- 172.21.15.16:0/1703788019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc102ef0 0x7f02bc10da40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.730+0000 7f02c3948640 1 --2- 172.21.15.16:0/1703788019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02bc100e20 0x7f02bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.730+0000 7f02c3948640 1 -- 172.21.15.16:0/1703788019 >> 172.21.15.16:0/1703788019 conn(0x7f02bc0fc7d0 msgr2=0x7f02bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.730+0000 7f02c3948640 1 -- 172.21.15.16:0/1703788019 shutdown_connections 2024-09-19T09:26:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.730+0000 7f02c3948640 1 -- 172.21.15.16:0/1703788019 wait complete. 2024-09-19T09:26:49.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.730+0000 7f02c3948640 1 Processor -- start 2024-09-19T09:26:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.730+0000 7f02c3948640 1 -- start start 2024-09-19T09:26:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c3948640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02bc100e20 0x7f02bc197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c3948640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02bc102570 0x7f02bc1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c3948640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc102ef0 0x7f02bc197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c3948640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc110fc0 con 0x7f02bc102ef0 2024-09-19T09:26:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c3948640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc110e40 con 0x7f02bc100e20 2024-09-19T09:26:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c3948640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc111140 con 0x7f02bc102570 2024-09-19T09:26:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c1ee5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc102ef0 0x7f02bc197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c1ee5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc102ef0 0x7f02bc197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46664/0 (socket says 172.21.15.16:46664) 2024-09-19T09:26:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c1ee5640 1 -- 172.21.15.16:0/3156925278 learned_addr learned my addr 172.21.15.16:0/3156925278 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c16e4640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02bc100e20 0x7f02bc197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02bc102570 0x7f02bc1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c1ee5640 1 -- 172.21.15.16:0/3156925278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02bc102570 msgr2=0x7f02bc1976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c1ee5640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02bc102570 0x7f02bc1976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c1ee5640 1 -- 172.21.15.16:0/3156925278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02bc100e20 msgr2=0x7f02bc197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c1ee5640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02bc100e20 0x7f02bc197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c1ee5640 1 -- 172.21.15.16:0/3156925278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02bc1a69d0 con 0x7f02bc102ef0 2024-09-19T09:26:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02bc102570 0x7f02bc1976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.731+0000 7f02c1ee5640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc102ef0 0x7f02bc197be0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f02b4006010 tx=0x7f02b402f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.732+0000 7f02b27fc640 1 -- 172.21.15.16:0/3156925278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4034040 con 0x7f02bc102ef0 2024-09-19T09:26:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.732+0000 7f02c3948640 1 -- 172.21.15.16:0/3156925278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02bc1a6c60 con 0x7f02bc102ef0 2024-09-19T09:26:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.732+0000 7f02b27fc640 1 -- 172.21.15.16:0/3156925278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b4003e00 con 0x7f02bc102ef0 2024-09-19T09:26:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.732+0000 7f02b27fc640 1 -- 172.21.15.16:0/3156925278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4045480 con 0x7f02bc102ef0 2024-09-19T09:26:49.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.732+0000 7f02c3948640 1 -- 172.21.15.16:0/3156925278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02bc1a71c0 con 0x7f02bc102ef0 2024-09-19T09:26:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.734+0000 7f02b27fc640 1 -- 172.21.15.16:0/3156925278 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f02b4037020 con 0x7f02bc102ef0 2024-09-19T09:26:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.734+0000 7f02c5d21740 1 Processor -- start 2024-09-19T09:26:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c5d21740 1 -- start start 2024-09-19T09:26:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c5d21740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e4edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c5d21740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e59670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c5d21740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e5bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c5d21740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1db5090 con 0x55c2c1e4f2f0 2024-09-19T09:26:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c5d21740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1d5ca70 con 0x55c2c1e4e9d0 2024-09-19T09:26:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c5d21740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1db1050 con 0x55c2c1e59bb0 2024-09-19T09:26:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02b27fc640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f02a80a32d0 0x7f02a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c16e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e59670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c16e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e59670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46672/0 (socket says 172.21.15.16:46672) 2024-09-19T09:26:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c16e4640 1 -- 172.21.15.16:0/1035429281 learned_addr learned my addr 172.21.15.16:0/1035429281 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/1035429281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e4edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c1ee5640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f02a80a32d0 0x7f02a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/1035429281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e5bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c16e4640 1 -- 172.21.15.16:0/1035429281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e59bb0 msgr2=0x55c2c1e5bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c16e4640 1 --2- 172.21.15.16:0/1035429281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e5bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c16e4640 1 -- 172.21.15.16:0/1035429281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4e9d0 msgr2=0x55c2c1e4edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c16e4640 1 --2- 172.21.15.16:0/1035429281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e4edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c16e4640 1 -- 172.21.15.16:0/1035429281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c1d69130 con 0x55c2c1e4f2f0 2024-09-19T09:26:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/1035429281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e5bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.735+0000 7f02b27fc640 1 -- 172.21.15.16:0/3156925278 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f02b40f0b50 con 0x7f02bc102ef0 2024-09-19T09:26:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.736+0000 7f02c16e4640 1 --2- 172.21.15.16:0/1035429281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e59670 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f02b800ac10 tx=0x7f02b800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6742108ca8efa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.736+0000 7f02c1ee5640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f02a80a32d0 0x7f02a80a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f02b40bcde0 tx=0x7f02b408be20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.736+0000 7f02a77fe640 1 -- 172.21.15.16:0/1035429281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b801d070 con 0x55c2c1e4f2f0 2024-09-19T09:26:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.736+0000 7f02a77fe640 1 -- 172.21.15.16:0/1035429281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b8018b20 con 0x55c2c1e4f2f0 2024-09-19T09:26:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.736+0000 7f02a77fe640 1 -- 172.21.15.16:0/1035429281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b8010b50 con 0x55c2c1e4f2f0 2024-09-19T09:26:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.737+0000 7f02c5d21740 1 -- 172.21.15.16:0/1035429281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4f2f0 msgr2=0x55c2c1e59670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.737+0000 7f02c5d21740 1 --2- 172.21.15.16:0/1035429281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e59670 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f02b800ac10 tx=0x7f02b800f790 comp rx=0 tx=0).stop 2024-09-19T09:26:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.737+0000 7f02c5d21740 1 -- 172.21.15.16:0/1035429281 shutdown_connections 2024-09-19T09:26:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.737+0000 7f02c5d21740 1 --2- 172.21.15.16:0/1035429281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e5bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.737+0000 7f02c5d21740 1 --2- 172.21.15.16:0/1035429281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e59670 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.737+0000 7f02c5d21740 1 --2- 172.21.15.16:0/1035429281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e4edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.737+0000 7f02c5d21740 1 -- 172.21.15.16:0/1035429281 >> 172.21.15.16:0/1035429281 conn(0x55c2c1e4af60 msgr2=0x55c2c1e4b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.737+0000 7f02c5d21740 1 -- 172.21.15.16:0/1035429281 shutdown_connections 2024-09-19T09:26:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.737+0000 7f02c5d21740 1 -- 172.21.15.16:0/1035429281 wait complete. 2024-09-19T09:26:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.737+0000 7f02c5d21740 1 -- start start 2024-09-19T09:26:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.737+0000 7f02c5d21740 1 Processor -- start 2024-09-19T09:26:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c5d21740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e59470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c5d21740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c5d21740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e56d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c5d21740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1e5c780 con 0x55c2c1e4e9d0 2024-09-19T09:26:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c5d21740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1d6ff20 con 0x55c2c1e4f2f0 2024-09-19T09:26:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c5d21740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1e5c900 con 0x55c2c1e59bb0 2024-09-19T09:26:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c5d21740 10 client.0 _ll_register_callbacks cb 0x55c2c1d39030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c0ee3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e59470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c0ee3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e59470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46674/0 (socket says 172.21.15.16:46674) 2024-09-19T09:26:49.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c0ee3640 1 -- 172.21.15.16:0/2509373423 learned_addr learned my addr 172.21.15.16:0/2509373423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:49.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c16e4640 1 --2- 172.21.15.16:0/2509373423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/2509373423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e56d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c0ee3640 1 -- 172.21.15.16:0/2509373423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e59bb0 msgr2=0x55c2c1e56d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:49.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/2509373423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e56d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c0ee3640 1 -- 172.21.15.16:0/2509373423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 msgr2=0x55c2c1e567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/2509373423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c0ee3640 1 -- 172.21.15.16:0/2509373423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c1b262b0 con 0x55c2c1e4e9d0 2024-09-19T09:26:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/2509373423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e59470 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00af70 tx=0x7f02ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc116e7e0897dca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:26:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02a4ff9640 1 -- 172.21.15.16:0/2509373423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac01f060 con 0x55c2c1e4e9d0 2024-09-19T09:26:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02a4ff9640 1 -- 172.21.15.16:0/2509373423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f02ac014850 con 0x55c2c1e4e9d0 2024-09-19T09:26:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.738+0000 7f02c5d21740 1 -- 172.21.15.16:0/2509373423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02b8018b20 con 0x55c2c1e4e9d0 2024-09-19T09:26:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.739+0000 7f02a4ff9640 1 -- 172.21.15.16:0/2509373423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f02ac014a50 con 0x55c2c1e4e9d0 2024-09-19T09:26:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.739+0000 7f02c5d21740 10 client.15021 fetch_fsmap learned FSMap version 27 2024-09-19T09:26:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.739+0000 7f02c5d21740 1 -- 172.21.15.16:0/2509373423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2c1e68160 con 0x55c2c1e4e9d0 2024-09-19T09:26:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.739+0000 7f02a4ff9640 1 -- 172.21.15.16:0/2509373423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b8018b20 con 0x55c2c1e4e9d0 2024-09-19T09:26:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.739+0000 7f02a4ff9640 1 -- 172.21.15.16:0/2509373423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f02ac024070 con 0x55c2c1e4e9d0 2024-09-19T09:26:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.739+0000 7f02c5d21740 10 client.15021 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:26:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.739+0000 7f02c5d21740 10 client.15021 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:26:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.739+0000 7f02c5d21740 20 client.15021 populate_metadata read hostname 'smithi016' 2024-09-19T09:26:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.739+0000 7f02c5d21740 1 --2- 172.21.15.16:0/2509373423 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55c2c1e6a740 0x55c2c1e6bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.739+0000 7f02c5d21740 4 client.15021 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:26:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.739+0000 7f02c5d21740 1 -- 172.21.15.16:0/2509373423 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2c1dc2600 con 0x55c2c1e6a740 2024-09-19T09:26:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.739+0000 7f02c16e4640 1 --2- 172.21.15.16:0/2509373423 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55c2c1e6a740 0x55c2c1e6bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.740+0000 7f02c16e4640 1 --2- 172.21.15.16:0/2509373423 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55c2c1e6a740 0x55c2c1e6bbb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64f6f9a191237cfe server_cookie=cfd994d76c12b27c in_seq=0 out_seq=0 2024-09-19T09:26:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.740+0000 7f02a4ff9640 10 client.15021 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:26:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02a4ff9640 1 -- 172.21.15.16:0/2509373423 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2c1dc2600 con 0x55c2c1e6a740 2024-09-19T09:26:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02a4ff9640 10 client.15021 handle_command_reply: tid=0 2024-09-19T09:26:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02c5d21740 1 client.15021 shutdown 2024-09-19T09:26:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02c5d21740 1 -- 172.21.15.16:0/2509373423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4e9d0 msgr2=0x55c2c1e59470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02c5d21740 1 --2- 172.21.15.16:0/2509373423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e59470 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00af70 tx=0x7f02ac0046c0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02c5d21740 1 -- 172.21.15.16:0/2509373423 shutdown_connections 2024-09-19T09:26:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02c5d21740 1 --2- 172.21.15.16:0/2509373423 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55c2c1e6a740 0x55c2c1e6bbb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02c5d21740 1 --2- 172.21.15.16:0/2509373423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e56d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02c5d21740 1 --2- 172.21.15.16:0/2509373423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02c5d21740 1 --2- 172.21.15.16:0/2509373423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e59470 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02a4ff9640 0 client.15021 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:26:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02c5d21740 1 -- 172.21.15.16:0/2509373423 >> 172.21.15.16:0/2509373423 conn(0x55c2c1e4af60 msgr2=0x55c2c1e50c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02c5d21740 1 -- 172.21.15.16:0/2509373423 shutdown_connections 2024-09-19T09:26:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02c5d21740 1 -- 172.21.15.16:0/2509373423 wait complete. 2024-09-19T09:26:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.741+0000 7f02c5d21740 20 client.15021 trim_cache size 0 max 16384 2024-09-19T09:26:49.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c5d21740 1 Processor -- start 2024-09-19T09:26:49.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c5d21740 1 -- start start 2024-09-19T09:26:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c5d21740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e6bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c5d21740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e49ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c5d21740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e53a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c5d21740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1e69ef0 con 0x55c2c1e59bb0 2024-09-19T09:26:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c5d21740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1e6a5c0 con 0x55c2c1e4f2f0 2024-09-19T09:26:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c5d21740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1e6ac90 con 0x55c2c1e4e9d0 2024-09-19T09:26:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c0ee3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e53a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c0ee3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e53a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46690/0 (socket says 172.21.15.16:46690) 2024-09-19T09:26:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c0ee3640 1 -- 172.21.15.16:0/986744200 learned_addr learned my addr 172.21.15.16:0/986744200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c0ee3640 1 -- 172.21.15.16:0/986744200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e4e9d0 msgr2=0x55c2c1e6bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/986744200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e6bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c0ee3640 1 -- 172.21.15.16:0/986744200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 msgr2=0x55c2c1e49ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/986744200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e49ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c0ee3640 1 -- 172.21.15.16:0/986744200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c1e597e0 con 0x55c2c1e59bb0 2024-09-19T09:26:49.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.759+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/986744200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e53a40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02ac008000 tx=0x7f02ac00ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fb7d76b9e89a12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.760+0000 7f02a6ffd640 1 -- 172.21.15.16:0/986744200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac02a070 con 0x55c2c1e59bb0 2024-09-19T09:26:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.760+0000 7f02a6ffd640 1 -- 172.21.15.16:0/986744200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02ac014bf0 con 0x55c2c1e59bb0 2024-09-19T09:26:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.760+0000 7f02a6ffd640 1 -- 172.21.15.16:0/986744200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac026460 con 0x55c2c1e59bb0 2024-09-19T09:26:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.760+0000 7f02c5d21740 1 -- 172.21.15.16:0/986744200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 msgr2=0x55c2c1e53a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.760+0000 7f02c5d21740 1 --2- 172.21.15.16:0/986744200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e53a40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02ac008000 tx=0x7f02ac00ae50 comp rx=0 tx=0).stop 2024-09-19T09:26:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.760+0000 7f02c5d21740 1 -- 172.21.15.16:0/986744200 shutdown_connections 2024-09-19T09:26:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.760+0000 7f02c5d21740 1 --2- 172.21.15.16:0/986744200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 0x55c2c1e53a40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.760+0000 7f02c5d21740 1 --2- 172.21.15.16:0/986744200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e49ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.760+0000 7f02c5d21740 1 --2- 172.21.15.16:0/986744200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e6bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.760+0000 7f02c5d21740 1 -- 172.21.15.16:0/986744200 >> 172.21.15.16:0/986744200 conn(0x55c2c1e4af60 msgr2=0x55c2c1e497e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:49.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.760+0000 7f02c5d21740 1 -- 172.21.15.16:0/986744200 shutdown_connections 2024-09-19T09:26:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.760+0000 7f02c5d21740 1 -- 172.21.15.16:0/986744200 wait complete. 2024-09-19T09:26:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.761+0000 7f02c5d21740 1 -- start start 2024-09-19T09:26:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.761+0000 7f02c5d21740 1 Processor -- start 2024-09-19T09:26:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.761+0000 7f02c5d21740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e6f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.761+0000 7f02c5d21740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e6faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.761+0000 7f02c5d21740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 0x55c2c1ecce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.761+0000 7f02c5d21740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1e6ba30 con 0x55c2c1e59bb0 2024-09-19T09:26:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.761+0000 7f02c5d21740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1e6b360 con 0x55c2c1e4f2f0 2024-09-19T09:26:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.761+0000 7f02c5d21740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c2c1e6c100 con 0x55c2c1e4e9d0 2024-09-19T09:26:49.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.761+0000 7f02c5d21740 10 client.0 _ll_register_callbacks cb 0x7f02ac00a5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:26:49.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.761+0000 7f02c0ee3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 0x55c2c1ecce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.761+0000 7f02c0ee3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 0x55c2c1ecce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46694/0 (socket says 172.21.15.16:46694) 2024-09-19T09:26:49.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.761+0000 7f02c0ee3640 1 -- 172.21.15.16:0/1307875705 learned_addr learned my addr 172.21.15.16:0/1307875705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:49.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.761+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e6f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02c16e4640 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e6faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02c0ee3640 1 -- 172.21.15.16:0/1307875705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e4e9d0 msgr2=0x55c2c1e6f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:49.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e6f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02c0ee3640 1 -- 172.21.15.16:0/1307875705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 msgr2=0x55c2c1e6faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:26:49.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e6faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02c0ee3640 1 -- 172.21.15.16:0/1307875705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c1ecd570 con 0x55c2c1e59bb0 2024-09-19T09:26:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02c16e4640 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e6faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02c0ee3640 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 0x55c2c1ecce80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f02ac000c00 tx=0x7f02ac00adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e772fb11bdba471a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:26:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02a4ff9640 1 -- 172.21.15.16:0/1307875705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac02a070 con 0x55c2c1e59bb0 2024-09-19T09:26:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02a4ff9640 1 -- 172.21.15.16:0/1307875705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f02ac0257f0 con 0x55c2c1e59bb0 2024-09-19T09:26:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02c5d21740 1 -- 172.21.15.16:0/1307875705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02ac014bf0 con 0x55c2c1e59bb0 2024-09-19T09:26:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02a4ff9640 1 -- 172.21.15.16:0/1307875705 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f02ac026460 con 0x55c2c1e59bb0 2024-09-19T09:26:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02c5d21740 10 client.15027 fetch_fsmap learned FSMap version 27 2024-09-19T09:26:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02c5d21740 1 -- 172.21.15.16:0/1307875705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2c1ed48f0 con 0x55c2c1e59bb0 2024-09-19T09:26:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.762+0000 7f02a4ff9640 1 -- 172.21.15.16:0/1307875705 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02ac014bf0 con 0x55c2c1e59bb0 2024-09-19T09:26:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.763+0000 7f02a4ff9640 1 -- 172.21.15.16:0/1307875705 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f02ac024040 con 0x55c2c1e59bb0 2024-09-19T09:26:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.763+0000 7f02c5d21740 10 client.15027 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:26:49.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.763+0000 7f02c5d21740 10 client.15027 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:26:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.763+0000 7f02c5d21740 20 client.15027 populate_metadata read hostname 'smithi016' 2024-09-19T09:26:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.763+0000 7f02c5d21740 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55c2c1ed8fe0 0x55c2c1ed7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.763+0000 7f02c5d21740 4 client.15027 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:26:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.763+0000 7f02c5d21740 1 -- 172.21.15.16:0/1307875705 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "status"}) -- 0x55c2c1dbcb10 con 0x55c2c1ed8fe0 2024-09-19T09:26:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.763+0000 7f02c16e4640 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55c2c1ed8fe0 0x55c2c1ed7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.763+0000 7f02c16e4640 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55c2c1ed8fe0 0x55c2c1ed7cf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bcf906260e3dace server_cookie=1620464de94e94e5 in_seq=0 out_seq=0 2024-09-19T09:26:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.763+0000 7f02a4ff9640 10 client.15027 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:26:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02a4ff9640 1 -- 172.21.15.16:0/1307875705 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c2c1dbcb10 con 0x55c2c1ed8fe0 2024-09-19T09:26:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02a4ff9640 10 client.15027 handle_command_reply: tid=0 2024-09-19T09:26:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02c5d21740 1 client.15027 shutdown 2024-09-19T09:26:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02c5d21740 1 -- 172.21.15.16:0/1307875705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 msgr2=0x55c2c1ecce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02c5d21740 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 0x55c2c1ecce80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f02ac000c00 tx=0x7f02ac00adb0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02c5d21740 1 -- 172.21.15.16:0/1307875705 shutdown_connections 2024-09-19T09:26:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02c5d21740 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55c2c1ed8fe0 0x55c2c1ed7cf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02c5d21740 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2c1e59bb0 0x55c2c1ecce80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02c5d21740 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2c1e4f2f0 0x55c2c1e6faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02c5d21740 1 --2- 172.21.15.16:0/1307875705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c2c1e4e9d0 0x55c2c1e6f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02a4ff9640 0 client.15027 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:26:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02c5d21740 1 -- 172.21.15.16:0/1307875705 >> 172.21.15.16:0/1307875705 conn(0x55c2c1e4af60 msgr2=0x55c2c1e4a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02c5d21740 1 -- 172.21.15.16:0/1307875705 shutdown_connections 2024-09-19T09:26:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02c5d21740 1 -- 172.21.15.16:0/1307875705 wait complete. 2024-09-19T09:26:49.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.764+0000 7f02c5d21740 20 client.15027 trim_cache size 0 max 16384 2024-09-19T09:26:49.775 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:26:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:26:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T09:26:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24500, 2024-09-19T09:26:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:26:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:26:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:26:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 79.477818627999994, 2024-09-19T09:26:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-19T09:26:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:26:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:26:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 90.433951851000003 2024-09-19T09:26:49.778 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:26:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.765+0000 7f02c3948640 1 -- 172.21.15.16:0/3156925278 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f02a80a32d0 msgr2=0x7f02a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.765+0000 7f02c3948640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f02a80a32d0 0x7f02a80a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f02b40bcde0 tx=0x7f02b408be20 comp rx=0 tx=0).stop 2024-09-19T09:26:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.765+0000 7f02c3948640 1 -- 172.21.15.16:0/3156925278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc102ef0 msgr2=0x7f02bc197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.765+0000 7f02c3948640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc102ef0 0x7f02bc197be0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f02b4006010 tx=0x7f02b402f6f0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.765+0000 7f02c3948640 1 -- 172.21.15.16:0/3156925278 shutdown_connections 2024-09-19T09:26:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.765+0000 7f02c3948640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f02a80a32d0 0x7f02a80a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.765+0000 7f02c3948640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02bc102570 0x7f02bc1976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.765+0000 7f02c3948640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc102ef0 0x7f02bc197be0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.765+0000 7f02c3948640 1 --2- 172.21.15.16:0/3156925278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02bc100e20 0x7f02bc197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.765+0000 7f02c3948640 1 -- 172.21.15.16:0/3156925278 >> 172.21.15.16:0/3156925278 conn(0x7f02bc0fc7d0 msgr2=0x7f02bc106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:49.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.765+0000 7f02c3948640 1 -- 172.21.15.16:0/3156925278 shutdown_connections 2024-09-19T09:26:49.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:49.765+0000 7f02c3948640 1 -- 172.21.15.16:0/3156925278 wait complete. 2024-09-19T09:26:50.466 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T09:26:50.466 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T09:26:50.478 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T09:26:50.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T09:26:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:51.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:50 smithi142 ceph-mon[35032]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:26:51.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:50 smithi016 ceph-mon[30979]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:26:51.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:50 smithi097 ceph-mon[34864]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:26:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.213+0000 7fb9e8ead640 1 -- 172.21.15.16:0/3211918957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4101c00 msgr2=0x7fb9e4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.213+0000 7fb9e8ead640 1 --2- 172.21.15.16:0/3211918957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4101c00 0x7fb9e4102000 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009900 tx=0x7fb9d402a410 comp rx=0 tx=0).stop 2024-09-19T09:26:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.214+0000 7fb9e8ead640 1 -- 172.21.15.16:0/3211918957 shutdown_connections 2024-09-19T09:26:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.214+0000 7fb9e8ead640 1 --2- 172.21.15.16:0/3211918957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e41037c0 0x7fb9e410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.214+0000 7fb9e8ead640 1 --2- 172.21.15.16:0/3211918957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e4102e00 0x7fb9e4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.214+0000 7fb9e8ead640 1 --2- 172.21.15.16:0/3211918957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4101c00 0x7fb9e4102000 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.214+0000 7fb9e8ead640 1 -- 172.21.15.16:0/3211918957 >> 172.21.15.16:0/3211918957 conn(0x7fb9e40fd390 msgr2=0x7fb9e40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.214+0000 7fb9e8ead640 1 -- 172.21.15.16:0/3211918957 shutdown_connections 2024-09-19T09:26:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.214+0000 7fb9e8ead640 1 -- 172.21.15.16:0/3211918957 wait complete. 2024-09-19T09:26:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e8ead640 1 Processor -- start 2024-09-19T09:26:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e8ead640 1 -- start start 2024-09-19T09:26:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e8ead640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4101c00 0x7fb9e4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e8ead640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e4102e00 0x7fb9e4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e8ead640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e41037c0 0x7fb9e419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e8ead640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e410d960 con 0x7fb9e4101c00 2024-09-19T09:26:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e8ead640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e410d7e0 con 0x7fb9e4102e00 2024-09-19T09:26:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e8ead640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e410dae0 con 0x7fb9e41037c0 2024-09-19T09:26:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4101c00 0x7fb9e4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4101c00 0x7fb9e4198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46710/0 (socket says 172.21.15.16:46710) 2024-09-19T09:26:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e259c640 1 -- 172.21.15.16:0/2745847593 learned_addr learned my addr 172.21.15.16:0/2745847593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e259c640 1 -- 172.21.15.16:0/2745847593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e41037c0 msgr2=0x7fb9e419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e259c640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e41037c0 0x7fb9e419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e259c640 1 -- 172.21.15.16:0/2745847593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e4102e00 msgr2=0x7fb9e4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e1d9b640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e4102e00 0x7fb9e4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e259c640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e4102e00 0x7fb9e4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e259c640 1 -- 172.21.15.16:0/2745847593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e419fc80 con 0x7fb9e4101c00 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.215+0000 7fb9e1d9b640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e4102e00 0x7fb9e4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.216+0000 7fb9e259c640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4101c00 0x7fb9e4198050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4031610 tx=0x7fb9d40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.216+0000 7fb9db7fe640 1 -- 172.21.15.16:0/2745847593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d403f030 con 0x7fb9e4101c00 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.216+0000 7fb9db7fe640 1 -- 172.21.15.16:0/2745847593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d4003ce0 con 0x7fb9e4101c00 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.216+0000 7fb9e8ead640 1 -- 172.21.15.16:0/2745847593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e419ff10 con 0x7fb9e4101c00 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.216+0000 7fb9db7fe640 1 -- 172.21.15.16:0/2745847593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4035070 con 0x7fb9e4101c00 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.216+0000 7fb9e8ead640 1 -- 172.21.15.16:0/2745847593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e41a03f0 con 0x7fb9e4101c00 2024-09-19T09:26:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.217+0000 7fb9e8ead640 1 -- 172.21.15.16:0/2745847593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9e41090a0 con 0x7fb9e4101c00 2024-09-19T09:26:51.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.218+0000 7fb9db7fe640 1 -- 172.21.15.16:0/2745847593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb9d4042030 con 0x7fb9e4101c00 2024-09-19T09:26:51.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.219+0000 7fb9db7fe640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9d00a3380 0x7fb9d00a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:51.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.219+0000 7fb9e1d9b640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9d00a3380 0x7fb9d00a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:51.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.219+0000 7fb9db7fe640 1 -- 172.21.15.16:0/2745847593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb9d40f03c0 con 0x7fb9e4101c00 2024-09-19T09:26:51.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.220+0000 7fb9e1d9b640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9d00a3380 0x7fb9d00a5840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc006fd0 tx=0x7fb9cc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:51.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.223+0000 7fb9db7fe640 1 -- 172.21.15.16:0/2745847593 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb9d40bcf80 con 0x7fb9e4101c00 2024-09-19T09:26:51.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.416+0000 7fb9e8ead640 1 -- 172.21.15.16:0/2745847593 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fb9e4062630 con 0x7fb9d00a3380 2024-09-19T09:26:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.418+0000 7fb9db7fe640 1 -- 172.21.15.16:0/2745847593 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fb9e4062630 con 0x7fb9d00a3380 2024-09-19T09:26:51.419 INFO:teuthology.orchestra.run.smithi016.stdout:Backend: cephadm 2024-09-19T09:26:51.419 INFO:teuthology.orchestra.run.smithi016.stdout:Available: Yes 2024-09-19T09:26:51.419 INFO:teuthology.orchestra.run.smithi016.stdout:Paused: No 2024-09-19T09:26:51.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.420+0000 7fb9e8ead640 1 -- 172.21.15.16:0/2745847593 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9d00a3380 msgr2=0x7fb9d00a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.420+0000 7fb9e8ead640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9d00a3380 0x7fb9d00a5840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc006fd0 tx=0x7fb9cc008040 comp rx=0 tx=0).stop 2024-09-19T09:26:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.420+0000 7fb9e8ead640 1 -- 172.21.15.16:0/2745847593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4101c00 msgr2=0x7fb9e4198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.420+0000 7fb9e8ead640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4101c00 0x7fb9e4198050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4031610 tx=0x7fb9d40043b0 comp rx=0 tx=0).stop 2024-09-19T09:26:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.420+0000 7fb9e8ead640 1 -- 172.21.15.16:0/2745847593 shutdown_connections 2024-09-19T09:26:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.420+0000 7fb9e8ead640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9d00a3380 0x7fb9d00a5840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.420+0000 7fb9e8ead640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e41037c0 0x7fb9e419f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.420+0000 7fb9e8ead640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e4102e00 0x7fb9e4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.420+0000 7fb9e8ead640 1 --2- 172.21.15.16:0/2745847593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9e4101c00 0x7fb9e4198050 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.420+0000 7fb9e8ead640 1 -- 172.21.15.16:0/2745847593 >> 172.21.15.16:0/2745847593 conn(0x7fb9e40fd390 msgr2=0x7fb9e4106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.420+0000 7fb9e8ead640 1 -- 172.21.15.16:0/2745847593 shutdown_connections 2024-09-19T09:26:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:51.420+0000 7fb9e8ead640 1 -- 172.21.15.16:0/2745847593 wait complete. 2024-09-19T09:26:52.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T09:26:52.293 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.728+0000 7fb043ff3640 1 -- 172.21.15.16:0/767242848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03c100490 msgr2=0x7fb03c100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.728+0000 7fb043ff3640 1 --2- 172.21.15.16:0/767242848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03c100490 0x7fb03c100890 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb038009900 tx=0x7fb03802a410 comp rx=0 tx=0).stop 2024-09-19T09:26:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.728+0000 7fb043ff3640 1 -- 172.21.15.16:0/767242848 shutdown_connections 2024-09-19T09:26:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.728+0000 7fb043ff3640 1 --2- 172.21.15.16:0/767242848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb03c100dd0 0x7fb03c101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.728+0000 7fb043ff3640 1 --2- 172.21.15.16:0/767242848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb03c101770 0x7fb03c101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.728+0000 7fb043ff3640 1 --2- 172.21.15.16:0/767242848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03c100490 0x7fb03c100890 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.728+0000 7fb043ff3640 1 -- 172.21.15.16:0/767242848 >> 172.21.15.16:0/767242848 conn(0x7fb03c0fc2a0 msgr2=0x7fb03c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.728+0000 7fb043ff3640 1 -- 172.21.15.16:0/767242848 shutdown_connections 2024-09-19T09:26:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.728+0000 7fb043ff3640 1 -- 172.21.15.16:0/767242848 wait complete. 2024-09-19T09:26:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.729+0000 7fb043ff3640 1 Processor -- start 2024-09-19T09:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.729+0000 7fb043ff3640 1 -- start start 2024-09-19T09:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.729+0000 7fb043ff3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb03c100490 0x7fb03c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.729+0000 7fb043ff3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb03c100dd0 0x7fb03c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.729+0000 7fb043ff3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03c101770 0x7fb03c1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.729+0000 7fb043ff3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c10fa60 con 0x7fb03c101770 2024-09-19T09:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.729+0000 7fb043ff3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c10f8e0 con 0x7fb03c100490 2024-09-19T09:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.729+0000 7fb043ff3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c10fbe0 con 0x7fb03c100dd0 2024-09-19T09:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.729+0000 7fb042590640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03c101770 0x7fb03c1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.729+0000 7fb042590640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03c101770 0x7fb03c1a1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46722/0 (socket says 172.21.15.16:46722) 2024-09-19T09:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.729+0000 7fb042590640 1 -- 172.21.15.16:0/1736931343 learned_addr learned my addr 172.21.15.16:0/1736931343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.729+0000 7fb041d8f640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb03c100490 0x7fb03c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.729+0000 7fb04158e640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb03c100dd0 0x7fb03c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.730+0000 7fb042590640 1 -- 172.21.15.16:0/1736931343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb03c100dd0 msgr2=0x7fb03c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.730+0000 7fb042590640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb03c100dd0 0x7fb03c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.730+0000 7fb042590640 1 -- 172.21.15.16:0/1736931343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb03c100490 msgr2=0x7fb03c19a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.730+0000 7fb042590640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb03c100490 0x7fb03c19a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.730+0000 7fb042590640 1 -- 172.21.15.16:0/1736931343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03c1a1c80 con 0x7fb03c101770 2024-09-19T09:26:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.730+0000 7fb042590640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03c101770 0x7fb03c1a1610 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb03400cce0 tx=0x7fb03400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.730+0000 7fb032ffd640 1 -- 172.21.15.16:0/1736931343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03400f450 con 0x7fb03c101770 2024-09-19T09:26:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.730+0000 7fb032ffd640 1 -- 172.21.15.16:0/1736931343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb03400fad0 con 0x7fb03c101770 2024-09-19T09:26:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.730+0000 7fb043ff3640 1 -- 172.21.15.16:0/1736931343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03c1a1f70 con 0x7fb03c101770 2024-09-19T09:26:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.730+0000 7fb032ffd640 1 -- 172.21.15.16:0/1736931343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034018c10 con 0x7fb03c101770 2024-09-19T09:26:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.730+0000 7fb043ff3640 1 -- 172.21.15.16:0/1736931343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03c1a23d0 con 0x7fb03c101770 2024-09-19T09:26:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.731+0000 7fb043ff3640 1 -- 172.21.15.16:0/1736931343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb03c10b1e0 con 0x7fb03c101770 2024-09-19T09:26:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.732+0000 7fb032ffd640 1 -- 172.21.15.16:0/1736931343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb03400f5f0 con 0x7fb03c101770 2024-09-19T09:26:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.733+0000 7fb032ffd640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb0280a3300 0x7fb0280a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.733+0000 7fb041d8f640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb0280a3300 0x7fb0280a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.733+0000 7fb032ffd640 1 -- 172.21.15.16:0/1736931343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb0340cffd0 con 0x7fb03c101770 2024-09-19T09:26:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.733+0000 7fb041d8f640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb0280a3300 0x7fb0280a57c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb0380314e0 tx=0x7fb038000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.737+0000 7fb032ffd640 1 -- 172.21.15.16:0/1736931343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb034014030 con 0x7fb03c101770 2024-09-19T09:26:52.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.927+0000 7fb043ff3640 1 -- 172.21.15.16:0/1736931343 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb03c062630 con 0x7fb0280a3300 2024-09-19T09:26:52.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.940+0000 7fb032ffd640 1 -- 172.21.15.16:0/1736931343 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fb03c062630 con 0x7fb0280a3300 2024-09-19T09:26:52.941 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T09:26:52.941 INFO:teuthology.orchestra.run.smithi016.stdout:mds.a smithi016 running (96s) 21s ago 96s 16.9M - 19.1.1-361-g2f599a46 d2baa855c067 b920f8580650 2024-09-19T09:26:52.941 INFO:teuthology.orchestra.run.smithi016.stdout:mds.b smithi097 running (93s) 26s ago 93s 29.0M - 19.1.1-361-g2f599a46 d2baa855c067 582e7ba753cc 2024-09-19T09:26:52.941 INFO:teuthology.orchestra.run.smithi016.stdout:mds.c smithi142 running (107s) 26s ago 106s 17.8M - 19.1.1-361-g2f599a46 d2baa855c067 e18c3349f4b6 2024-09-19T09:26:52.942 INFO:teuthology.orchestra.run.smithi016.stdout:mds.d smithi016 running (114s) 21s ago 114s 13.4M - 19.1.1-361-g2f599a46 d2baa855c067 acd1cdb9e6fc 2024-09-19T09:26:52.942 INFO:teuthology.orchestra.run.smithi016.stdout:mds.e smithi097 running (112s) 26s ago 111s 18.6M - 19.1.1-361-g2f599a46 d2baa855c067 d0bae33a229b 2024-09-19T09:26:52.942 INFO:teuthology.orchestra.run.smithi016.stdout:mds.f smithi142 running (109s) 26s ago 109s 19.3M - 19.1.1-361-g2f599a46 d2baa855c067 0317d9ea27f7 2024-09-19T09:26:52.942 INFO:teuthology.orchestra.run.smithi016.stdout:mds.g smithi016 running (99s) 21s ago 99s 17.0M - 19.1.1-361-g2f599a46 d2baa855c067 2f96fd20c7fc 2024-09-19T09:26:52.942 INFO:teuthology.orchestra.run.smithi016.stdout:mds.h smithi097 running (104s) 26s ago 104s 20.5M - 19.1.1-361-g2f599a46 d2baa855c067 08718aa33904 2024-09-19T09:26:52.942 INFO:teuthology.orchestra.run.smithi016.stdout:mds.i smithi142 running (102s) 26s ago 101s 17.9M - 19.1.1-361-g2f599a46 d2baa855c067 bcc57a0f2b71 2024-09-19T09:26:52.942 INFO:teuthology.orchestra.run.smithi016.stdout:mds.j smithi016 running (118s) 21s ago 117s 13.7M - 19.1.1-361-g2f599a46 d2baa855c067 f9a7443f2ede 2024-09-19T09:26:52.942 INFO:teuthology.orchestra.run.smithi016.stdout:mds.k smithi097 running (2m) 26s ago 2m 18.4M - 19.1.1-361-g2f599a46 d2baa855c067 baea3117dccf 2024-09-19T09:26:52.942 INFO:teuthology.orchestra.run.smithi016.stdout:mds.l smithi142 running (2m) 26s ago 2m 18.1M - 19.1.1-361-g2f599a46 d2baa855c067 bda487c24401 2024-09-19T09:26:52.942 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.x smithi016 *:9283,8765 running (8m) 21s ago 8m 509M - 19.1.1-361-g2f599a46 d2baa855c067 ef968a832801 2024-09-19T09:26:52.942 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.y smithi097 *:8443,8765 running (7m) 26s ago 7m 441M - 19.1.1-361-g2f599a46 d2baa855c067 ec045325526f 2024-09-19T09:26:52.942 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.z smithi142 *:8443,8765 running (7m) 26s ago 7m 443M - 19.1.1-361-g2f599a46 d2baa855c067 65283b2503fb 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:mon.a smithi016 running (8m) 21s ago 9m 49.6M 2048M 19.1.1-361-g2f599a46 d2baa855c067 844da19cfdd1 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:mon.b smithi097 running (7m) 26s ago 7m 43.2M 2048M 19.1.1-361-g2f599a46 d2baa855c067 09082255ca7d 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:mon.c smithi142 running (7m) 26s ago 7m 40.9M 2048M 19.1.1-361-g2f599a46 d2baa855c067 e7d1338d9de7 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (6m) 21s ago 6m 55.6M 4273M 19.1.1-361-g2f599a46 d2baa855c067 4222acc51115 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi097 running (6m) 26s ago 6m 56.1M 4272M 19.1.1-361-g2f599a46 d2baa855c067 d0dd91099bbf 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi142 running (6m) 26s ago 6m 50.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 88b700d7e60e 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi016 running (5m) 21s ago 5m 56.4M 4273M 19.1.1-361-g2f599a46 d2baa855c067 2936d667e921 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi097 running (5m) 26s ago 5m 57.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 55aa73db119e 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi142 running (4m) 26s ago 4m 47.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 0dbab7611710 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 smithi016 running (4m) 21s ago 4m 51.9M 4273M 19.1.1-361-g2f599a46 d2baa855c067 39740579cdc9 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 smithi097 running (4m) 26s ago 4m 46.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 76b5d73c6e82 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 smithi142 running (3m) 26s ago 3m 55.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 a116edc2e79b 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 smithi016 running (3m) 21s ago 3m 53.5M 4273M 19.1.1-361-g2f599a46 d2baa855c067 4429da0321e4 2024-09-19T09:26:52.943 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 smithi097 running (2m) 26s ago 2m 45.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 bc16396fdd2b 2024-09-19T09:26:52.944 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 smithi142 running (2m) 26s ago 2m 44.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 04b1da0a587e 2024-09-19T09:26:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.942+0000 7fb043ff3640 1 -- 172.21.15.16:0/1736931343 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb0280a3300 msgr2=0x7fb0280a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.942+0000 7fb043ff3640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb0280a3300 0x7fb0280a57c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb0380314e0 tx=0x7fb038000bc0 comp rx=0 tx=0).stop 2024-09-19T09:26:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.942+0000 7fb043ff3640 1 -- 172.21.15.16:0/1736931343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03c101770 msgr2=0x7fb03c1a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.942+0000 7fb043ff3640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03c101770 0x7fb03c1a1610 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb03400cce0 tx=0x7fb03400a620 comp rx=0 tx=0).stop 2024-09-19T09:26:52.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.942+0000 7fb043ff3640 1 -- 172.21.15.16:0/1736931343 shutdown_connections 2024-09-19T09:26:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.942+0000 7fb043ff3640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb0280a3300 0x7fb0280a57c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.942+0000 7fb043ff3640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb03c100dd0 0x7fb03c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.942+0000 7fb043ff3640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03c101770 0x7fb03c1a1610 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.942+0000 7fb043ff3640 1 --2- 172.21.15.16:0/1736931343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb03c100490 0x7fb03c19a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.942+0000 7fb043ff3640 1 -- 172.21.15.16:0/1736931343 >> 172.21.15.16:0/1736931343 conn(0x7fb03c0fc2a0 msgr2=0x7fb03c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.942+0000 7fb043ff3640 1 -- 172.21.15.16:0/1736931343 shutdown_connections 2024-09-19T09:26:52.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:52.942+0000 7fb043ff3640 1 -- 172.21.15.16:0/1736931343 wait complete. 2024-09-19T09:26:53.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:52 smithi016 ceph-mon[30979]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:53.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:52 smithi016 ceph-mon[30979]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:53.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:52 smithi142 ceph-mon[35032]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:53.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:52 smithi142 ceph-mon[35032]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:53.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:52 smithi097 ceph-mon[34864]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:53.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:52 smithi097 ceph-mon[34864]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:53.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T09:26:53.911 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:54.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:53 smithi016 ceph-mon[30979]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:54.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:53 smithi142 ceph-mon[35032]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:54.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:53 smithi097 ceph-mon[34864]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.387+0000 7fe2f5460640 1 -- 172.21.15.16:0/3273616360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f0102e10 msgr2=0x7fe2f0103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.387+0000 7fe2f5460640 1 --2- 172.21.15.16:0/3273616360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f0102e10 0x7fe2f0103290 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009900 tx=0x7fe2e402a410 comp rx=0 tx=0).stop 2024-09-19T09:26:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.387+0000 7fe2f5460640 1 -- 172.21.15.16:0/3273616360 shutdown_connections 2024-09-19T09:26:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.387+0000 7fe2f5460640 1 --2- 172.21.15.16:0/3273616360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f01037d0 0x7fe2f010a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.387+0000 7fe2f5460640 1 --2- 172.21.15.16:0/3273616360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f0102e10 0x7fe2f0103290 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.387+0000 7fe2f5460640 1 --2- 172.21.15.16:0/3273616360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f0101c10 0x7fe2f0102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.387+0000 7fe2f5460640 1 -- 172.21.15.16:0/3273616360 >> 172.21.15.16:0/3273616360 conn(0x7fe2f00fd3c0 msgr2=0x7fe2f00ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.387+0000 7fe2f5460640 1 -- 172.21.15.16:0/3273616360 shutdown_connections 2024-09-19T09:26:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.387+0000 7fe2f5460640 1 -- 172.21.15.16:0/3273616360 wait complete. 2024-09-19T09:26:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2f5460640 1 Processor -- start 2024-09-19T09:26:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2f5460640 1 -- start start 2024-09-19T09:26:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2f5460640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f0101c10 0x7fe2f0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2f5460640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f0102e10 0x7fe2f0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2f5460640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f01037d0 0x7fe2f019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2f5460640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f010d890 con 0x7fe2f0101c10 2024-09-19T09:26:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2f5460640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f010d710 con 0x7fe2f0102e10 2024-09-19T09:26:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2f5460640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f010da10 con 0x7fe2f01037d0 2024-09-19T09:26:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f0101c10 0x7fe2f0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f0101c10 0x7fe2f0197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46732/0 (socket says 172.21.15.16:46732) 2024-09-19T09:26:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2eeffd640 1 -- 172.21.15.16:0/2901350758 learned_addr learned my addr 172.21.15.16:0/2901350758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2ee7fc640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f0102e10 0x7fe2f0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2ef7fe640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f01037d0 0x7fe2f019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2eeffd640 1 -- 172.21.15.16:0/2901350758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f01037d0 msgr2=0x7fe2f019f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2eeffd640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f01037d0 0x7fe2f019f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2eeffd640 1 -- 172.21.15.16:0/2901350758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f0102e10 msgr2=0x7fe2f0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2eeffd640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f0102e10 0x7fe2f0198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2eeffd640 1 -- 172.21.15.16:0/2901350758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f019fb00 con 0x7fe2f0101c10 2024-09-19T09:26:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2ef7fe640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f01037d0 0x7fe2f019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.388+0000 7fe2eeffd640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f0101c10 0x7fe2f0197ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000d950 tx=0x7fe2e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.389+0000 7fe2d7fff640 1 -- 172.21.15.16:0/2901350758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0017020 con 0x7fe2f0101c10 2024-09-19T09:26:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.389+0000 7fe2f5460640 1 -- 172.21.15.16:0/2901350758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f019fdf0 con 0x7fe2f0101c10 2024-09-19T09:26:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.389+0000 7fe2d7fff640 1 -- 172.21.15.16:0/2901350758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e0009c30 con 0x7fe2f0101c10 2024-09-19T09:26:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.389+0000 7fe2d7fff640 1 -- 172.21.15.16:0/2901350758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e000f740 con 0x7fe2f0101c10 2024-09-19T09:26:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.389+0000 7fe2f5460640 1 -- 172.21.15.16:0/2901350758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f01a0250 con 0x7fe2f0101c10 2024-09-19T09:26:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.390+0000 7fe2f5460640 1 -- 172.21.15.16:0/2901350758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f0108fb0 con 0x7fe2f0101c10 2024-09-19T09:26:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.391+0000 7fe2d7fff640 1 -- 172.21.15.16:0/2901350758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe2e001a020 con 0x7fe2f0101c10 2024-09-19T09:26:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.392+0000 7fe2d7fff640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe2d80a3300 0x7fe2d80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.392+0000 7fe2ee7fc640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe2d80a3300 0x7fe2d80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.392+0000 7fe2d7fff640 1 -- 172.21.15.16:0/2901350758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe2e00cf7d0 con 0x7fe2f0101c10 2024-09-19T09:26:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.392+0000 7fe2ee7fc640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe2d80a3300 0x7fe2d80a57c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40314e0 tx=0x7fe2e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.395+0000 7fe2d7fff640 1 -- 172.21.15.16:0/2901350758 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe2e0021070 con 0x7fe2f0101c10 2024-09-19T09:26:54.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.577+0000 7fe2f5460640 1 -- 172.21.15.16:0/2901350758 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fe2f0062630 con 0x7fe2d80a3300 2024-09-19T09:26:54.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.583+0000 7fe2d7fff640 1 -- 172.21.15.16:0/2901350758 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fe2f0062630 con 0x7fe2d80a3300 2024-09-19T09:26:54.585 INFO:teuthology.orchestra.run.smithi016.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T09:26:54.585 INFO:teuthology.orchestra.run.smithi016.stdout:mds.all 12/12 28s ago 2m smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-09-19T09:26:54.585 INFO:teuthology.orchestra.run.smithi016.stdout:mgr 3/3 28s ago 7m smithi016=x;smithi097=y;smithi142=z;count:3 2024-09-19T09:26:54.585 INFO:teuthology.orchestra.run.smithi016.stdout:mon 3/3 28s ago 7m smithi016:172.21.15.16=a;smithi097:172.21.15.97=b;smithi142:172.21.15.142=c;count:3 2024-09-19T09:26:54.585 INFO:teuthology.orchestra.run.smithi016.stdout:osd 12 28s ago - 2024-09-19T09:26:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.585+0000 7fe2f5460640 1 -- 172.21.15.16:0/2901350758 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe2d80a3300 msgr2=0x7fe2d80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.585+0000 7fe2f5460640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe2d80a3300 0x7fe2d80a57c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40314e0 tx=0x7fe2e4000bc0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.585+0000 7fe2f5460640 1 -- 172.21.15.16:0/2901350758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f0101c10 msgr2=0x7fe2f0197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.585+0000 7fe2f5460640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f0101c10 0x7fe2f0197ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000d950 tx=0x7fe2e000de10 comp rx=0 tx=0).stop 2024-09-19T09:26:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.585+0000 7fe2f5460640 1 -- 172.21.15.16:0/2901350758 shutdown_connections 2024-09-19T09:26:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.585+0000 7fe2f5460640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe2d80a3300 0x7fe2d80a57c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.585+0000 7fe2f5460640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f01037d0 0x7fe2f019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.585+0000 7fe2f5460640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f0102e10 0x7fe2f0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.585+0000 7fe2f5460640 1 --2- 172.21.15.16:0/2901350758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f0101c10 0x7fe2f0197ed0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.585+0000 7fe2f5460640 1 -- 172.21.15.16:0/2901350758 >> 172.21.15.16:0/2901350758 conn(0x7fe2f00fd3c0 msgr2=0x7fe2f0106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:54.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.585+0000 7fe2f5460640 1 -- 172.21.15.16:0/2901350758 shutdown_connections 2024-09-19T09:26:54.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:54.585+0000 7fe2f5460640 1 -- 172.21.15.16:0/2901350758 wait complete. 2024-09-19T09:26:54.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:54 smithi016 ceph-mon[30979]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:26:55.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:54 smithi142 ceph-mon[35032]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:26:55.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:54 smithi097 ceph-mon[34864]: pgmap v317: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:26:55.255 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T09:26:55.516 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:55.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:55 smithi016 ceph-mon[30979]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.953+0000 7f1da344f640 1 -- 172.21.15.16:0/2276840287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c102e00 msgr2=0x7f1d9c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.953+0000 7f1da344f640 1 --2- 172.21.15.16:0/2276840287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c102e00 0x7f1d9c103280 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009860 tx=0x7f1d9002a340 comp rx=0 tx=0).stop 2024-09-19T09:26:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.954+0000 7f1da344f640 1 -- 172.21.15.16:0/2276840287 shutdown_connections 2024-09-19T09:26:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.954+0000 7f1da344f640 1 --2- 172.21.15.16:0/2276840287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d9c1037c0 0x7f1d9c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.954+0000 7f1da344f640 1 --2- 172.21.15.16:0/2276840287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c102e00 0x7f1d9c103280 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.954+0000 7f1da344f640 1 --2- 172.21.15.16:0/2276840287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9c101c00 0x7f1d9c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.954+0000 7f1da344f640 1 -- 172.21.15.16:0/2276840287 >> 172.21.15.16:0/2276840287 conn(0x7f1d9c0fd390 msgr2=0x7f1d9c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.954+0000 7f1da344f640 1 -- 172.21.15.16:0/2276840287 shutdown_connections 2024-09-19T09:26:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.954+0000 7f1da344f640 1 -- 172.21.15.16:0/2276840287 wait complete. 2024-09-19T09:26:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.954+0000 7f1da344f640 1 Processor -- start 2024-09-19T09:26:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.954+0000 7f1da344f640 1 -- start start 2024-09-19T09:26:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da344f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9c101c00 0x7f1d9c19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da344f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d9c102e00 0x7f1d9c19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da344f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c1037c0 0x7f1d9c1a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da344f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c10d860 con 0x7f1d9c1037c0 2024-09-19T09:26:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da344f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c10d6e0 con 0x7f1d9c102e00 2024-09-19T09:26:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da344f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c10d9e0 con 0x7f1d9c101c00 2024-09-19T09:26:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da19ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c1037c0 0x7f1d9c1a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da09ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d9c102e00 0x7f1d9c19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da19ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c1037c0 0x7f1d9c1a5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50966/0 (socket says 172.21.15.16:50966) 2024-09-19T09:26:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da19ec640 1 -- 172.21.15.16:0/2815752837 learned_addr learned my addr 172.21.15.16:0/2815752837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da11eb640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9c101c00 0x7f1d9c19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da09ea640 1 -- 172.21.15.16:0/2815752837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9c101c00 msgr2=0x7f1d9c19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da09ea640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9c101c00 0x7f1d9c19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da09ea640 1 -- 172.21.15.16:0/2815752837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c1037c0 msgr2=0x7f1d9c1a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da09ea640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c1037c0 0x7f1d9c1a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da09ea640 1 -- 172.21.15.16:0/2815752837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c1a6180 con 0x7f1d9c102e00 2024-09-19T09:26:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da19ec640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c1037c0 0x7f1d9c1a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:26:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da11eb640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9c101c00 0x7f1d9c19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.955+0000 7f1da09ea640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d9c102e00 0x7f1d9c19ea90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d90035040 tx=0x7f1d90033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.956+0000 7f1d9a7fc640 1 -- 172.21.15.16:0/2815752837 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90037040 con 0x7f1d9c102e00 2024-09-19T09:26:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.956+0000 7f1da344f640 1 -- 172.21.15.16:0/2815752837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9c1a6410 con 0x7f1d9c102e00 2024-09-19T09:26:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.956+0000 7f1d9a7fc640 1 -- 172.21.15.16:0/2815752837 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d90004040 con 0x7f1d9c102e00 2024-09-19T09:26:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.956+0000 7f1d9a7fc640 1 -- 172.21.15.16:0/2815752837 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d900046c0 con 0x7f1d9c102e00 2024-09-19T09:26:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.956+0000 7f1da344f640 1 -- 172.21.15.16:0/2815752837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9c1a6940 con 0x7f1d9c102e00 2024-09-19T09:26:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.957+0000 7f1da344f640 1 -- 172.21.15.16:0/2815752837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9c106fc0 con 0x7f1d9c102e00 2024-09-19T09:26:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.958+0000 7f1d9a7fc640 1 -- 172.21.15.16:0/2815752837 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1d90042020 con 0x7f1d9c102e00 2024-09-19T09:26:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.958+0000 7f1d9a7fc640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d840a3300 0x7f1d840a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.958+0000 7f1da11eb640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d840a3300 0x7f1d840a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.958+0000 7f1d9a7fc640 1 -- 172.21.15.16:0/2815752837 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f1d900ef250 con 0x7f1d9c102e00 2024-09-19T09:26:56.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.959+0000 7f1da11eb640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d840a3300 0x7f1d840a57c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c102c60 tx=0x7f1d94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:56.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:55.963+0000 7f1d9a7fc640 1 -- 172.21.15.16:0/2815752837 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1d90049020 con 0x7f1d9c102e00 2024-09-19T09:26:56.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:55 smithi142 ceph-mon[35032]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:56.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:55 smithi097 ceph-mon[34864]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:56.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.148+0000 7f1da344f640 1 -- 172.21.15.16:0/2815752837 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f1d9c062630 con 0x7f1d840a3300 2024-09-19T09:26:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.149+0000 7f1d9a7fc640 1 -- 172.21.15.16:0/2815752837 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f1d9c062630 con 0x7f1d840a3300 2024-09-19T09:26:56.150 INFO:teuthology.orchestra.run.smithi016.stdout:HOST ADDR LABELS STATUS 2024-09-19T09:26:56.150 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 172.21.15.16 2024-09-19T09:26:56.151 INFO:teuthology.orchestra.run.smithi016.stdout:smithi097 172.21.15.97 2024-09-19T09:26:56.151 INFO:teuthology.orchestra.run.smithi016.stdout:smithi142 172.21.15.142 2024-09-19T09:26:56.151 INFO:teuthology.orchestra.run.smithi016.stdout:3 hosts in cluster 2024-09-19T09:26:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.151+0000 7f1da344f640 1 -- 172.21.15.16:0/2815752837 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d840a3300 msgr2=0x7f1d840a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.151+0000 7f1da344f640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d840a3300 0x7f1d840a57c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c102c60 tx=0x7f1d94009260 comp rx=0 tx=0).stop 2024-09-19T09:26:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.151+0000 7f1da344f640 1 -- 172.21.15.16:0/2815752837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d9c102e00 msgr2=0x7f1d9c19ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.151+0000 7f1da344f640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d9c102e00 0x7f1d9c19ea90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d90035040 tx=0x7f1d90033d90 comp rx=0 tx=0).stop 2024-09-19T09:26:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.152+0000 7f1da344f640 1 -- 172.21.15.16:0/2815752837 shutdown_connections 2024-09-19T09:26:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.152+0000 7f1da344f640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c1037c0 0x7f1d9c1a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.152+0000 7f1da344f640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d9c102e00 0x7f1d9c19ea90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.152+0000 7f1da344f640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d840a3300 0x7f1d840a57c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.152+0000 7f1da344f640 1 --2- 172.21.15.16:0/2815752837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9c101c00 0x7f1d9c19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.152+0000 7f1da344f640 1 -- 172.21.15.16:0/2815752837 >> 172.21.15.16:0/2815752837 conn(0x7f1d9c0fd390 msgr2=0x7f1d9c104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:56.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.152+0000 7f1da344f640 1 -- 172.21.15.16:0/2815752837 shutdown_connections 2024-09-19T09:26:56.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:56.152+0000 7f1da344f640 1 -- 172.21.15.16:0/2815752837 wait complete. 2024-09-19T09:26:56.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T09:26:57.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:56 smithi016 ceph-mon[30979]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:57.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:56 smithi016 ceph-mon[30979]: from='client.24776 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:57.110 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:57.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:56 smithi142 ceph-mon[35032]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:57.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:56 smithi142 ceph-mon[35032]: from='client.24776 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:57.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:56 smithi097 ceph-mon[34864]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:26:57.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:56 smithi097 ceph-mon[34864]: from='client.24776 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.579+0000 7fa33c7c8640 1 -- 172.21.15.16:0/1365996273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa334101d10 msgr2=0x7fa334102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.579+0000 7fa33c7c8640 1 --2- 172.21.15.16:0/1365996273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa334101d10 0x7fa334102190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa324009900 tx=0x7fa32402a410 comp rx=0 tx=0).stop 2024-09-19T09:26:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.579+0000 7fa33c7c8640 1 -- 172.21.15.16:0/1365996273 shutdown_connections 2024-09-19T09:26:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.579+0000 7fa33c7c8640 1 --2- 172.21.15.16:0/1365996273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3341026d0 0x7fa334108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.579+0000 7fa33c7c8640 1 --2- 172.21.15.16:0/1365996273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa334101d10 0x7fa334102190 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.579+0000 7fa33c7c8640 1 --2- 172.21.15.16:0/1365996273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa334100b10 0x7fa334100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.579+0000 7fa33c7c8640 1 -- 172.21.15.16:0/1365996273 >> 172.21.15.16:0/1365996273 conn(0x7fa3340fc2a0 msgr2=0x7fa3340fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.579+0000 7fa33c7c8640 1 -- 172.21.15.16:0/1365996273 shutdown_connections 2024-09-19T09:26:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.579+0000 7fa33c7c8640 1 -- 172.21.15.16:0/1365996273 wait complete. 2024-09-19T09:26:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.580+0000 7fa33c7c8640 1 Processor -- start 2024-09-19T09:26:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.580+0000 7fa33c7c8640 1 -- start start 2024-09-19T09:26:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.580+0000 7fa33c7c8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa334100b10 0x7fa334197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.580+0000 7fa33c7c8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa334101d10 0x7fa3341984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.580+0000 7fa33c7c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3341026d0 0x7fa33419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.580+0000 7fa33c7c8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa33410c790 con 0x7fa3341026d0 2024-09-19T09:26:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.580+0000 7fa33c7c8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa33410c610 con 0x7fa334100b10 2024-09-19T09:26:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.580+0000 7fa33c7c8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa33410c910 con 0x7fa334101d10 2024-09-19T09:26:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.580+0000 7fa33ad65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3341026d0 0x7fa33419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.580+0000 7fa33ad65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3341026d0 0x7fa33419f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50986/0 (socket says 172.21.15.16:50986) 2024-09-19T09:26:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.580+0000 7fa33ad65640 1 -- 172.21.15.16:0/476256574 learned_addr learned my addr 172.21.15.16:0/476256574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.580+0000 7fa33a564640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa334100b10 0x7fa334197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.580+0000 7fa339d63640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa334101d10 0x7fa3341984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.581+0000 7fa33ad65640 1 -- 172.21.15.16:0/476256574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa334101d10 msgr2=0x7fa3341984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.581+0000 7fa33ad65640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa334101d10 0x7fa3341984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.581+0000 7fa33ad65640 1 -- 172.21.15.16:0/476256574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa334100b10 msgr2=0x7fa334197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.581+0000 7fa33ad65640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa334100b10 0x7fa334197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.581+0000 7fa33ad65640 1 -- 172.21.15.16:0/476256574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33419fb90 con 0x7fa3341026d0 2024-09-19T09:26:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.581+0000 7fa33ad65640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3341026d0 0x7fa33419f520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00eaa0 tx=0x7fa32c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.581+0000 7fa32b7fe640 1 -- 172.21.15.16:0/476256574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c00cdb0 con 0x7fa3341026d0 2024-09-19T09:26:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.581+0000 7fa33c7c8640 1 -- 172.21.15.16:0/476256574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33419fe80 con 0x7fa3341026d0 2024-09-19T09:26:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.581+0000 7fa32b7fe640 1 -- 172.21.15.16:0/476256574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa32c005020 con 0x7fa3341026d0 2024-09-19T09:26:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.581+0000 7fa32b7fe640 1 -- 172.21.15.16:0/476256574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0116b0 con 0x7fa3341026d0 2024-09-19T09:26:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.581+0000 7fa33c7c8640 1 -- 172.21.15.16:0/476256574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3341a02e0 con 0x7fa3341026d0 2024-09-19T09:26:57.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.582+0000 7fa33c7c8640 1 -- 172.21.15.16:0/476256574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3341a0570 con 0x7fa3341026d0 2024-09-19T09:26:57.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.583+0000 7fa32b7fe640 1 -- 172.21.15.16:0/476256574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa32c005560 con 0x7fa3341026d0 2024-09-19T09:26:57.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.584+0000 7fa32b7fe640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa3200a3300 0x7fa3200a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:57.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.584+0000 7fa33a564640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa3200a3300 0x7fa3200a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:57.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.584+0000 7fa32b7fe640 1 -- 172.21.15.16:0/476256574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa32c0cfca0 con 0x7fa3341026d0 2024-09-19T09:26:57.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.584+0000 7fa33a564640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa3200a3300 0x7fa3200a57c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa330005fd0 tx=0x7fa33000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:57.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.588+0000 7fa32b7fe640 1 -- 172.21.15.16:0/476256574 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa32c016030 con 0x7fa3341026d0 2024-09-19T09:26:57.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.771+0000 7fa33c7c8640 1 -- 172.21.15.16:0/476256574 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa3341a0740 con 0x7fa3200a3300 2024-09-19T09:26:57.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.773+0000 7fa32b7fe640 1 -- 172.21.15.16:0/476256574 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa3341a0740 con 0x7fa3200a3300 2024-09-19T09:26:57.774 INFO:teuthology.orchestra.run.smithi016.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T09:26:57.774 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-19T09:26:57.774 INFO:teuthology.orchestra.run.smithi016.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T09:26:57.774 INFO:teuthology.orchestra.run.smithi016.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T09:26:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.775+0000 7fa33c7c8640 1 -- 172.21.15.16:0/476256574 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa3200a3300 msgr2=0x7fa3200a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.775+0000 7fa33c7c8640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa3200a3300 0x7fa3200a57c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa330005fd0 tx=0x7fa33000b390 comp rx=0 tx=0).stop 2024-09-19T09:26:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.775+0000 7fa33c7c8640 1 -- 172.21.15.16:0/476256574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3341026d0 msgr2=0x7fa33419f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.775+0000 7fa33c7c8640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3341026d0 0x7fa33419f520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00eaa0 tx=0x7fa32c00ef60 comp rx=0 tx=0).stop 2024-09-19T09:26:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.775+0000 7fa33c7c8640 1 -- 172.21.15.16:0/476256574 shutdown_connections 2024-09-19T09:26:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.775+0000 7fa33c7c8640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa3200a3300 0x7fa3200a57c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.775+0000 7fa33c7c8640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa334101d10 0x7fa3341984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.775+0000 7fa33c7c8640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3341026d0 0x7fa33419f520 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.775+0000 7fa33c7c8640 1 --2- 172.21.15.16:0/476256574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa334100b10 0x7fa334197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.775+0000 7fa33c7c8640 1 -- 172.21.15.16:0/476256574 >> 172.21.15.16:0/476256574 conn(0x7fa3340fc2a0 msgr2=0x7fa33406a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.775+0000 7fa33c7c8640 1 -- 172.21.15.16:0/476256574 shutdown_connections 2024-09-19T09:26:57.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:57.775+0000 7fa33c7c8640 1 -- 172.21.15.16:0/476256574 wait complete. 2024-09-19T09:26:58.485 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T09:26:58.495 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T09:26:58.495 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T09:26:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:26:59.089 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:58 smithi016 ceph-mon[30979]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:26:59.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:58 smithi016 ceph-mon[30979]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:58 smithi142 ceph-mon[35032]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:26:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:58 smithi142 ceph-mon[35032]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:59.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:58 smithi097 ceph-mon[34864]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:26:59.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:58 smithi097 ceph-mon[34864]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T09:26:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.171+0000 7fd88f38a640 1 -- 172.21.15.16:0/1718145660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd888101d10 msgr2=0x7fd888102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.171+0000 7fd88f38a640 1 --2- 172.21.15.16:0/1718145660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd888101d10 0x7fd888102190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009900 tx=0x7fd87c02a410 comp rx=0 tx=0).stop 2024-09-19T09:26:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.171+0000 7fd88f38a640 1 -- 172.21.15.16:0/1718145660 shutdown_connections 2024-09-19T09:26:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.171+0000 7fd88f38a640 1 --2- 172.21.15.16:0/1718145660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8881026d0 0x7fd888108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.171+0000 7fd88f38a640 1 --2- 172.21.15.16:0/1718145660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd888101d10 0x7fd888102190 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.171+0000 7fd88f38a640 1 --2- 172.21.15.16:0/1718145660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd888100b10 0x7fd888100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.171+0000 7fd88f38a640 1 -- 172.21.15.16:0/1718145660 >> 172.21.15.16:0/1718145660 conn(0x7fd8880fc2a0 msgr2=0x7fd8880fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.171+0000 7fd88f38a640 1 -- 172.21.15.16:0/1718145660 shutdown_connections 2024-09-19T09:26:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.171+0000 7fd88f38a640 1 -- 172.21.15.16:0/1718145660 wait complete. 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.171+0000 7fd88f38a640 1 Processor -- start 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88f38a640 1 -- start start 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88f38a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd888100b10 0x7fd888197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88f38a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd888101d10 0x7fd888198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88f38a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8881026d0 0x7fd88819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88f38a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd88810d920 con 0x7fd888100b10 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88f38a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd88810d7a0 con 0x7fd888101d10 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88f38a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd88810daa0 con 0x7fd8881026d0 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88d126640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd888100b10 0x7fd888197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88d126640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd888100b10 0x7fd888197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51004/0 (socket says 172.21.15.16:51004) 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88d126640 1 -- 172.21.15.16:0/625276058 learned_addr learned my addr 172.21.15.16:0/625276058 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88c925640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd888101d10 0x7fd888198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88d927640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8881026d0 0x7fd88819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88d126640 1 -- 172.21.15.16:0/625276058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8881026d0 msgr2=0x7fd88819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88d126640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8881026d0 0x7fd88819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88d126640 1 -- 172.21.15.16:0/625276058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd888101d10 msgr2=0x7fd888198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88d126640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd888101d10 0x7fd888198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88d126640 1 -- 172.21.15.16:0/625276058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88819fb40 con 0x7fd888100b10 2024-09-19T09:26:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88d927640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8881026d0 0x7fd88819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:26:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd88d126640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd888100b10 0x7fd888197f10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd88400d770 tx=0x7fd88400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.172+0000 7fd87a7fc640 1 -- 172.21.15.16:0/625276058 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884017070 con 0x7fd888100b10 2024-09-19T09:26:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.173+0000 7fd88f38a640 1 -- 172.21.15.16:0/625276058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88819fe30 con 0x7fd888100b10 2024-09-19T09:26:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.173+0000 7fd87a7fc640 1 -- 172.21.15.16:0/625276058 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd884009c30 con 0x7fd888100b10 2024-09-19T09:26:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.173+0000 7fd87a7fc640 1 -- 172.21.15.16:0/625276058 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884003dd0 con 0x7fd888100b10 2024-09-19T09:26:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.173+0000 7fd88f38a640 1 -- 172.21.15.16:0/625276058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8881a0280 con 0x7fd888100b10 2024-09-19T09:26:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.174+0000 7fd88f38a640 1 -- 172.21.15.16:0/625276058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd888107f00 con 0x7fd888100b10 2024-09-19T09:26:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.175+0000 7fd87a7fc640 1 -- 172.21.15.16:0/625276058 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd884003f70 con 0x7fd888100b10 2024-09-19T09:26:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.175+0000 7fd87a7fc640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd8740a3300 0x7fd8740a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:26:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.176+0000 7fd88c925640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd8740a3300 0x7fd8740a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:26:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.176+0000 7fd87a7fc640 1 -- 172.21.15.16:0/625276058 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd8840cf410 con 0x7fd888100b10 2024-09-19T09:26:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.176+0000 7fd88c925640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd8740a3300 0x7fd8740a57c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd87c031440 tx=0x7fd87c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:26:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.180+0000 7fd87a7fc640 1 -- 172.21.15.16:0/625276058 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd88409bfd0 con 0x7fd888100b10 2024-09-19T09:26:59.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.393+0000 7fd88f38a640 1 -- 172.21.15.16:0/625276058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fd888062ef0 con 0x7fd888100b10 2024-09-19T09:26:59.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.394+0000 7fd87a7fc640 1 -- 172.21.15.16:0/625276058 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3124 (secure 0 0 0) 0x7fd884009de0 con 0x7fd888100b10 2024-09-19T09:26:59.395 INFO:teuthology.orchestra.run.smithi016.stdout:e27 2024-09-19T09:26:59.396 INFO:teuthology.orchestra.run.smithi016.stdout:btime 2024-09-19T09:25:40:972080+0000 2024-09-19T09:26:59.396 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T09:26:59.396 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T09:26:59.396 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-09-19T09:26:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:26:59.396 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-09-19T09:26:59.396 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-09-19T09:26:59.396 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 27 2024-09-19T09:26:59.396 INFO:teuthology.orchestra.run.smithi016.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T09:26:59.396 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-19T09:25:29.829112+0000 2024-09-19T09:26:59.396 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-19T09:25:40.972058+0000 2024-09-19T09:26:59.396 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 300 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 0 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 5 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:in 0,1,2,3,4 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24500,1=24484,2=24470,3=24428,4=24424} 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-09-19T09:26:59.397 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-09-19T09:26:59.398 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-09-19T09:26:59.398 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-09-19T09:26:59.398 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-09-19T09:26:59.398 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data disabled 2024-09-19T09:26:59.398 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-09-19T09:26:59.398 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-09-19T09:26:59.398 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 0 2024-09-19T09:26:59.398 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 24500 members: 24500,24484,24470,24428,24424 2024-09-19T09:26:59.398 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b{0:24500} state up:active seq 4 addr [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:26:59.398 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c{0:24460} state up:standby-replay seq 1 addr [v2:172.21.15.142:6836/81091251,v1:172.21.15.142:6837/81091251] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:26:59.398 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.i{1:24484} state up:active seq 8 addr [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:26:59.398 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.e{1:24455} state up:standby-replay seq 1 addr [v2:172.21.15.97:6834/4135704970,v1:172.21.15.97:6835/4135704970] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:26:59.399 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.h{2:24470} state up:active seq 9 addr [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:26:59.399 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.f{2:24454} state up:standby-replay seq 1 addr [v2:172.21.15.142:6834/2657443138,v1:172.21.15.142:6835/2657443138] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:26:59.399 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.k{3:24428} state up:active seq 13 addr [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:26:59.399 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a{3:14592} state up:standby-replay seq 1 addr [v2:172.21.15.16:6840/4017062522,v1:172.21.15.16:6841/4017062522] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:26:59.399 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.l{4:24424} state up:active seq 14 addr [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:26:59.399 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.g{4:14580} state up:standby-replay seq 1 addr [v2:172.21.15.16:6838/3573603820,v1:172.21.15.16:6839/3573603820] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:26:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:26:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:26:59.400 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-09-19T09:26:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:26:59.400 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.j{-1:14520} state up:standby seq 1 addr [v2:172.21.15.16:6834/3514257756,v1:172.21.15.16:6835/3514257756] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:26:59.400 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d{-1:14532} state up:standby seq 1 addr [v2:172.21.15.16:6836/2828730634,v1:172.21.15.16:6837/2828730634] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T09:26:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-19T09:26:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.397+0000 7fd88f38a640 1 -- 172.21.15.16:0/625276058 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd8740a3300 msgr2=0x7fd8740a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.397+0000 7fd88f38a640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd8740a3300 0x7fd8740a57c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd87c031440 tx=0x7fd87c0315b0 comp rx=0 tx=0).stop 2024-09-19T09:26:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.397+0000 7fd88f38a640 1 -- 172.21.15.16:0/625276058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd888100b10 msgr2=0x7fd888197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:26:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.397+0000 7fd88f38a640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd888100b10 0x7fd888197f10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd88400d770 tx=0x7fd88400dc30 comp rx=0 tx=0).stop 2024-09-19T09:26:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.397+0000 7fd88f38a640 1 -- 172.21.15.16:0/625276058 shutdown_connections 2024-09-19T09:26:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.397+0000 7fd88f38a640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd8740a3300 0x7fd8740a57c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.397+0000 7fd88f38a640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8881026d0 0x7fd88819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.397+0000 7fd88f38a640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd888101d10 0x7fd888198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.397+0000 7fd88f38a640 1 --2- 172.21.15.16:0/625276058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd888100b10 0x7fd888197f10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:26:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.397+0000 7fd88f38a640 1 -- 172.21.15.16:0/625276058 >> 172.21.15.16:0/625276058 conn(0x7fd8880fc2a0 msgr2=0x7fd8881050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:26:59.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.397+0000 7fd88f38a640 1 -- 172.21.15.16:0/625276058 shutdown_connections 2024-09-19T09:26:59.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:26:59.397+0000 7fd88f38a640 1 -- 172.21.15.16:0/625276058 wait complete. 2024-09-19T09:27:00.065 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:26:59 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/625276058' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T09:27:00.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T09:27:00.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:26:59 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/625276058' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T09:27:00.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:26:59 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/625276058' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T09:27:00.354 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.793+0000 7fb730e8e640 1 -- 172.21.15.16:0/631102852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb72c101d10 msgr2=0x7fb72c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.793+0000 7fb730e8e640 1 --2- 172.21.15.16:0/631102852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb72c101d10 0x7fb72c102190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb714009ca0 tx=0x7fb71402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.794+0000 7fb730e8e640 1 -- 172.21.15.16:0/631102852 shutdown_connections 2024-09-19T09:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.794+0000 7fb730e8e640 1 --2- 172.21.15.16:0/631102852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb72c1026d0 0x7fb72c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.794+0000 7fb730e8e640 1 --2- 172.21.15.16:0/631102852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb72c101d10 0x7fb72c102190 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.794+0000 7fb730e8e640 1 --2- 172.21.15.16:0/631102852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb72c100b10 0x7fb72c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.794+0000 7fb730e8e640 1 -- 172.21.15.16:0/631102852 >> 172.21.15.16:0/631102852 conn(0x7fb72c0fc2a0 msgr2=0x7fb72c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.794+0000 7fb730e8e640 1 -- 172.21.15.16:0/631102852 shutdown_connections 2024-09-19T09:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.794+0000 7fb730e8e640 1 -- 172.21.15.16:0/631102852 wait complete. 2024-09-19T09:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.794+0000 7fb730e8e640 1 Processor -- start 2024-09-19T09:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.794+0000 7fb730e8e640 1 -- start start 2024-09-19T09:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb730e8e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb72c100b10 0x7fb72c104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb730e8e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb72c101d10 0x7fb72c1030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb730e8e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb72c1026d0 0x7fb72c103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb730e8e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c10c770 con 0x7fb72c1026d0 2024-09-19T09:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb730e8e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c10c5f0 con 0x7fb72c100b10 2024-09-19T09:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb730e8e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c10c8f0 con 0x7fb72c101d10 2024-09-19T09:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb72ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb72c1026d0 0x7fb72c103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb72ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb72c1026d0 0x7fb72c103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51032/0 (socket says 172.21.15.16:51032) 2024-09-19T09:27:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb72a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb72c100b10 0x7fb72c104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb72ad9d640 1 -- 172.21.15.16:0/2818472924 learned_addr learned my addr 172.21.15.16:0/2818472924 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb729d9b640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb72c101d10 0x7fb72c1030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb72ad9d640 1 -- 172.21.15.16:0/2818472924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb72c101d10 msgr2=0x7fb72c1030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb72ad9d640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb72c101d10 0x7fb72c1030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb72ad9d640 1 -- 172.21.15.16:0/2818472924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb72c100b10 msgr2=0x7fb72c104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb72ad9d640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb72c100b10 0x7fb72c104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb72ad9d640 1 -- 172.21.15.16:0/2818472924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c103ef0 con 0x7fb72c1026d0 2024-09-19T09:27:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb72a59c640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb72c100b10 0x7fb72c104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb72ad9d640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb72c1026d0 0x7fb72c103630 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb72400d750 tx=0x7fb72400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.795+0000 7fb7237fe640 1 -- 172.21.15.16:0/2818472924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724017070 con 0x7fb72c1026d0 2024-09-19T09:27:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.796+0000 7fb7237fe640 1 -- 172.21.15.16:0/2818472924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb724005500 con 0x7fb72c1026d0 2024-09-19T09:27:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.796+0000 7fb730e8e640 1 -- 172.21.15.16:0/2818472924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c1a20a0 con 0x7fb72c1026d0 2024-09-19T09:27:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.796+0000 7fb7237fe640 1 -- 172.21.15.16:0/2818472924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724004d90 con 0x7fb72c1026d0 2024-09-19T09:27:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.796+0000 7fb730e8e640 1 -- 172.21.15.16:0/2818472924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c1a2560 con 0x7fb72c1026d0 2024-09-19T09:27:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.797+0000 7fb730e8e640 1 -- 172.21.15.16:0/2818472924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72c1a2790 con 0x7fb72c1026d0 2024-09-19T09:27:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.798+0000 7fb7237fe640 1 -- 172.21.15.16:0/2818472924 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb724004f30 con 0x7fb72c1026d0 2024-09-19T09:27:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.799+0000 7fb7237fe640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7180a32d0 0x7fb7180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.799+0000 7fb72a59c640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7180a32d0 0x7fb7180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.799+0000 7fb7237fe640 1 -- 172.21.15.16:0/2818472924 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb7240cf750 con 0x7fb72c1026d0 2024-09-19T09:27:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.799+0000 7fb72a59c640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7180a32d0 0x7fb7180a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb72c101b70 tx=0x7fb71c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.802+0000 7fb7237fe640 1 -- 172.21.15.16:0/2818472924 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb72409bcc0 con 0x7fb72c1026d0 2024-09-19T09:27:00.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.979+0000 7fb730e8e640 1 -- 172.21.15.16:0/2818472924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fb72c1a2960 con 0x7fb72c1026d0 2024-09-19T09:27:00.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.980+0000 7fb7237fe640 1 -- 172.21.15.16:0/2818472924 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v81) ==== 56+0+5291 (secure 0 0 0) 0x7fb72409ffc0 con 0x7fb72c1026d0 2024-09-19T09:27:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 81 2024-09-19T09:27:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:fsid ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:27:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-19T09:17:48.242238+0000 2024-09-19T09:27:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-19T09:25:42.982671+0000 2024-09-19T09:27:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T09:27:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:crush_version 26 2024-09-19T09:27:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:full_ratio 0.95 2024-09-19T09:27:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:backfillfull_ratio 0.9 2024-09-19T09:27:00.982 INFO:teuthology.orchestra.run.smithi016.stdout:nearfull_ratio 0.85 2024-09-19T09:27:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:require_min_compat_client luminous 2024-09-19T09:27:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:min_compat_client luminous 2024-09-19T09:27:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:require_osd_release squid 2024-09-19T09:27:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:stretch_mode_enabled false 2024-09-19T09:27:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T09:27:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-19T09:27:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T09:27:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:max_osd 12 2024-09-19T09:27:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 up in weight 1 up_from 8 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6802/1578957623,v1:172.21.15.16:6803/1578957623] [v2:172.21.15.16:6804/1578957623,v1:172.21.15.16:6805/1578957623] exists,up dbf3b28b-2282-4057-9e65-19ceb78ff19f 2024-09-19T09:27:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6800/3804152349,v1:172.21.15.97:6801/3804152349] [v2:172.21.15.97:6802/3804152349,v1:172.21.15.97:6803/3804152349] exists,up c2517aeb-1d79-41ad-a180-369eecb9732d 2024-09-19T09:27:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 up in weight 1 up_from 18 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6800/2692613951,v1:172.21.15.142:6801/2692613951] [v2:172.21.15.142:6802/2692613951,v1:172.21.15.142:6803/2692613951] exists,up 3c367518-5085-471a-98cb-ec5bf5facdf4 2024-09-19T09:27:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 up in weight 1 up_from 24 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6810/3169290004,v1:172.21.15.16:6811/3169290004] [v2:172.21.15.16:6812/3169290004,v1:172.21.15.16:6813/3169290004] exists,up e7118925-f0fa-46b3-8552-3e3bb76a6489 2024-09-19T09:27:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 up in weight 1 up_from 31 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6808/3491353825,v1:172.21.15.97:6809/3491353825] [v2:172.21.15.97:6810/3491353825,v1:172.21.15.97:6811/3491353825] exists,up dc1964f7-af9c-4853-843b-9818f87c7768 2024-09-19T09:27:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 up in weight 1 up_from 36 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6808/577099986,v1:172.21.15.142:6809/577099986] [v2:172.21.15.142:6810/577099986,v1:172.21.15.142:6811/577099986] exists,up 4d7fbafc-d9ba-4036-a89d-aa295b8a23a7 2024-09-19T09:27:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 up in weight 1 up_from 43 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6818/4053539536,v1:172.21.15.16:6819/4053539536] [v2:172.21.15.16:6820/4053539536,v1:172.21.15.16:6821/4053539536] exists,up e3f346fc-c027-44f8-bd37-bccf6ec09e3b 2024-09-19T09:27:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 up in weight 1 up_from 47 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6816/1157122670,v1:172.21.15.97:6817/1157122670] [v2:172.21.15.97:6818/1157122670,v1:172.21.15.97:6819/1157122670] exists,up d390b547-8949-4d4b-8481-c49d6ae7e424 2024-09-19T09:27:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 up in weight 1 up_from 54 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6816/643795004,v1:172.21.15.142:6817/643795004] [v2:172.21.15.142:6818/643795004,v1:172.21.15.142:6819/643795004] exists,up 78a476c4-5deb-4f9e-a266-db4493850732 2024-09-19T09:27:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 up in weight 1 up_from 61 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6826/2651132360,v1:172.21.15.16:6827/2651132360] [v2:172.21.15.16:6828/2651132360,v1:172.21.15.16:6829/2651132360] exists,up 0415d187-4a77-42f7-b8bb-2d0b52e9cb63 2024-09-19T09:27:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 up in weight 1 up_from 66 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6824/1848371486,v1:172.21.15.97:6825/1848371486] [v2:172.21.15.97:6826/1848371486,v1:172.21.15.97:6827/1848371486] exists,up 6fa78f96-cdfe-42a0-8d27-0705a0c7df8e 2024-09-19T09:27:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 up in weight 1 up_from 73 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6824/155083878,v1:172.21.15.142:6825/155083878] [v2:172.21.15.142:6826/155083878,v1:172.21.15.142:6827/155083878] exists,up d27f20e3-84ce-470c-9889-cd86906b158d 2024-09-19T09:27:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.0 [8,4] 2024-09-19T09:27:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.a [2,11] 2024-09-19T09:27:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.c [2,4] 2024-09-19T09:27:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.1f [7,11] 2024-09-19T09:27:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.0 [4,8] 2024-09-19T09:27:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.11 [4,6] 2024-09-19T09:27:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.29 [3,9] 2024-09-19T09:27:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.33 [5,9] 2024-09-19T09:27:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.34 [4,9] 2024-09-19T09:27:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/1680178094 expires 2024-09-20T09:18:41.244076+0000 2024-09-19T09:27:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3710267992 expires 2024-09-20T09:18:41.244076+0000 2024-09-19T09:27:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/1680178094 expires 2024-09-20T09:18:41.244076+0000 2024-09-19T09:27:00.985 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3740275561 expires 2024-09-20T09:18:41.244076+0000 2024-09-19T09:27:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2755421444 expires 2024-09-20T09:18:41.244076+0000 2024-09-19T09:27:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/183909517 expires 2024-09-20T09:18:17.070034+0000 2024-09-19T09:27:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/357390846 expires 2024-09-20T09:18:17.070034+0000 2024-09-19T09:27:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/183909517 expires 2024-09-20T09:18:17.070034+0000 2024-09-19T09:27:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3104087402 expires 2024-09-20T09:18:17.070034+0000 2024-09-19T09:27:00.986 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3546947161 expires 2024-09-20T09:18:17.070034+0000 2024-09-19T09:27:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.982+0000 7fb730e8e640 1 -- 172.21.15.16:0/2818472924 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7180a32d0 msgr2=0x7fb7180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.982+0000 7fb730e8e640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7180a32d0 0x7fb7180a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb72c101b70 tx=0x7fb71c005f90 comp rx=0 tx=0).stop 2024-09-19T09:27:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.982+0000 7fb730e8e640 1 -- 172.21.15.16:0/2818472924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb72c1026d0 msgr2=0x7fb72c103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.982+0000 7fb730e8e640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb72c1026d0 0x7fb72c103630 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb72400d750 tx=0x7fb72400dc10 comp rx=0 tx=0).stop 2024-09-19T09:27:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.982+0000 7fb730e8e640 1 -- 172.21.15.16:0/2818472924 shutdown_connections 2024-09-19T09:27:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.982+0000 7fb730e8e640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7180a32d0 0x7fb7180a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.982+0000 7fb730e8e640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb72c101d10 0x7fb72c1030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.982+0000 7fb730e8e640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb72c1026d0 0x7fb72c103630 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.982+0000 7fb730e8e640 1 --2- 172.21.15.16:0/2818472924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb72c100b10 0x7fb72c104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.982+0000 7fb730e8e640 1 -- 172.21.15.16:0/2818472924 >> 172.21.15.16:0/2818472924 conn(0x7fb72c0fc2a0 msgr2=0x7fb72c06a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.982+0000 7fb730e8e640 1 -- 172.21.15.16:0/2818472924 shutdown_connections 2024-09-19T09:27:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:00.982+0000 7fb730e8e640 1 -- 172.21.15.16:0/2818472924 wait complete. 2024-09-19T09:27:01.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:00 smithi142 ceph-mon[35032]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:27:01.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:00 smithi097 ceph-mon[34864]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:27:01.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:00 smithi016 ceph-mon[30979]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:27:01.681 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T09:27:01.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:27:01.951 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:02.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:01 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2818472924' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T09:27:02.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:01 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2818472924' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T09:27:02.179 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:01 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2818472924' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T09:27:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.401+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/2238186533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f3810e480 msgr2=0x7f5f38110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.401+0000 7f5f3fc1b640 1 --2- 172.21.15.16:0/2238186533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f3810e480 0x7f5f38110870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009ca0 tx=0x7f5f3002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:27:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.401+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/2238186533 shutdown_connections 2024-09-19T09:27:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.401+0000 7f5f3fc1b640 1 --2- 172.21.15.16:0/2238186533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f38101280 0x7f5f3810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.401+0000 7f5f3fc1b640 1 --2- 172.21.15.16:0/2238186533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f3810e480 0x7f5f38110870 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.401+0000 7f5f3fc1b640 1 --2- 172.21.15.16:0/2238186533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f381008d0 0x7f5f38100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.401+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/2238186533 >> 172.21.15.16:0/2238186533 conn(0x7f5f380fc7a0 msgr2=0x7f5f380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.401+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/2238186533 shutdown_connections 2024-09-19T09:27:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.401+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/2238186533 wait complete. 2024-09-19T09:27:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3fc1b640 1 Processor -- start 2024-09-19T09:27:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3fc1b640 1 -- start start 2024-09-19T09:27:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3fc1b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f381008d0 0x7f5f3819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3fc1b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f38101280 0x7f5f3819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3fc1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f3810e480 0x7f5f3819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3fc1b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5f381141c0 con 0x7f5f3810e480 2024-09-19T09:27:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3fc1b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f38114040 con 0x7f5f381008d0 2024-09-19T09:27:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3fc1b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f38114340 con 0x7f5f38101280 2024-09-19T09:27:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3e1b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f3810e480 0x7f5f3819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3e1b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f3810e480 0x7f5f3819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51046/0 (socket says 172.21.15.16:51046) 2024-09-19T09:27:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3e1b8640 1 -- 172.21.15.16:0/286116565 learned_addr learned my addr 172.21.15.16:0/286116565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3e1b8640 1 -- 172.21.15.16:0/286116565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f38101280 msgr2=0x7f5f3819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3e1b8640 1 --2- 172.21.15.16:0/286116565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f38101280 0x7f5f3819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3e1b8640 1 -- 172.21.15.16:0/286116565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f381008d0 msgr2=0x7f5f3819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3d9b7640 1 --2- 172.21.15.16:0/286116565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f381008d0 0x7f5f3819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3e1b8640 1 --2- 172.21.15.16:0/286116565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f381008d0 0x7f5f3819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.402+0000 7f5f3e1b8640 1 -- 172.21.15.16:0/286116565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3819f690 con 0x7f5f3810e480 2024-09-19T09:27:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.403+0000 7f5f3e1b8640 1 --2- 172.21.15.16:0/286116565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f3810e480 0x7f5f3819efb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5f3002ade0 tx=0x7f5f30031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.403+0000 7f5f2effd640 1 -- 172.21.15.16:0/286116565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30003c30 con 0x7f5f3810e480 2024-09-19T09:27:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.403+0000 7f5f2effd640 1 -- 172.21.15.16:0/286116565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f30003dd0 con 0x7f5f3810e480 2024-09-19T09:27:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.403+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/286116565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3819f920 con 0x7f5f3810e480 2024-09-19T09:27:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.403+0000 7f5f2effd640 1 -- 172.21.15.16:0/286116565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300047a0 con 0x7f5f3810e480 2024-09-19T09:27:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.403+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/286116565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f381a7260 con 0x7f5f3810e480 2024-09-19T09:27:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.404+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/286116565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f38100cb0 con 0x7f5f3810e480 2024-09-19T09:27:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.405+0000 7f5f2effd640 1 -- 172.21.15.16:0/286116565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5f3002f710 con 0x7f5f3810e480 2024-09-19T09:27:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.406+0000 7f5f2effd640 1 --2- 172.21.15.16:0/286116565 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5f240a3350 0x7f5f240a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.406+0000 7f5f3d9b7640 1 --2- 172.21.15.16:0/286116565 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5f240a3350 0x7f5f240a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.406+0000 7f5f2effd640 1 -- 172.21.15.16:0/286116565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5f300f0e40 con 0x7f5f3810e480 2024-09-19T09:27:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.406+0000 7f5f3d9b7640 1 --2- 172.21.15.16:0/286116565 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5f240a3350 0x7f5f240a5810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f340058c0 tx=0x7f5f34005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.409+0000 7f5f2effd640 1 -- 172.21.15.16:0/286116565 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5f300bdb10 con 0x7f5f3810e480 2024-09-19T09:27:02.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.620+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/286116565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f3819ba40 con 0x7f5f3810e480 2024-09-19T09:27:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.621+0000 7f5f2effd640 1 -- 172.21.15.16:0/286116565 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11967 (secure 0 0 0) 0x7f5f300bd410 con 0x7f5f3810e480 2024-09-19T09:27:02.623 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:02.623 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"btime":"2024-09-19T09:25:40:972080+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:25:40.972058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24484,24470,24428,24424]},"id":1}]} 2024-09-19T09:27:02.623 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-19T09:27:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.623+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/286116565 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5f240a3350 msgr2=0x7f5f240a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.623+0000 7f5f3fc1b640 1 --2- 172.21.15.16:0/286116565 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5f240a3350 0x7f5f240a5810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f340058c0 tx=0x7f5f34005c60 comp rx=0 tx=0).stop 2024-09-19T09:27:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.623+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/286116565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f3810e480 msgr2=0x7f5f3819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.623+0000 7f5f3fc1b640 1 --2- 172.21.15.16:0/286116565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f3810e480 0x7f5f3819efb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5f3002ade0 tx=0x7f5f30031a60 comp rx=0 tx=0).stop 2024-09-19T09:27:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.623+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/286116565 shutdown_connections 2024-09-19T09:27:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.623+0000 7f5f3fc1b640 1 --2- 172.21.15.16:0/286116565 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5f240a3350 0x7f5f240a5810 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.623+0000 7f5f3fc1b640 1 --2- 172.21.15.16:0/286116565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f38101280 0x7f5f3819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.623+0000 7f5f3fc1b640 1 --2- 172.21.15.16:0/286116565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f3810e480 0x7f5f3819efb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.623+0000 7f5f3fc1b640 1 --2- 172.21.15.16:0/286116565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f381008d0 0x7f5f3819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.623+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/286116565 >> 172.21.15.16:0/286116565 conn(0x7f5f380fc7a0 msgr2=0x7f5f380feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.623+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/286116565 shutdown_connections 2024-09-19T09:27:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:02.623+0000 7f5f3fc1b640 1 -- 172.21.15.16:0/286116565 wait complete. 2024-09-19T09:27:02.967 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:02 smithi016 ceph-mon[30979]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:27:02.968 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:02 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/286116565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:03.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:02 smithi142 ceph-mon[35032]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:27:03.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:02 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/286116565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:03.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:02 smithi097 ceph-mon[34864]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:27:03.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:02 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/286116565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:03.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:27:03.573 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:04.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.027+0000 7fbd7c280640 1 -- 172.21.15.16:0/199871172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74101420 msgr2=0x7fbd7410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:04.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.027+0000 7fbd7c280640 1 --2- 172.21.15.16:0/199871172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74101420 0x7fbd7410d950 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009930 tx=0x7fbd6c02a420 comp rx=0 tx=0).stop 2024-09-19T09:27:04.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.027+0000 7fbd7c280640 1 -- 172.21.15.16:0/199871172 shutdown_connections 2024-09-19T09:27:04.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.027+0000 7fbd7c280640 1 --2- 172.21.15.16:0/199871172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74101420 0x7fbd7410d950 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:04.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.027+0000 7fbd7c280640 1 --2- 172.21.15.16:0/199871172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd74100a80 0x7fbd74100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:04.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.027+0000 7fbd7c280640 1 --2- 172.21.15.16:0/199871172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd74106a80 0x7fbd74106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:04.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.027+0000 7fbd7c280640 1 -- 172.21.15.16:0/199871172 >> 172.21.15.16:0/199871172 conn(0x7fbd740fc7a0 msgr2=0x7fbd740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:04.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.027+0000 7fbd7c280640 1 -- 172.21.15.16:0/199871172 shutdown_connections 2024-09-19T09:27:04.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.027+0000 7fbd7c280640 1 -- 172.21.15.16:0/199871172 wait complete. 2024-09-19T09:27:04.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7c280640 1 Processor -- start 2024-09-19T09:27:04.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7c280640 1 -- start start 2024-09-19T09:27:04.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7c280640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd74100a80 0x7fbd74199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:04.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7c280640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74101420 0x7fbd74199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:04.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7c280640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd74106a80 0x7fbd741a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:04.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7c280640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74110e80 con 0x7fbd74101420 2024-09-19T09:27:04.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7c280640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74110d00 con 0x7fbd74100a80 2024-09-19T09:27:04.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7c280640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74111000 con 0x7fbd74106a80 2024-09-19T09:27:04.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7981b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74101420 0x7fbd74199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:04.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7981b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74101420 0x7fbd74199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51062/0 (socket says 172.21.15.16:51062) 2024-09-19T09:27:04.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7981b640 1 -- 172.21.15.16:0/58672799 learned_addr learned my addr 172.21.15.16:0/58672799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:04.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7a81d640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd74106a80 0x7fbd741a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:04.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7a01c640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd74100a80 0x7fbd74199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:04.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7981b640 1 -- 172.21.15.16:0/58672799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd74106a80 msgr2=0x7fbd741a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:04.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7981b640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd74106a80 0x7fbd741a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:04.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7981b640 1 -- 172.21.15.16:0/58672799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd74100a80 msgr2=0x7fbd74199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:04.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7981b640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd74100a80 0x7fbd74199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:04.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.028+0000 7fbd7981b640 1 -- 172.21.15.16:0/58672799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd741a47c0 con 0x7fbd74101420 2024-09-19T09:27:04.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.029+0000 7fbd7a01c640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd74100a80 0x7fbd74199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:04.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.029+0000 7fbd7981b640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74101420 0x7fbd74199740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800c920 tx=0x7fbd6800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:04.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.029+0000 7fbd737fe640 1 -- 172.21.15.16:0/58672799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68007c70 con 0x7fbd74101420 2024-09-19T09:27:04.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.029+0000 7fbd737fe640 1 -- 172.21.15.16:0/58672799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd68007e10 con 0x7fbd74101420 2024-09-19T09:27:04.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.029+0000 7fbd7c280640 1 -- 172.21.15.16:0/58672799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd741a4990 con 0x7fbd74101420 2024-09-19T09:27:04.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.029+0000 7fbd737fe640 1 -- 172.21.15.16:0/58672799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68018890 con 0x7fbd74101420 2024-09-19T09:27:04.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.029+0000 7fbd7c280640 1 -- 172.21.15.16:0/58672799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd741a4ed0 con 0x7fbd74101420 2024-09-19T09:27:04.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.030+0000 7fbd7c280640 1 -- 172.21.15.16:0/58672799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd74100ee0 con 0x7fbd74101420 2024-09-19T09:27:04.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.031+0000 7fbd737fe640 1 -- 172.21.15.16:0/58672799 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fbd68018a50 con 0x7fbd74101420 2024-09-19T09:27:04.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.032+0000 7fbd737fe640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbd5c0a3350 0x7fbd5c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:04.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.032+0000 7fbd7a01c640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbd5c0a3350 0x7fbd5c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:04.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.032+0000 7fbd737fe640 1 -- 172.21.15.16:0/58672799 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fbd680cfe60 con 0x7fbd74101420 2024-09-19T09:27:04.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.033+0000 7fbd7a01c640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbd5c0a3350 0x7fbd5c0a5810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400a2e0 tx=0x7fbd6400a270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:04.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.036+0000 7fbd737fe640 1 -- 172.21.15.16:0/58672799 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbd68014030 con 0x7fbd74101420 2024-09-19T09:27:04.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.248+0000 7fbd7c280640 1 -- 172.21.15.16:0/58672799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd741a5190 con 0x7fbd74101420 2024-09-19T09:27:04.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.249+0000 7fbd737fe640 1 -- 172.21.15.16:0/58672799 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11967 (secure 0 0 0) 0x7fbd6809caf0 con 0x7fbd74101420 2024-09-19T09:27:04.252 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:04.252 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"btime":"2024-09-19T09:25:40:972080+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:25:40.972058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24484,24470,24428,24424]},"id":1}]} 2024-09-19T09:27:04.252 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-19T09:27:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.251+0000 7fbd7c280640 1 -- 172.21.15.16:0/58672799 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbd5c0a3350 msgr2=0x7fbd5c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.251+0000 7fbd7c280640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbd5c0a3350 0x7fbd5c0a5810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400a2e0 tx=0x7fbd6400a270 comp rx=0 tx=0).stop 2024-09-19T09:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.251+0000 7fbd7c280640 1 -- 172.21.15.16:0/58672799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74101420 msgr2=0x7fbd74199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.251+0000 7fbd7c280640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74101420 0x7fbd74199740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800c920 tx=0x7fbd6800cde0 comp rx=0 tx=0).stop 2024-09-19T09:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.251+0000 7fbd7c280640 1 -- 172.21.15.16:0/58672799 shutdown_connections 2024-09-19T09:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.251+0000 7fbd7c280640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbd5c0a3350 0x7fbd5c0a5810 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.251+0000 7fbd7c280640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd74106a80 0x7fbd741a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.251+0000 7fbd7c280640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd74101420 0x7fbd74199740 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.251+0000 7fbd7c280640 1 --2- 172.21.15.16:0/58672799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd74100a80 0x7fbd74199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.251+0000 7fbd7c280640 1 -- 172.21.15.16:0/58672799 >> 172.21.15.16:0/58672799 conn(0x7fbd740fc7a0 msgr2=0x7fbd740fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.251+0000 7fbd7c280640 1 -- 172.21.15.16:0/58672799 shutdown_connections 2024-09-19T09:27:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:04.251+0000 7fbd7c280640 1 -- 172.21.15.16:0/58672799 wait complete. 2024-09-19T09:27:04.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:04 smithi016 ceph-mon[30979]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:27:04.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:04 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/58672799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:04.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:27:05.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:04 smithi142 ceph-mon[35032]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:27:05.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:04 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/58672799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:05.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:04 smithi097 ceph-mon[34864]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:27:05.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:04 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/58672799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:05.203 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.876+0000 7f985d66f640 1 -- 172.21.15.16:0/1670238546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9858101420 msgr2=0x7f985810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.876+0000 7f985d66f640 1 --2- 172.21.15.16:0/1670238546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9858101420 0x7f985810d950 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9850009cd0 tx=0x7f985002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:27:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.876+0000 7f985d66f640 1 -- 172.21.15.16:0/1670238546 shutdown_connections 2024-09-19T09:27:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.876+0000 7f985d66f640 1 --2- 172.21.15.16:0/1670238546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9858100a80 0x7f9858100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.876+0000 7f985d66f640 1 --2- 172.21.15.16:0/1670238546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9858101420 0x7f985810d950 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.876+0000 7f985d66f640 1 --2- 172.21.15.16:0/1670238546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9858106a80 0x7f9858106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.876+0000 7f985d66f640 1 -- 172.21.15.16:0/1670238546 >> 172.21.15.16:0/1670238546 conn(0x7f98580fc7a0 msgr2=0x7f98580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.876+0000 7f985d66f640 1 -- 172.21.15.16:0/1670238546 shutdown_connections 2024-09-19T09:27:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.876+0000 7f985d66f640 1 -- 172.21.15.16:0/1670238546 wait complete. 2024-09-19T09:27:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.877+0000 7f985d66f640 1 Processor -- start 2024-09-19T09:27:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.877+0000 7f985d66f640 1 -- start start 2024-09-19T09:27:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.877+0000 7f985d66f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9858100a80 0x7f9858198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.877+0000 7f985d66f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9858101420 0x7f9858198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.877+0000 7f985d66f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9858106a80 0x7f985819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.877+0000 7f985d66f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9858110f20 con 0x7f9858101420 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.877+0000 7f985d66f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9858110da0 con 0x7f9858100a80 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.877+0000 7f985d66f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98581110a0 con 0x7f9858106a80 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.877+0000 7f984ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9858101420 0x7f9858198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.877+0000 7f984ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9858101420 0x7f9858198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53038/0 (socket says 172.21.15.16:53038) 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.877+0000 7f984ffff640 1 -- 172.21.15.16:0/4093541596 learned_addr learned my addr 172.21.15.16:0/4093541596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.877+0000 7f9856ffd640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9858100a80 0x7f9858198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.877+0000 7f98577fe640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9858106a80 0x7f985819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.878+0000 7f984ffff640 1 -- 172.21.15.16:0/4093541596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9858106a80 msgr2=0x7f985819ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.878+0000 7f984ffff640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9858106a80 0x7f985819ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.878+0000 7f984ffff640 1 -- 172.21.15.16:0/4093541596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9858100a80 msgr2=0x7f9858198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.878+0000 7f984ffff640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9858100a80 0x7f9858198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.878+0000 7f984ffff640 1 -- 172.21.15.16:0/4093541596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985819d530 con 0x7f9858101420 2024-09-19T09:27:05.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.878+0000 7f984ffff640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9858101420 0x7f9858198ac0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f984000b4a0 tx=0x7f984000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:05.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.878+0000 7f9854ff9640 1 -- 172.21.15.16:0/4093541596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840015020 con 0x7f9858101420 2024-09-19T09:27:05.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.878+0000 7f985d66f640 1 -- 172.21.15.16:0/4093541596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f985819d820 con 0x7f9858101420 2024-09-19T09:27:05.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.878+0000 7f9854ff9640 1 -- 172.21.15.16:0/4093541596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9840011d10 con 0x7f9858101420 2024-09-19T09:27:05.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.878+0000 7f9854ff9640 1 -- 172.21.15.16:0/4093541596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984000f2e0 con 0x7f9858101420 2024-09-19T09:27:05.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.878+0000 7f985d66f640 1 -- 172.21.15.16:0/4093541596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98580fcd30 con 0x7f9858101420 2024-09-19T09:27:05.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.879+0000 7f985d66f640 1 -- 172.21.15.16:0/4093541596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9858100f60 con 0x7f9858101420 2024-09-19T09:27:05.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.880+0000 7f9854ff9640 1 -- 172.21.15.16:0/4093541596 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9840018020 con 0x7f9858101420 2024-09-19T09:27:05.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.881+0000 7f9854ff9640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f98440a32d0 0x7f98440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:05.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.881+0000 7f9856ffd640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f98440a32d0 0x7f98440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:05.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.881+0000 7f9854ff9640 1 -- 172.21.15.16:0/4093541596 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f98400cf320 con 0x7f9858101420 2024-09-19T09:27:05.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.882+0000 7f9856ffd640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f98440a32d0 0x7f98440a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9848009990 tx=0x7f9848009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:05.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:05.885+0000 7f9854ff9640 1 -- 172.21.15.16:0/4093541596 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f984001f030 con 0x7f9858101420 2024-09-19T09:27:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.060+0000 7f985d66f640 1 -- 172.21.15.16:0/4093541596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98581998e0 con 0x7f9858101420 2024-09-19T09:27:06.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.063+0000 7f9854ff9640 1 -- 172.21.15.16:0/4093541596 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22665 (secure 0 0 0) 0x7f98581998e0 con 0x7f9858101420 2024-09-19T09:27:06.064 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:06.064 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","created":"2024-09-19T09:17:48.242238+0000","modified":"2024-09-19T09:25:42.982671+0000","last_up_change":"2024-09-19T09:24:40.190289+0000","last_in_change":"2024-09-19T09:24:17.190380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:20:57.748686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:25:25.694451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:25:28.262997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbf3b28b-2282-4057-9e65-19ceb78ff19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6803","nonce":1578957623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6805","nonce":1578957623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6809","nonce":1578957623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6807","nonce":1578957623}]},"public_addr":"172.21.15.16:6803/1578957623","cluster_addr":"172.21.15.16:6805/1578957623","heartbeat_back_addr":"172.21.15.16:6809/1578957623","heartbeat_front_addr":"172.21.15.16:6807/1578957623","state":["exists","up"]},{"osd":1,"uuid":"c2517aeb-1d79-41ad-a180-369eecb9732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6801","nonce":3804152349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6803","nonce":3804152349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6807","nonce":3804152349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6805","nonce":3804152349}]},"public_addr":"172.21.15.97:6801/3804152349","cluster_addr":"172.21.15.97:6803/3804152349","heartbeat_back_addr":"172.21.15.97:6807/3804152349","heartbeat_front_addr":"172.21.15.97:6805/3804152349","state":["exists","up"]},{"osd":2,"uuid":"3c367518-5085-471a-98cb-ec5bf5facdf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6801","nonce":2692613951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6803","nonce":2692613951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6807","nonce":2692613951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6805","nonce":2692613951}]},"public_addr":"172.21.15.142:6801/2692613951","cluster_addr":"172.21.15.142:6803/2692613951","heartbeat_back_addr":"172.21.15.142:6807/2692613951","heartbeat_front_addr":"172.21.15.142:6805/2692613951","state":["exists","up"]},{"osd":3,"uuid":"e7118925-f0fa-46b3-8552-3e3bb76a6489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6811","nonce":3169290004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6813","nonce":3169290004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6817","nonce":3169290004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6815","nonce":3169290004}]},"public_addr":"172.21.15.16:6811/3169290004","cluster_addr":"172.21.15.16:6813/3169290004","heartbeat_back_addr":"172.21.15.16:6817/3169290004","heartbeat_front_addr":"172.21.15.16:6815/3169290004","state":["exists","up"]},{"osd":4,"uuid":"dc1964f7-af9c-4853-843b-9818f87c7768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6809","nonce":3491353825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6811","nonce":3491353825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6815","nonce":3491353825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6813","nonce":3491353825}]},"public_addr":"172.21.15.97:6809/3491353825","cluster_addr":"172.21.15.97:6811/3491353825","heartbeat_back_addr":"172.21.15.97:6815/3491353825","heartbeat_front_addr":"172.21.15.97:6813/3491353825","state":["exists","up"]},{"osd":5,"uuid":"4d7fbafc-d9ba-4036-a89d-aa295b8a23a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6809","nonce":577099986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6811","nonce":577099986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6815","nonce":577099986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6813","nonce":577099986}]},"public_addr":"172.21.15.142:6809/577099986","cluster_addr":"172.21.15.142:6811/577099986","heartbeat_back_addr":"172.21.15.142:6815/577099986","heartbeat_front_addr":"172.21.15.142:6813/577099986","state":["exists","up"]},{"osd":6,"uuid":"e3f346fc-c027-44f8-bd37-bccf6ec09e3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6819","nonce":4053539536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6821","nonce":4053539536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6825","nonce":4053539536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6823","nonce":4053539536}]},"public_addr":"172.21.15.16:6819/4053539536","cluster_addr":"172.21.15.16:6821/4053539536","heartbeat_back_addr":"172.21.15.16:6825/4053539536","heartbeat_front_addr":"172.21.15.16:6823/4053539536","state":["exists","up"]},{"osd":7,"uuid":"d390b547-8949-4d4b-8481-c49d6ae7e424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6817","nonce":1157122670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6819","nonce":1157122670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6823","nonce":1157122670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6821","nonce":1157122670}]},"public_addr":"172.21.15.97:6817/1157122670","cluster_addr":"172.21.15.97:6819/1157122670","heartbeat_back_addr":"172.21.15.97:6823/1157122670","heartbeat_front_addr":"172.21.15.97:6821/1157122670","state":["exists","up"]},{"osd":8,"uuid":"78a476c4-5deb-4f9e-a266-db4493850732","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6817","nonce":643795004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6819","nonce":643795004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6823","nonce":643795004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6821","nonce":643795004}]},"public_addr":"172.21.15.142:6817/643795004","cluster_addr":"172.21.15.142:6819/643795004","heartbeat_back_addr":"172.21.15.142:6823/643795004","heartbeat_front_addr":"172.21.15.142:6821/643795004","state":["exists","up"]},{"osd":9,"uuid":"0415d187-4a77-42f7-b8bb-2d0b52e9cb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6827","nonce":2651132360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6829","nonce":2651132360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2651132360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6831","nonce":2651132360}]},"public_addr":"172.21.15.16:6827/2651132360","cluster_addr":"172.21.15.16:6829/2651132360","heartbeat_back_addr":"172.21.15.16:6833/2651132360","heartbeat_front_addr":"172.21.15.16:6831/2651132360","state":["exists","up"]},{"osd":10,"uuid":"6fa78f96-cdfe-42a0-8d27-0705a0c7df8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6825","nonce":1848371486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6827","nonce":1848371486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6831","nonce":1848371486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6829","nonce":1848371486}]},"public_addr":"172.21.15.97:6825/1848371486","cluster_addr":"172.21.15.97:6827/1848371486","heartbeat_back_addr":"172.21.15.97:6831/1848371486","heartbeat_front_addr":"172.21.15.97:6829/1848371486","state":["exists","up"]},{"osd":11,"uuid":"d27f20e3-84ce-470c-9889-cd86906b158d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6825","nonce":155083878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6827","nonce":155083878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6831","nonce":155083878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6829","nonce":155083878}]},"public_addr":"172.21.15.142:6825/155083878","cluster_addr":"172.21.15.142:6827/155083878","heartbeat_back_addr":"172.21.15.142:6831/155083878","heartbeat_front_addr":"172.21.15.142:6829/155083878","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:05.088764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:29.928529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:55.236930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:20.379471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:44.984151+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:10.410074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:35.233553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:59.928700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:24.503069+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:49.853073+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:14.006841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:38.546467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3710267992":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3740275561":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/2755421444":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/357390846":"2024-09-20T09:18:17.070034+0000","172.21.15.16:6800/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3104087402":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3546947161":"2024-09-20T09:18:17.070034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:27:06.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.065+0000 7f985d66f640 1 -- 172.21.15.16:0/4093541596 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f98440a32d0 msgr2=0x7f98440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:06.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.065+0000 7f985d66f640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f98440a32d0 0x7f98440a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9848009990 tx=0x7f9848009110 comp rx=0 tx=0).stop 2024-09-19T09:27:06.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.065+0000 7f985d66f640 1 -- 172.21.15.16:0/4093541596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9858101420 msgr2=0x7f9858198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:06.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.065+0000 7f985d66f640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9858101420 0x7f9858198ac0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f984000b4a0 tx=0x7f984000b8a0 comp rx=0 tx=0).stop 2024-09-19T09:27:06.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.065+0000 7f985d66f640 1 -- 172.21.15.16:0/4093541596 shutdown_connections 2024-09-19T09:27:06.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.065+0000 7f985d66f640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f98440a32d0 0x7f98440a5790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:06.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.065+0000 7f985d66f640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9858106a80 0x7f985819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:06.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.065+0000 7f985d66f640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9858101420 0x7f9858198ac0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:06.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.065+0000 7f985d66f640 1 --2- 172.21.15.16:0/4093541596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9858100a80 0x7f9858198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:06.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.065+0000 7f985d66f640 1 -- 172.21.15.16:0/4093541596 >> 172.21.15.16:0/4093541596 conn(0x7f98580fc7a0 msgr2=0x7f985806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:06.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.065+0000 7f985d66f640 1 -- 172.21.15.16:0/4093541596 shutdown_connections 2024-09-19T09:27:06.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:06.065+0000 7f985d66f640 1 -- 172.21.15.16:0/4093541596 wait complete. 2024-09-19T09:27:06.870 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T09:25:29.829112+0000', 'modified': '2024-09-19T09:25:40.972058+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24500, 'mds_1': 24484, 'mds_2': 24470, 'mds_3': 24428, 'mds_4': 24424}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14580': {'gid': 14580, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/3573603820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 3573603820}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 3573603820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14592': {'gid': 14592, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/4017062522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 4017062522}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 4017062522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.142:6833/3906366829', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 3906366829}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 3906366829}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.97:6833/108652274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 108652274}, {'type': 'v1', 'addr': '172.21.15.97:6833', 'nonce': 108652274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/2657443138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 2657443138}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 2657443138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/4135704970', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 4135704970}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 4135704970}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6837/81091251', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 81091251}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 81091251}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.97:6837/2018284957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 2018284957}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 2018284957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.142:6839/3822134597', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3822134597}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3822134597}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6839/3225392797', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3225392797}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3225392797}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24500, 'qdb_cluster': [24500, 24484, 24470, 24428, 24424]} 2024-09-19T09:27:06.870 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T09:27:06.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.l status 2024-09-19T09:27:07.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:06 smithi142 ceph-mon[35032]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:27:07.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:06 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/4093541596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:27:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:07.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:06 smithi097 ceph-mon[34864]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:27:07.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:06 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/4093541596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:27:07.252 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:06 smithi016 ceph-mon[30979]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:27:07.252 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:06 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/4093541596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:27:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.616+0000 7fe2fd773640 1 -- 172.21.15.16:0/1059859010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f81037a0 msgr2=0x7fe2f810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.616+0000 7fe2fd773640 1 --2- 172.21.15.16:0/1059859010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f81037a0 0x7fe2f810bc00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009900 tx=0x7fe2ec02a410 comp rx=0 tx=0).stop 2024-09-19T09:27:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.616+0000 7fe2fd773640 1 -- 172.21.15.16:0/1059859010 shutdown_connections 2024-09-19T09:27:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.616+0000 7fe2fd773640 1 --2- 172.21.15.16:0/1059859010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f810c2f0 0x7fe2f810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.616+0000 7fe2fd773640 1 --2- 172.21.15.16:0/1059859010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f81037a0 0x7fe2f810bc00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.616+0000 7fe2fd773640 1 --2- 172.21.15.16:0/1059859010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f8102df0 0x7fe2f81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.616+0000 7fe2fd773640 1 -- 172.21.15.16:0/1059859010 >> 172.21.15.16:0/1059859010 conn(0x7fe2f80fc7d0 msgr2=0x7fe2f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.617+0000 7fe2fd773640 1 -- 172.21.15.16:0/1059859010 shutdown_connections 2024-09-19T09:27:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.617+0000 7fe2fd773640 1 -- 172.21.15.16:0/1059859010 wait complete. 2024-09-19T09:27:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.617+0000 7fe2fd773640 1 Processor -- start 2024-09-19T09:27:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.617+0000 7fe2fd773640 1 -- start start 2024-09-19T09:27:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.617+0000 7fe2fd773640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f8102df0 0x7fe2f8198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2fd773640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f81037a0 0x7fe2f8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2fd773640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f810c2f0 0x7fe2f819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2fd773640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f81120d0 con 0x7fe2f81037a0 2024-09-19T09:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2fd773640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f8111f50 con 0x7fe2f810c2f0 2024-09-19T09:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2fd773640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f8112250 con 0x7fe2f8102df0 2024-09-19T09:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f81037a0 0x7fe2f8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f81037a0 0x7fe2f8198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53048/0 (socket says 172.21.15.16:53048) 2024-09-19T09:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2f67fc640 1 -- 172.21.15.16:0/1953114382 learned_addr learned my addr 172.21.15.16:0/1953114382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2f6ffd640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f8102df0 0x7fe2f8198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f810c2f0 0x7fe2f819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2f67fc640 1 -- 172.21.15.16:0/1953114382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f8102df0 msgr2=0x7fe2f8198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2f67fc640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f8102df0 0x7fe2f8198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2f67fc640 1 -- 172.21.15.16:0/1953114382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f810c2f0 msgr2=0x7fe2f819ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2f67fc640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f810c2f0 0x7fe2f819ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2f67fc640 1 -- 172.21.15.16:0/1953114382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f819d530 con 0x7fe2f81037a0 2024-09-19T09:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f810c2f0 0x7fe2f819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2f6ffd640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f8102df0 0x7fe2f8198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2f67fc640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f81037a0 0x7fe2f8198ac0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec005940 tx=0x7fe2ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2dffff640 1 -- 172.21.15.16:0/1953114382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec03f030 con 0x7fe2f81037a0 2024-09-19T09:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2fd773640 1 -- 172.21.15.16:0/1953114382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f819d7c0 con 0x7fe2f81037a0 2024-09-19T09:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2dffff640 1 -- 172.21.15.16:0/1953114382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ec003ce0 con 0x7fe2f81037a0 2024-09-19T09:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2dffff640 1 -- 172.21.15.16:0/1953114382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec035070 con 0x7fe2f81037a0 2024-09-19T09:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.618+0000 7fe2fd773640 1 -- 172.21.15.16:0/1953114382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f81a5100 con 0x7fe2f81037a0 2024-09-19T09:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.621+0000 7fe2dffff640 1 -- 172.21.15.16:0/1953114382 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe2ec042030 con 0x7fe2f81037a0 2024-09-19T09:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.621+0000 7fe2ffb4c740 1 Processor -- start 2024-09-19T09:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.621+0000 7fe2ffb4c740 1 -- start start 2024-09-19T09:27:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.621+0000 7fe2ffb4c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 0x55ead6442db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.621+0000 7fe2ffb4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 0x55ead644d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.621+0000 7fe2ffb4c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead644dbb0 0x55ead644ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.621+0000 7fe2ffb4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ead63a9090 con 0x55ead64432f0 2024-09-19T09:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.621+0000 7fe2ffb4c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ead6350a70 con 0x55ead64429d0 2024-09-19T09:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.621+0000 7fe2ffb4c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ead63a5050 con 0x55ead644dbb0 2024-09-19T09:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.621+0000 7fe2f6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 0x55ead644d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.621+0000 7fe2f6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 0x55ead644d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53054/0 (socket says 172.21.15.16:53054) 2024-09-19T09:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.621+0000 7fe2f6ffd640 1 -- 172.21.15.16:0/307867975 learned_addr learned my addr 172.21.15.16:0/307867975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2dffff640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe2e00a32d0 0x7fe2e00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/307867975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 0x55ead6442db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2f67fc640 1 --2- 172.21.15.16:0/307867975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead644dbb0 0x55ead644ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2f6ffd640 1 -- 172.21.15.16:0/307867975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead644dbb0 msgr2=0x55ead644ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2f6ffd640 1 --2- 172.21.15.16:0/307867975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead644dbb0 0x55ead644ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2f6ffd640 1 -- 172.21.15.16:0/307867975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 msgr2=0x55ead6442db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2f6ffd640 1 --2- 172.21.15.16:0/307867975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 0x55ead6442db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2f6ffd640 1 -- 172.21.15.16:0/307867975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ead635d130 con 0x55ead64432f0 2024-09-19T09:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/307867975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 0x55ead6442db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2f67fc640 1 --2- 172.21.15.16:0/307867975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead644dbb0 0x55ead644ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe2e00a32d0 0x7fe2e00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2dffff640 1 -- 172.21.15.16:0/1953114382 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe2ec0f00a0 con 0x7fe2f81037a0 2024-09-19T09:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2f6ffd640 1 --2- 172.21.15.16:0/307867975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 0x55ead644d670 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8008c60 tx=0x7fe2e800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b38720ec0b11019 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2dd7fa640 1 -- 172.21.15.16:0/307867975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8014070 con 0x55ead64432f0 2024-09-19T09:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2dd7fa640 1 -- 172.21.15.16:0/307867975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e80047a0 con 0x55ead64432f0 2024-09-19T09:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.622+0000 7fe2dd7fa640 1 -- 172.21.15.16:0/307867975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8010760 con 0x55ead64432f0 2024-09-19T09:27:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.623+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/307867975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 msgr2=0x55ead644d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.623+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe2e00a32d0 0x7fe2e00a5790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00092f0 tx=0x7fe2f0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.623+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/307867975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 0x55ead644d670 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8008c60 tx=0x7fe2e800e720 comp rx=0 tx=0).stop 2024-09-19T09:27:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.623+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/307867975 shutdown_connections 2024-09-19T09:27:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.623+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/307867975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead644dbb0 0x55ead644ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.623+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/307867975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 0x55ead644d670 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.623+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/307867975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 0x55ead6442db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.623+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/307867975 >> 172.21.15.16:0/307867975 conn(0x55ead643ef60 msgr2=0x55ead643f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.623+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/307867975 shutdown_connections 2024-09-19T09:27:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.623+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/307867975 wait complete. 2024-09-19T09:27:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.623+0000 7fe2ffb4c740 1 -- start start 2024-09-19T09:27:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.623+0000 7fe2ffb4c740 1 Processor -- start 2024-09-19T09:27:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2ffb4c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 0x55ead644d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2ffb4c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64432f0 0x55ead644a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2ffb4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 0x55ead644ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2ffb4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ead6450780 con 0x55ead644dbb0 2024-09-19T09:27:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2ffb4c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ead6363f20 con 0x55ead64429d0 2024-09-19T09:27:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2ffb4c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ead6450900 con 0x55ead64432f0 2024-09-19T09:27:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2ffb4c740 10 client.0 _ll_register_callbacks cb 0x55ead632d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 0x55ead644ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 0x55ead644ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53058/0 (socket says 172.21.15.16:53058) 2024-09-19T09:27:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2f77fe640 1 -- 172.21.15.16:0/2297136323 learned_addr learned my addr 172.21.15.16:0/2297136323 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2f6ffd640 1 --2- 172.21.15.16:0/2297136323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 0x55ead644d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2f67fc640 1 --2- 172.21.15.16:0/2297136323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64432f0 0x55ead644a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2f77fe640 1 -- 172.21.15.16:0/2297136323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64432f0 msgr2=0x55ead644a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/2297136323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64432f0 0x55ead644a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2f77fe640 1 -- 172.21.15.16:0/2297136323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 msgr2=0x55ead644d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/2297136323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 0x55ead644d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2f77fe640 1 -- 172.21.15.16:0/2297136323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ead611a2b0 con 0x55ead644dbb0 2024-09-19T09:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/2297136323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 0x55ead644ac20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00136d0 tx=0x7fe2f0009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd30d4ab4d04c6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.624+0000 7fe2daff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2daff5640 1 -- 172.21.15.16:0/2297136323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f001c9e0 con 0x55ead644dbb0 2024-09-19T09:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2daff5640 1 -- 172.21.15.16:0/2297136323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe2f001cb80 con 0x55ead644dbb0 2024-09-19T09:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/2297136323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2e80047a0 con 0x55ead644dbb0 2024-09-19T09:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2daff5640 1 -- 172.21.15.16:0/2297136323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe2f001cd80 con 0x55ead644dbb0 2024-09-19T09:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2ffb4c740 10 client.15099 fetch_fsmap learned FSMap version 27 2024-09-19T09:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/2297136323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ead645bfb0 con 0x55ead644dbb0 2024-09-19T09:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2daff5640 1 -- 172.21.15.16:0/2297136323 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e80047a0 con 0x55ead644dbb0 2024-09-19T09:27:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2daff5640 1 -- 172.21.15.16:0/2297136323 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe2f0019030 con 0x55ead644dbb0 2024-09-19T09:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2ffb4c740 10 client.15099 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2ffb4c740 10 client.15099 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-19T09:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2ffb4c740 20 client.15099 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/2297136323 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55ead645e640 0x55ead645fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2ffb4c740 4 client.15099 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.625+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/2297136323 --> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ead63b6600 con 0x55ead645e640 2024-09-19T09:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.626+0000 7fe2f6ffd640 1 --2- 172.21.15.16:0/2297136323 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55ead645e640 0x55ead645fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.626+0000 7fe2daff5640 10 client.15099 ms_handle_connect on v2:172.21.15.142:6832/3906366829 2024-09-19T09:27:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.626+0000 7fe2f6ffd640 1 --2- 172.21.15.16:0/2297136323 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55ead645e640 0x55ead645fab0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9031f92340f18e37 server_cookie=f2f42c402d3d55d8 in_seq=0 out_seq=0 2024-09-19T09:27:07.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2daff5640 1 -- 172.21.15.16:0/2297136323 <== mds.4 v2:172.21.15.142:6832/3906366829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ead63b6600 con 0x55ead645e640 2024-09-19T09:27:07.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2daff5640 10 client.15099 handle_command_reply: tid=0 2024-09-19T09:27:07.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2ffb4c740 1 client.15099 shutdown 2024-09-19T09:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/2297136323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 msgr2=0x55ead644ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/2297136323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 0x55ead644ac20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00136d0 tx=0x7fe2f0009280 comp rx=0 tx=0).stop 2024-09-19T09:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/2297136323 shutdown_connections 2024-09-19T09:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/2297136323 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55ead645e640 0x55ead645fab0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2daff5640 0 client.15099 ms_handle_reset on v2:172.21.15.142:6832/3906366829 2024-09-19T09:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/2297136323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64432f0 0x55ead644a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/2297136323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 0x55ead644ac20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/2297136323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 0x55ead644d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/2297136323 >> 172.21.15.16:0/2297136323 conn(0x55ead643ef60 msgr2=0x55ead6444c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/2297136323 shutdown_connections 2024-09-19T09:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/2297136323 wait complete. 2024-09-19T09:27:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.627+0000 7fe2ffb4c740 20 client.15099 trim_cache size 0 max 16384 2024-09-19T09:27:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.645+0000 7fe2ffb4c740 1 Processor -- start 2024-09-19T09:27:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.645+0000 7fe2ffb4c740 1 -- start start 2024-09-19T09:27:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.645+0000 7fe2ffb4c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 0x55ead645fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.645+0000 7fe2ffb4c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64432f0 0x55ead643e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.645+0000 7fe2ffb4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 0x55ead643f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.645+0000 7fe2ffb4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ead645ddf0 con 0x55ead644dbb0 2024-09-19T09:27:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.645+0000 7fe2ffb4c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ead645e4c0 con 0x55ead64429d0 2024-09-19T09:27:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.645+0000 7fe2ffb4c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ead645eb90 con 0x55ead64432f0 2024-09-19T09:27:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.645+0000 7fe2f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 0x55ead643f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.646+0000 7fe2f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 0x55ead643f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53070/0 (socket says 172.21.15.16:53070) 2024-09-19T09:27:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.646+0000 7fe2f77fe640 1 -- 172.21.15.16:0/2960474853 learned_addr learned my addr 172.21.15.16:0/2960474853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.646+0000 7fe2f6ffd640 1 --2- 172.21.15.16:0/2960474853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 0x55ead645fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.646+0000 7fe2f67fc640 1 --2- 172.21.15.16:0/2960474853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64432f0 0x55ead643e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.646+0000 7fe2f77fe640 1 -- 172.21.15.16:0/2960474853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64432f0 msgr2=0x55ead643e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.646+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/2960474853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64432f0 0x55ead643e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.646+0000 7fe2f77fe640 1 -- 172.21.15.16:0/2960474853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 msgr2=0x55ead645fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.646+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/2960474853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 0x55ead645fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.646+0000 7fe2f77fe640 1 -- 172.21.15.16:0/2960474853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ead643fbe0 con 0x55ead644dbb0 2024-09-19T09:27:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.646+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/2960474853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 0x55ead643f340 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00255c0 tx=0x7fe2f0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd2d0d975d380d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.646+0000 7fe2dcff9640 1 -- 172.21.15.16:0/2960474853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f003d070 con 0x55ead644dbb0 2024-09-19T09:27:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.646+0000 7fe2dcff9640 1 -- 172.21.15.16:0/2960474853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f00134f0 con 0x55ead644dbb0 2024-09-19T09:27:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.646+0000 7fe2dcff9640 1 -- 172.21.15.16:0/2960474853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0034040 con 0x55ead644dbb0 2024-09-19T09:27:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.647+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/2960474853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 msgr2=0x55ead643f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.647+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/2960474853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 0x55ead643f340 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00255c0 tx=0x7fe2f0005fd0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.647+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/2960474853 shutdown_connections 2024-09-19T09:27:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.647+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/2960474853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64432f0 0x55ead643e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.647+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/2960474853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead644dbb0 0x55ead643f340 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.647+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/2960474853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead64429d0 0x55ead645fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.647+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/2960474853 >> 172.21.15.16:0/2960474853 conn(0x55ead643ef60 msgr2=0x55ead644a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.647+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/2960474853 shutdown_connections 2024-09-19T09:27:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.647+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/2960474853 wait complete. 2024-09-19T09:27:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.647+0000 7fe2ffb4c740 1 -- start start 2024-09-19T09:27:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.647+0000 7fe2ffb4c740 1 Processor -- start 2024-09-19T09:27:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2ffb4c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64429d0 0x55ead6468190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2ffb4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 0x55ead6466c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2ffb4c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead644dbb0 0x55ead6467160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2ffb4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ead645f930 con 0x55ead64432f0 2024-09-19T09:27:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2ffb4c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ead645f260 con 0x55ead644dbb0 2024-09-19T09:27:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2ffb4c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ead6460000 con 0x55ead64429d0 2024-09-19T09:27:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2ffb4c740 10 client.0 _ll_register_callbacks cb 0x7fe2f0009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 0x55ead6466c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 0x55ead6466c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53076/0 (socket says 172.21.15.16:53076) 2024-09-19T09:27:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2f67fc640 1 -- 172.21.15.16:0/3377742942 learned_addr learned my addr 172.21.15.16:0/3377742942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2f6ffd640 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64429d0 0x55ead6468190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead644dbb0 0x55ead6467160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2f67fc640 1 -- 172.21.15.16:0/3377742942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64429d0 msgr2=0x55ead6468190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2f67fc640 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64429d0 0x55ead6468190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2f67fc640 1 -- 172.21.15.16:0/3377742942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead644dbb0 msgr2=0x55ead6467160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2f67fc640 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead644dbb0 0x55ead6467160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2f67fc640 1 -- 172.21.15.16:0/3377742942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ead64676a0 con 0x55ead64432f0 2024-09-19T09:27:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2f77fe640 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead644dbb0 0x55ead6467160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2f67fc640 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 0x55ead6466c20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0b89c0 tx=0x7fe2ec08b2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68901360f391f08a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2daff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2daff5640 1 -- 172.21.15.16:0/3377742942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec049050 con 0x55ead64432f0 2024-09-19T09:27:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2daff5640 1 -- 172.21.15.16:0/3377742942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe2ec0b8bd0 con 0x55ead64432f0 2024-09-19T09:27:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.648+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/3377742942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2f00134f0 con 0x55ead64432f0 2024-09-19T09:27:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.649+0000 7fe2daff5640 1 -- 172.21.15.16:0/3377742942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe2ec0b8800 con 0x55ead64432f0 2024-09-19T09:27:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.649+0000 7fe2ffb4c740 10 client.15105 fetch_fsmap learned FSMap version 27 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.649+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/3377742942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ead64cca10 con 0x55ead64432f0 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.649+0000 7fe2daff5640 1 -- 172.21.15.16:0/3377742942 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f00134f0 con 0x55ead64432f0 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.649+0000 7fe2daff5640 1 -- 172.21.15.16:0/3377742942 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe2ec03bc50 con 0x55ead64432f0 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.649+0000 7fe2ffb4c740 10 client.15105 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.649+0000 7fe2ffb4c740 10 client.15105 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.649+0000 7fe2ffb4c740 20 client.15105 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.649+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55ead645e640 0x55ead64d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.649+0000 7fe2ffb4c740 4 client.15105 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.649+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/3377742942 --> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] -- command(tid 0: {"prefix": "status"}) -- 0x55ead63b0b10 con 0x55ead645e640 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.649+0000 7fe2f6ffd640 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55ead645e640 0x55ead64d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2f6ffd640 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55ead645e640 0x55ead64d02d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e5cae5b1c033f7a7 server_cookie=2bfa7b48e37eae83 in_seq=0 out_seq=0 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2daff5640 10 client.15105 ms_handle_connect on v2:172.21.15.142:6832/3906366829 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2daff5640 1 -- 172.21.15.16:0/3377742942 <== mds.4 v2:172.21.15.142:6832/3906366829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ead63b0b10 con 0x55ead645e640 2024-09-19T09:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2daff5640 10 client.15105 handle_command_reply: tid=0 2024-09-19T09:27:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2ffb4c740 1 client.15105 shutdown 2024-09-19T09:27:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/3377742942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 msgr2=0x55ead6466c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 0x55ead6466c20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0b89c0 tx=0x7fe2ec08b2a0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/3377742942 shutdown_connections 2024-09-19T09:27:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55ead645e640 0x55ead64d02d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ead644dbb0 0x55ead6467160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ead64432f0 0x55ead6466c20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2ffb4c740 1 --2- 172.21.15.16:0/3377742942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ead64429d0 0x55ead6468190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/3377742942 >> 172.21.15.16:0/3377742942 conn(0x55ead643ef60 msgr2=0x55ead6448bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2daff5640 0 client.15105 ms_handle_reset on v2:172.21.15.142:6832/3906366829 2024-09-19T09:27:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/3377742942 shutdown_connections 2024-09-19T09:27:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2ffb4c740 1 -- 172.21.15.16:0/3377742942 wait complete. 2024-09-19T09:27:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.650+0000 7fe2ffb4c740 20 client.15105 trim_cache size 0 max 16384 2024-09-19T09:27:07.659 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:27:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:27:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-09-19T09:27:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24424, 2024-09-19T09:27:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:27:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:27:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:27:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 87.683592794999996, 2024-09-19T09:27:07.659 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-19T09:27:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:27:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:27:07.660 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 138.10245625600001 2024-09-19T09:27:07.660 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:27:07.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.651+0000 7fe2fd773640 1 -- 172.21.15.16:0/1953114382 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe2e00a32d0 msgr2=0x7fe2e00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.651+0000 7fe2fd773640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe2e00a32d0 0x7fe2e00a5790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00092f0 tx=0x7fe2f0004a40 comp rx=0 tx=0).stop 2024-09-19T09:27:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.651+0000 7fe2fd773640 1 -- 172.21.15.16:0/1953114382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f81037a0 msgr2=0x7fe2f8198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.651+0000 7fe2fd773640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f81037a0 0x7fe2f8198ac0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec005940 tx=0x7fe2ec0042f0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.651+0000 7fe2fd773640 1 -- 172.21.15.16:0/1953114382 shutdown_connections 2024-09-19T09:27:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.651+0000 7fe2fd773640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f810c2f0 0x7fe2f819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.651+0000 7fe2fd773640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f81037a0 0x7fe2f8198ac0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.651+0000 7fe2fd773640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe2e00a32d0 0x7fe2e00a5790 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.651+0000 7fe2fd773640 1 --2- 172.21.15.16:0/1953114382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2f8102df0 0x7fe2f8198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.651+0000 7fe2fd773640 1 -- 172.21.15.16:0/1953114382 >> 172.21.15.16:0/1953114382 conn(0x7fe2f80fc7d0 msgr2=0x7fe2f80fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.651+0000 7fe2fd773640 1 -- 172.21.15.16:0/1953114382 shutdown_connections 2024-09-19T09:27:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:07.651+0000 7fe2fd773640 1 -- 172.21.15.16:0/1953114382 wait complete. 2024-09-19T09:27:08.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.k status 2024-09-19T09:27:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:09.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:08 smithi016 ceph-mon[30979]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:27:09.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:08 smithi142 ceph-mon[35032]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:27:09.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:08 smithi097 ceph-mon[34864]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-19T09:27:09.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.195+0000 7f2d4aacd640 1 -- 172.21.15.16:0/1141312703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d44106f40 msgr2=0x7f2d4410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.195+0000 7f2d4aacd640 1 --2- 172.21.15.16:0/1141312703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d44106f40 0x7f2d4410eac0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009ca0 tx=0x7f2d3c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.195+0000 7f2d4aacd640 1 -- 172.21.15.16:0/1141312703 shutdown_connections 2024-09-19T09:27:09.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.195+0000 7f2d4aacd640 1 --2- 172.21.15.16:0/1141312703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d441027c0 0x7f2d44106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.195+0000 7f2d4aacd640 1 --2- 172.21.15.16:0/1141312703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d44106f40 0x7f2d4410eac0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.195+0000 7f2d4aacd640 1 --2- 172.21.15.16:0/1141312703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d44101e10 0x7f2d441021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.196+0000 7f2d4aacd640 1 -- 172.21.15.16:0/1141312703 >> 172.21.15.16:0/1141312703 conn(0x7f2d440fd8d0 msgr2=0x7f2d440ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.196+0000 7f2d4aacd640 1 -- 172.21.15.16:0/1141312703 shutdown_connections 2024-09-19T09:27:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.196+0000 7f2d4aacd640 1 -- 172.21.15.16:0/1141312703 wait complete. 2024-09-19T09:27:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.196+0000 7f2d4aacd640 1 Processor -- start 2024-09-19T09:27:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.196+0000 7f2d4aacd640 1 -- start start 2024-09-19T09:27:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.196+0000 7f2d4aacd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d44101e10 0x7f2d44198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.196+0000 7f2d4aacd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d441027c0 0x7f2d44198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.196+0000 7f2d4aacd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d44106f40 0x7f2d4419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.196+0000 7f2d4aacd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d44111fc0 con 0x7f2d441027c0 2024-09-19T09:27:09.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.196+0000 7f2d4aacd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d44111e40 con 0x7f2d44101e10 2024-09-19T09:27:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.196+0000 7f2d4aacd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d44112140 con 0x7f2d44106f40 2024-09-19T09:27:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d441027c0 0x7f2d44198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d441027c0 0x7f2d44198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53096/0 (socket says 172.21.15.16:53096) 2024-09-19T09:27:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d43fff640 1 -- 172.21.15.16:0/829706112 learned_addr learned my addr 172.21.15.16:0/829706112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d48869640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d44101e10 0x7f2d44198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d4906a640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d44106f40 0x7f2d4419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d43fff640 1 -- 172.21.15.16:0/829706112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d44106f40 msgr2=0x7f2d4419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d43fff640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d44106f40 0x7f2d4419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d43fff640 1 -- 172.21.15.16:0/829706112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d44101e10 msgr2=0x7f2d44198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d43fff640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d44101e10 0x7f2d44198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d43fff640 1 -- 172.21.15.16:0/829706112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4419d3b0 con 0x7f2d441027c0 2024-09-19T09:27:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d43fff640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d441027c0 0x7f2d44198940 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800b4a0 tx=0x7f2d3800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:09.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d41ffb640 1 -- 172.21.15.16:0/829706112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38015020 con 0x7f2d441027c0 2024-09-19T09:27:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d41ffb640 1 -- 172.21.15.16:0/829706112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d38011d10 con 0x7f2d441027c0 2024-09-19T09:27:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d4aacd640 1 -- 172.21.15.16:0/829706112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4419d6a0 con 0x7f2d441027c0 2024-09-19T09:27:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d41ffb640 1 -- 172.21.15.16:0/829706112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800f2e0 con 0x7f2d441027c0 2024-09-19T09:27:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.197+0000 7f2d4aacd640 1 -- 172.21.15.16:0/829706112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d441a4f80 con 0x7f2d441027c0 2024-09-19T09:27:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d41ffb640 1 -- 172.21.15.16:0/829706112 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2d38018020 con 0x7f2d441027c0 2024-09-19T09:27:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d4cea6740 1 Processor -- start 2024-09-19T09:27:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d4cea6740 1 -- start start 2024-09-19T09:27:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d41ffb640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2d2c0a3350 0x7f2d2c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d4cea6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b445bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d4cea6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b445c2f0 0x5645b4466670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d4cea6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b4466bb0 0x5645b4468fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d4cea6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5645b43c2090 con 0x5645b445b9d0 2024-09-19T09:27:09.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d4cea6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5645b4369a70 con 0x5645b445c2f0 2024-09-19T09:27:09.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d4cea6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5645b43be050 con 0x5645b4466bb0 2024-09-19T09:27:09.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d48869640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b445bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d4906a640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2d2c0a3350 0x7f2d2c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d41ffb640 1 -- 172.21.15.16:0/829706112 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2d380cf240 con 0x7f2d441027c0 2024-09-19T09:27:09.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d48869640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b445bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53110/0 (socket says 172.21.15.16:53110) 2024-09-19T09:27:09.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.200+0000 7f2d48869640 1 -- 172.21.15.16:0/1414621587 learned_addr learned my addr 172.21.15.16:0/1414621587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:09.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.201+0000 7f2d4906a640 1 --2- 172.21.15.16:0/1414621587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b4466bb0 0x5645b4468fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.201+0000 7f2d43fff640 1 --2- 172.21.15.16:0/1414621587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b445c2f0 0x5645b4466670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.201+0000 7f2d48869640 1 -- 172.21.15.16:0/1414621587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b4466bb0 msgr2=0x5645b4468fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.201+0000 7f2d48869640 1 --2- 172.21.15.16:0/1414621587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b4466bb0 0x5645b4468fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.201+0000 7f2d48869640 1 -- 172.21.15.16:0/1414621587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b445c2f0 msgr2=0x5645b4466670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.201+0000 7f2d48869640 1 --2- 172.21.15.16:0/1414621587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b445c2f0 0x5645b4466670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.201+0000 7f2d48869640 1 -- 172.21.15.16:0/1414621587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b4376130 con 0x5645b445b9d0 2024-09-19T09:27:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.201+0000 7f2d48869640 1 --2- 172.21.15.16:0/1414621587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b445bdb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400ad00 tx=0x7f2d3400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f27214dc78d502 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.201+0000 7f2d4906a640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2d2c0a3350 0x7f2d2c0a5810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5645b44592e0 tx=0x7f2d3c005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.201+0000 7f2d2affd640 1 -- 172.21.15.16:0/1414621587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34021070 con 0x5645b445b9d0 2024-09-19T09:27:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.201+0000 7f2d2affd640 1 -- 172.21.15.16:0/1414621587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3401c440 con 0x5645b445b9d0 2024-09-19T09:27:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.201+0000 7f2d2affd640 1 -- 172.21.15.16:0/1414621587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3401a560 con 0x5645b445b9d0 2024-09-19T09:27:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.202+0000 7f2d4cea6740 1 -- 172.21.15.16:0/1414621587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 msgr2=0x5645b445bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.202+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/1414621587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b445bdb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400ad00 tx=0x7f2d3400f790 comp rx=0 tx=0).stop 2024-09-19T09:27:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.202+0000 7f2d4cea6740 1 -- 172.21.15.16:0/1414621587 shutdown_connections 2024-09-19T09:27:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.202+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/1414621587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b4466bb0 0x5645b4468fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.202+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/1414621587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b445c2f0 0x5645b4466670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.202+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/1414621587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b445bdb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.202+0000 7f2d4cea6740 1 -- 172.21.15.16:0/1414621587 >> 172.21.15.16:0/1414621587 conn(0x5645b4457f60 msgr2=0x5645b4458340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.202+0000 7f2d4cea6740 1 -- 172.21.15.16:0/1414621587 shutdown_connections 2024-09-19T09:27:09.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.202+0000 7f2d4cea6740 1 -- 172.21.15.16:0/1414621587 wait complete. 2024-09-19T09:27:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.202+0000 7f2d4cea6740 1 -- start start 2024-09-19T09:27:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.202+0000 7f2d4cea6740 1 Processor -- start 2024-09-19T09:27:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d4cea6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b44663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d4cea6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b445c2f0 0x5645b4463770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d4cea6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b4466bb0 0x5645b4463cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d4cea6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5645b4469780 con 0x5645b445b9d0 2024-09-19T09:27:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d4cea6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5645b437cf20 con 0x5645b445c2f0 2024-09-19T09:27:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d4cea6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5645b4469900 con 0x5645b4466bb0 2024-09-19T09:27:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d4cea6740 10 client.0 _ll_register_callbacks cb 0x5645b4346030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d48869640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b44663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d48869640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b44663d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53118/0 (socket says 172.21.15.16:53118) 2024-09-19T09:27:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d48869640 1 -- 172.21.15.16:0/3175136810 learned_addr learned my addr 172.21.15.16:0/3175136810 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d4906a640 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b4466bb0 0x5645b4463cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d43fff640 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b445c2f0 0x5645b4463770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d48869640 1 -- 172.21.15.16:0/3175136810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b4466bb0 msgr2=0x5645b4463cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d48869640 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b4466bb0 0x5645b4463cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d48869640 1 -- 172.21.15.16:0/3175136810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b445c2f0 msgr2=0x5645b4463770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d48869640 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b445c2f0 0x5645b4463770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d48869640 1 -- 172.21.15.16:0/3175136810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b41332b0 con 0x5645b445b9d0 2024-09-19T09:27:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.203+0000 7f2d43fff640 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b445c2f0 0x5645b4463770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d48869640 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b44663d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2d3401f040 tx=0x7f2d3401c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f572f062c43ff386 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d23fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d23fff640 1 -- 172.21.15.16:0/3175136810 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34021040 con 0x5645b445b9d0 2024-09-19T09:27:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d23fff640 1 -- 172.21.15.16:0/3175136810 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2d34005240 con 0x5645b445b9d0 2024-09-19T09:27:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d4cea6740 1 -- 172.21.15.16:0/3175136810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d3401c440 con 0x5645b445b9d0 2024-09-19T09:27:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d23fff640 1 -- 172.21.15.16:0/3175136810 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d3401ca10 con 0x5645b445b9d0 2024-09-19T09:27:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d4cea6740 10 client.15117 fetch_fsmap learned FSMap version 27 2024-09-19T09:27:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d4cea6740 1 -- 172.21.15.16:0/3175136810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5645b4475050 con 0x5645b445b9d0 2024-09-19T09:27:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d23fff640 1 -- 172.21.15.16:0/3175136810 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3401c440 con 0x5645b445b9d0 2024-09-19T09:27:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d23fff640 1 -- 172.21.15.16:0/3175136810 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2d34024030 con 0x5645b445b9d0 2024-09-19T09:27:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d4cea6740 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:27:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d4cea6740 10 client.15117 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-19T09:27:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d4cea6740 20 client.15117 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x5645b44772e0 0x5645b4478750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d4cea6740 4 client.15117 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.204+0000 7f2d4cea6740 1 -- 172.21.15.16:0/3175136810 --> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645b43cf600 con 0x5645b44772e0 2024-09-19T09:27:09.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.205+0000 7f2d43fff640 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x5645b44772e0 0x5645b4478750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.205+0000 7f2d43fff640 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x5645b44772e0 0x5645b4478750 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd06d68a63c656be server_cookie=fb6947c20c2ad819 in_seq=0 out_seq=0 2024-09-19T09:27:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.205+0000 7f2d23fff640 10 client.15117 ms_handle_connect on v2:172.21.15.97:6832/108652274 2024-09-19T09:27:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d23fff640 1 -- 172.21.15.16:0/3175136810 <== mds.3 v2:172.21.15.97:6832/108652274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645b43cf600 con 0x5645b44772e0 2024-09-19T09:27:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d23fff640 10 client.15117 handle_command_reply: tid=0 2024-09-19T09:27:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d4cea6740 1 client.15117 shutdown 2024-09-19T09:27:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d4cea6740 1 -- 172.21.15.16:0/3175136810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 msgr2=0x5645b44663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b44663d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2d3401f040 tx=0x7f2d3401c820 comp rx=0 tx=0).stop 2024-09-19T09:27:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d4cea6740 1 -- 172.21.15.16:0/3175136810 shutdown_connections 2024-09-19T09:27:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x5645b44772e0 0x5645b4478750 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b4466bb0 0x5645b4463cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d23fff640 0 client.15117 ms_handle_reset on v2:172.21.15.97:6832/108652274 2024-09-19T09:27:09.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b445c2f0 0x5645b4463770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/3175136810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b44663d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d4cea6740 1 -- 172.21.15.16:0/3175136810 >> 172.21.15.16:0/3175136810 conn(0x5645b4457f60 msgr2=0x5645b445dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:09.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d4cea6740 1 -- 172.21.15.16:0/3175136810 shutdown_connections 2024-09-19T09:27:09.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d4cea6740 1 -- 172.21.15.16:0/3175136810 wait complete. 2024-09-19T09:27:09.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.206+0000 7f2d4cea6740 20 client.15117 trim_cache size 0 max 16384 2024-09-19T09:27:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.224+0000 7f2d4cea6740 1 Processor -- start 2024-09-19T09:27:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.224+0000 7f2d4cea6740 1 -- start start 2024-09-19T09:27:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d4cea6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b445b9d0 0x5645b4478750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d4cea6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445c2f0 0x5645b44655d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d4cea6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b4466bb0 0x5645b44609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d4cea6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5645b4476a90 con 0x5645b445c2f0 2024-09-19T09:27:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d4cea6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5645b4477160 con 0x5645b4466bb0 2024-09-19T09:27:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d4cea6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5645b4477830 con 0x5645b445b9d0 2024-09-19T09:27:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445c2f0 0x5645b44655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445c2f0 0x5645b44655d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53132/0 (socket says 172.21.15.16:53132) 2024-09-19T09:27:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d43fff640 1 -- 172.21.15.16:0/1669253595 learned_addr learned my addr 172.21.15.16:0/1669253595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d4906a640 1 --2- 172.21.15.16:0/1669253595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b4466bb0 0x5645b44609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d43fff640 1 -- 172.21.15.16:0/1669253595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b445b9d0 msgr2=0x5645b4478750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d48869640 1 --2- 172.21.15.16:0/1669253595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b445b9d0 0x5645b4478750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d43fff640 1 --2- 172.21.15.16:0/1669253595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b445b9d0 0x5645b4478750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d43fff640 1 -- 172.21.15.16:0/1669253595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b4466bb0 msgr2=0x5645b44609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d43fff640 1 --2- 172.21.15.16:0/1669253595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b4466bb0 0x5645b44609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d43fff640 1 -- 172.21.15.16:0/1669253595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b445c120 con 0x5645b445c2f0 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d48869640 1 --2- 172.21.15.16:0/1669253595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b445b9d0 0x5645b4478750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d4906a640 1 --2- 172.21.15.16:0/1669253595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b4466bb0 0x5645b44609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d43fff640 1 --2- 172.21.15.16:0/1669253595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445c2f0 0x5645b44655d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800b470 tx=0x7f2d380047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d028259ab77987 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d2a7fc640 1 -- 172.21.15.16:0/1669253595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d380787f0 con 0x5645b445c2f0 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d2a7fc640 1 -- 172.21.15.16:0/1669253595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3809b7d0 con 0x5645b445c2f0 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.225+0000 7f2d2a7fc640 1 -- 172.21.15.16:0/1669253595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3807a070 con 0x5645b445c2f0 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.226+0000 7f2d4cea6740 1 -- 172.21.15.16:0/1669253595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445c2f0 msgr2=0x5645b44655d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.226+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/1669253595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445c2f0 0x5645b44655d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800b470 tx=0x7f2d380047d0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.226+0000 7f2d4cea6740 1 -- 172.21.15.16:0/1669253595 shutdown_connections 2024-09-19T09:27:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.226+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/1669253595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b4466bb0 0x5645b44609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.226+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/1669253595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445c2f0 0x5645b44655d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.226+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/1669253595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b445b9d0 0x5645b4478750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.226+0000 7f2d4cea6740 1 -- 172.21.15.16:0/1669253595 >> 172.21.15.16:0/1669253595 conn(0x5645b4457f60 msgr2=0x5645b4464ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.226+0000 7f2d4cea6740 1 -- 172.21.15.16:0/1669253595 shutdown_connections 2024-09-19T09:27:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.226+0000 7f2d4cea6740 1 -- 172.21.15.16:0/1669253595 wait complete. 2024-09-19T09:27:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.226+0000 7f2d4cea6740 1 -- start start 2024-09-19T09:27:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.226+0000 7f2d4cea6740 1 Processor -- start 2024-09-19T09:27:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d4cea6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b4480fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d4cea6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b445c2f0 0x5645b447c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d4cea6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b4466bb0 0x5645b447c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d4cea6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5645b44785d0 con 0x5645b445b9d0 2024-09-19T09:27:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d4cea6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5645b4477f00 con 0x5645b4466bb0 2024-09-19T09:27:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d4cea6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5645b4478ca0 con 0x5645b445c2f0 2024-09-19T09:27:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d4cea6740 10 client.0 _ll_register_callbacks cb 0x7f2d3401f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d48869640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b4480fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d48869640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b4480fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53144/0 (socket says 172.21.15.16:53144) 2024-09-19T09:27:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d48869640 1 -- 172.21.15.16:0/447485224 learned_addr learned my addr 172.21.15.16:0/447485224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d4906a640 1 --2- 172.21.15.16:0/447485224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b4466bb0 0x5645b447c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d43fff640 1 --2- 172.21.15.16:0/447485224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b445c2f0 0x5645b447c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d48869640 1 -- 172.21.15.16:0/447485224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b445c2f0 msgr2=0x5645b447c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d48869640 1 --2- 172.21.15.16:0/447485224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b445c2f0 0x5645b447c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d48869640 1 -- 172.21.15.16:0/447485224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b4466bb0 msgr2=0x5645b447c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d48869640 1 --2- 172.21.15.16:0/447485224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b4466bb0 0x5645b447c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.227+0000 7f2d48869640 1 -- 172.21.15.16:0/447485224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b447ce50 con 0x5645b445b9d0 2024-09-19T09:27:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d48869640 1 --2- 172.21.15.16:0/447485224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b4480fa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400f7d0 tx=0x7f2d340274c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b315073986cfb14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:09.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d23fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:09.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d23fff640 1 -- 172.21.15.16:0/447485224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34040060 con 0x5645b445b9d0 2024-09-19T09:27:09.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d23fff640 1 -- 172.21.15.16:0/447485224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2d34014840 con 0x5645b445b9d0 2024-09-19T09:27:09.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d4cea6740 1 -- 172.21.15.16:0/447485224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d3809b7d0 con 0x5645b445b9d0 2024-09-19T09:27:09.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d23fff640 1 -- 172.21.15.16:0/447485224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d34021040 con 0x5645b445b9d0 2024-09-19T09:27:09.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d4cea6740 10 client.15123 fetch_fsmap learned FSMap version 27 2024-09-19T09:27:09.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d4cea6740 1 -- 172.21.15.16:0/447485224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5645b44e17b0 con 0x5645b445b9d0 2024-09-19T09:27:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d23fff640 1 -- 172.21.15.16:0/447485224 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3809b7d0 con 0x5645b445b9d0 2024-09-19T09:27:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d23fff640 1 -- 172.21.15.16:0/447485224 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2d34045070 con 0x5645b445b9d0 2024-09-19T09:27:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d4cea6740 10 client.15123 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:27:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d4cea6740 10 client.15123 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-19T09:27:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d4cea6740 20 client.15123 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/447485224 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x5645b44e6300 0x5645b44e5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d4cea6740 4 client.15123 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:27:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.228+0000 7f2d4cea6740 1 -- 172.21.15.16:0/447485224 --> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] -- command(tid 0: {"prefix": "status"}) -- 0x5645b43c9b10 con 0x5645b44e6300 2024-09-19T09:27:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d43fff640 1 --2- 172.21.15.16:0/447485224 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x5645b44e6300 0x5645b44e5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d23fff640 10 client.15123 ms_handle_connect on v2:172.21.15.97:6832/108652274 2024-09-19T09:27:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d43fff640 1 --2- 172.21.15.16:0/447485224 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x5645b44e6300 0x5645b44e5010 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1cdeed156bc2e7fb server_cookie=b60c1bf151cf703c in_seq=0 out_seq=0 2024-09-19T09:27:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d23fff640 1 -- 172.21.15.16:0/447485224 <== mds.3 v2:172.21.15.97:6832/108652274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5645b43c9b10 con 0x5645b44e6300 2024-09-19T09:27:09.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d23fff640 10 client.15123 handle_command_reply: tid=0 2024-09-19T09:27:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d4cea6740 1 client.15123 shutdown 2024-09-19T09:27:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d4cea6740 1 -- 172.21.15.16:0/447485224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 msgr2=0x5645b4480fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/447485224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b4480fa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400f7d0 tx=0x7f2d340274c0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d4cea6740 1 -- 172.21.15.16:0/447485224 shutdown_connections 2024-09-19T09:27:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/447485224 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x5645b44e6300 0x5645b44e5010 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d23fff640 0 client.15123 ms_handle_reset on v2:172.21.15.97:6832/108652274 2024-09-19T09:27:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/447485224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5645b445c2f0 0x5645b447c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/447485224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5645b4466bb0 0x5645b447c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d4cea6740 1 --2- 172.21.15.16:0/447485224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b445b9d0 0x5645b4480fa0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.229+0000 7f2d4cea6740 1 -- 172.21.15.16:0/447485224 >> 172.21.15.16:0/447485224 conn(0x5645b4457f60 msgr2=0x5645b4457000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4cea6740 1 -- 172.21.15.16:0/447485224 shutdown_connections 2024-09-19T09:27:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4cea6740 1 -- 172.21.15.16:0/447485224 wait complete. 2024-09-19T09:27:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4cea6740 20 client.15123 trim_cache size 0 max 16384 2024-09-19T09:27:09.239 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:27:09.239 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:27:09.239 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-09-19T09:27:09.239 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24428, 2024-09-19T09:27:09.239 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:27:09.239 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:27:09.239 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:27:09.239 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 90.271504641999996, 2024-09-19T09:27:09.239 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-19T09:27:09.240 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:27:09.240 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:27:09.240 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 137.109683158 2024-09-19T09:27:09.240 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:27:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4aacd640 1 -- 172.21.15.16:0/829706112 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2d2c0a3350 msgr2=0x7f2d2c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4aacd640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2d2c0a3350 0x7f2d2c0a5810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5645b44592e0 tx=0x7f2d3c005c30 comp rx=0 tx=0).stop 2024-09-19T09:27:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4aacd640 1 -- 172.21.15.16:0/829706112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d441027c0 msgr2=0x7f2d44198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4aacd640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d441027c0 0x7f2d44198940 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800b4a0 tx=0x7f2d3800b8a0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4aacd640 1 -- 172.21.15.16:0/829706112 shutdown_connections 2024-09-19T09:27:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4aacd640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2d2c0a3350 0x7f2d2c0a5810 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4aacd640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d44106f40 0x7f2d4419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4aacd640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d441027c0 0x7f2d44198940 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4aacd640 1 --2- 172.21.15.16:0/829706112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d44101e10 0x7f2d44198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4aacd640 1 -- 172.21.15.16:0/829706112 >> 172.21.15.16:0/829706112 conn(0x7f2d440fd8d0 msgr2=0x7f2d440fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4aacd640 1 -- 172.21.15.16:0/829706112 shutdown_connections 2024-09-19T09:27:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:09.230+0000 7f2d4aacd640 1 -- 172.21.15.16:0/829706112 wait complete. 2024-09-19T09:27:10.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T09:27:10.371 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.916+0000 7feab6410640 1 -- 172.21.15.16:0/31128083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab0102570 msgr2=0x7feab01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.916+0000 7feab6410640 1 --2- 172.21.15.16:0/31128083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab0102570 0x7feab01029b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009900 tx=0x7feaa402a480 comp rx=0 tx=0).stop 2024-09-19T09:27:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.916+0000 7feab6410640 1 -- 172.21.15.16:0/31128083 shutdown_connections 2024-09-19T09:27:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.916+0000 7feab6410640 1 --2- 172.21.15.16:0/31128083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab0102ef0 0x7feab010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.916+0000 7feab6410640 1 --2- 172.21.15.16:0/31128083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab0102570 0x7feab01029b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.916+0000 7feab6410640 1 --2- 172.21.15.16:0/31128083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0100e20 0x7feab0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.916+0000 7feab6410640 1 -- 172.21.15.16:0/31128083 >> 172.21.15.16:0/31128083 conn(0x7feab00fc7d0 msgr2=0x7feab00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.916+0000 7feab6410640 1 -- 172.21.15.16:0/31128083 shutdown_connections 2024-09-19T09:27:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.916+0000 7feab6410640 1 -- 172.21.15.16:0/31128083 wait complete. 2024-09-19T09:27:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.917+0000 7feab6410640 1 Processor -- start 2024-09-19T09:27:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.917+0000 7feab6410640 1 -- start start 2024-09-19T09:27:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.917+0000 7feab6410640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab0100e20 0x7feab01a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.917+0000 7feab6410640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0102570 0x7feab01a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.917+0000 7feab6410640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab0102ef0 0x7feab019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.917+0000 7feab6410640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feab0110f80 con 0x7feab0100e20 2024-09-19T09:27:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.917+0000 7feab6410640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feab0110e00 con 0x7feab0102ef0 2024-09-19T09:27:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.917+0000 7feab6410640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feab0111100 con 0x7feab0102570 2024-09-19T09:27:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.917+0000 7feaaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab0100e20 0x7feab01a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.917+0000 7feaaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab0100e20 0x7feab01a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53170/0 (socket says 172.21.15.16:53170) 2024-09-19T09:27:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.917+0000 7feaaffff640 1 -- 172.21.15.16:0/921106019 learned_addr learned my addr 172.21.15.16:0/921106019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feaaf7fe640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0102570 0x7feab01a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feab49ad640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab0102ef0 0x7feab019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feaaffff640 1 -- 172.21.15.16:0/921106019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0102570 msgr2=0x7feab01a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feaaffff640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0102570 0x7feab01a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feaaffff640 1 -- 172.21.15.16:0/921106019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab0102ef0 msgr2=0x7feab019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feaaffff640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab0102ef0 0x7feab019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feaaffff640 1 -- 172.21.15.16:0/921106019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab019acf0 con 0x7feab0100e20 2024-09-19T09:27:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feaaf7fe640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0102570 0x7feab01a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feab49ad640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab0102ef0 0x7feab019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feaaffff640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab0100e20 0x7feab01a0400 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feaa000e9f0 tx=0x7feaa000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feaad7fa640 1 -- 172.21.15.16:0/921106019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa000cd80 con 0x7feab0100e20 2024-09-19T09:27:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feaad7fa640 1 -- 172.21.15.16:0/921106019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa0005500 con 0x7feab0100e20 2024-09-19T09:27:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feab6410640 1 -- 172.21.15.16:0/921106019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab019afe0 con 0x7feab0100e20 2024-09-19T09:27:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feaad7fa640 1 -- 172.21.15.16:0/921106019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0010690 con 0x7feab0100e20 2024-09-19T09:27:10.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.918+0000 7feab6410640 1 -- 172.21.15.16:0/921106019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab01a72d0 con 0x7feab0100e20 2024-09-19T09:27:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.921+0000 7feaad7fa640 1 -- 172.21.15.16:0/921106019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7feaa00046c0 con 0x7feab0100e20 2024-09-19T09:27:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.921+0000 7feab87e9740 1 Processor -- start 2024-09-19T09:27:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.921+0000 7feab87e9740 1 -- start start 2024-09-19T09:27:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.921+0000 7feab87e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 0x559c45b2ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.921+0000 7feab87e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2e2f0 0x559c45b38670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.921+0000 7feab87e9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b3afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.921+0000 7feab87e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559c45a94090 con 0x559c45b2d9d0 2024-09-19T09:27:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.921+0000 7feab87e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559c45a3ba70 con 0x559c45b2e2f0 2024-09-19T09:27:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.921+0000 7feab87e9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559c45a90050 con 0x559c45b38bb0 2024-09-19T09:27:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.921+0000 7feab49ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 0x559c45b2ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.921+0000 7feab49ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 0x559c45b2ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53178/0 (socket says 172.21.15.16:53178) 2024-09-19T09:27:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.921+0000 7feab49ad640 1 -- 172.21.15.16:0/2796110725 learned_addr learned my addr 172.21.15.16:0/2796110725 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feaad7fa640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fea980a32d0 0x7fea980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feaaf7fe640 1 --2- 172.21.15.16:0/2796110725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b3afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feaaffff640 1 --2- 172.21.15.16:0/2796110725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2e2f0 0x559c45b38670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feab49ad640 1 -- 172.21.15.16:0/2796110725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 msgr2=0x559c45b3afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feab49ad640 1 --2- 172.21.15.16:0/2796110725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b3afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feab49ad640 1 -- 172.21.15.16:0/2796110725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2e2f0 msgr2=0x559c45b38670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feab49ad640 1 --2- 172.21.15.16:0/2796110725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2e2f0 0x559c45b38670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feab49ad640 1 -- 172.21.15.16:0/2796110725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c45a48130 con 0x559c45b2d9d0 2024-09-19T09:27:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feaad7fa640 1 -- 172.21.15.16:0/921106019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7feaa00d0000 con 0x7feab0100e20 2024-09-19T09:27:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feaaf7fe640 1 --2- 172.21.15.16:0/2796110725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b3afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feaaffff640 1 --2- 172.21.15.16:0/2796110725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2e2f0 0x559c45b38670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feab49ad640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fea980a32d0 0x7fea980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feab49ad640 1 --2- 172.21.15.16:0/2796110725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 0x559c45b2ddb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009fa0 tx=0x7feaa800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2706796f9adbdc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7fea967fc640 1 -- 172.21.15.16:0/2796110725 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa801a610 con 0x559c45b2d9d0 2024-09-19T09:27:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7fea967fc640 1 -- 172.21.15.16:0/2796110725 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa801ac90 con 0x559c45b2d9d0 2024-09-19T09:27:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7fea967fc640 1 -- 172.21.15.16:0/2796110725 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa800db80 con 0x559c45b2d9d0 2024-09-19T09:27:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.922+0000 7feab49ad640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fea980a32d0 0x7fea980a5790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feaa8020000 tx=0x7feaa801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.923+0000 7feab87e9740 1 -- 172.21.15.16:0/2796110725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 msgr2=0x559c45b2ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.923+0000 7feab87e9740 1 --2- 172.21.15.16:0/2796110725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 0x559c45b2ddb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009fa0 tx=0x7feaa800f7f0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.923+0000 7feab87e9740 1 -- 172.21.15.16:0/2796110725 shutdown_connections 2024-09-19T09:27:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.923+0000 7feab87e9740 1 --2- 172.21.15.16:0/2796110725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b3afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.923+0000 7feab87e9740 1 --2- 172.21.15.16:0/2796110725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2e2f0 0x559c45b38670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.923+0000 7feab87e9740 1 --2- 172.21.15.16:0/2796110725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 0x559c45b2ddb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.923+0000 7feab87e9740 1 -- 172.21.15.16:0/2796110725 >> 172.21.15.16:0/2796110725 conn(0x559c45b29f60 msgr2=0x559c45b2a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.923+0000 7feab87e9740 1 -- 172.21.15.16:0/2796110725 shutdown_connections 2024-09-19T09:27:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.923+0000 7feab87e9740 1 -- 172.21.15.16:0/2796110725 wait complete. 2024-09-19T09:27:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feab87e9740 1 -- start start 2024-09-19T09:27:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feab87e9740 1 Processor -- start 2024-09-19T09:27:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feab87e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 0x559c45b38390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feab87e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2e2f0 0x559c45b35730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feab87e9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b35c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feab87e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559c45b3b780 con 0x559c45b2d9d0 2024-09-19T09:27:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feab87e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559c45a4ef20 con 0x559c45b2e2f0 2024-09-19T09:27:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feab87e9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559c45b3b900 con 0x559c45b38bb0 2024-09-19T09:27:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feaaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 0x559c45b38390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feab87e9740 10 client.0 _ll_register_callbacks cb 0x559c45a18030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feaaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 0x559c45b38390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53192/0 (socket says 172.21.15.16:53192) 2024-09-19T09:27:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feaaffff640 1 -- 172.21.15.16:0/3700073558 learned_addr learned my addr 172.21.15.16:0/3700073558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feab49ad640 1 --2- 172.21.15.16:0/3700073558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b35c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.924+0000 7feaaf7fe640 1 --2- 172.21.15.16:0/3700073558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2e2f0 0x559c45b35730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7feaaffff640 1 -- 172.21.15.16:0/3700073558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 msgr2=0x559c45b35c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7feaaffff640 1 --2- 172.21.15.16:0/3700073558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b35c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7feaaffff640 1 -- 172.21.15.16:0/3700073558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2e2f0 msgr2=0x559c45b35730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7feaaffff640 1 --2- 172.21.15.16:0/3700073558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2e2f0 0x559c45b35730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7feaaffff640 1 -- 172.21.15.16:0/3700073558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c458052b0 con 0x559c45b2d9d0 2024-09-19T09:27:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7feaaffff640 1 --2- 172.21.15.16:0/3700073558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 0x559c45b38390 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7feaa00cd9e0 tx=0x7feaa006b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cba57cea8cdb48d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7fea8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7fea8f7fe640 1 -- 172.21.15.16:0/3700073558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa009cbc0 con 0x559c45b2d9d0 2024-09-19T09:27:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7fea8f7fe640 1 -- 172.21.15.16:0/3700073558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7feaa0016df0 con 0x559c45b2d9d0 2024-09-19T09:27:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7feab87e9740 1 -- 172.21.15.16:0/3700073558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaa801ac90 con 0x559c45b2d9d0 2024-09-19T09:27:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7fea8f7fe640 1 -- 172.21.15.16:0/3700073558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feaa009c4c0 con 0x559c45b2d9d0 2024-09-19T09:27:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7feab87e9740 10 client.15135 fetch_fsmap learned FSMap version 27 2024-09-19T09:27:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7feab87e9740 1 -- 172.21.15.16:0/3700073558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c45b46f90 con 0x559c45b2d9d0 2024-09-19T09:27:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7fea8f7fe640 1 -- 172.21.15.16:0/3700073558 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa801ac90 con 0x559c45b2d9d0 2024-09-19T09:27:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.925+0000 7fea8f7fe640 1 -- 172.21.15.16:0/3700073558 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feaa00d20e0 con 0x559c45b2d9d0 2024-09-19T09:27:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.926+0000 7feab87e9740 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.926+0000 7feab87e9740 10 client.15135 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.926+0000 7feab87e9740 20 client.15135 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.926+0000 7feab87e9740 1 --2- 172.21.15.16:0/3700073558 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x559c45b49550 0x559c45b4a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.926+0000 7feab87e9740 4 client.15135 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.926+0000 7feab87e9740 1 -- 172.21.15.16:0/3700073558 --> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c45aa1600 con 0x559c45b49550 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.926+0000 7feaaf7fe640 1 --2- 172.21.15.16:0/3700073558 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x559c45b49550 0x559c45b4a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.926+0000 7feaaf7fe640 1 --2- 172.21.15.16:0/3700073558 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x559c45b49550 0x559c45b4a9c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=151b0cfc71774dca server_cookie=3164d19cde68a459 in_seq=0 out_seq=0 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.926+0000 7fea8f7fe640 10 client.15135 ms_handle_connect on v2:172.21.15.97:6836/2018284957 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.927+0000 7fea8f7fe640 1 -- 172.21.15.16:0/3700073558 <== mds.2 v2:172.21.15.97:6836/2018284957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c45aa1600 con 0x559c45b49550 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.927+0000 7fea8f7fe640 10 client.15135 handle_command_reply: tid=0 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.927+0000 7feab87e9740 1 client.15135 shutdown 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.927+0000 7feab87e9740 1 -- 172.21.15.16:0/3700073558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 msgr2=0x559c45b38390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.927+0000 7feab87e9740 1 --2- 172.21.15.16:0/3700073558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 0x559c45b38390 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7feaa00cd9e0 tx=0x7feaa006b2c0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.927+0000 7feab87e9740 1 -- 172.21.15.16:0/3700073558 shutdown_connections 2024-09-19T09:27:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.927+0000 7feab87e9740 1 --2- 172.21.15.16:0/3700073558 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x559c45b49550 0x559c45b4a9c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.927+0000 7feab87e9740 1 --2- 172.21.15.16:0/3700073558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b35c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.927+0000 7feab87e9740 1 --2- 172.21.15.16:0/3700073558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2e2f0 0x559c45b35730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.927+0000 7feab87e9740 1 --2- 172.21.15.16:0/3700073558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2d9d0 0x559c45b38390 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.927+0000 7feab87e9740 1 -- 172.21.15.16:0/3700073558 >> 172.21.15.16:0/3700073558 conn(0x559c45b29f60 msgr2=0x559c45b2fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.927+0000 7fea8f7fe640 0 client.15135 ms_handle_reset on v2:172.21.15.97:6836/2018284957 2024-09-19T09:27:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.928+0000 7feab87e9740 1 -- 172.21.15.16:0/3700073558 shutdown_connections 2024-09-19T09:27:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.928+0000 7feab87e9740 1 -- 172.21.15.16:0/3700073558 wait complete. 2024-09-19T09:27:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.928+0000 7feab87e9740 20 client.15135 trim_cache size 0 max 16384 2024-09-19T09:27:10.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.945+0000 7feab87e9740 1 Processor -- start 2024-09-19T09:27:10.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.945+0000 7feab87e9740 1 -- start start 2024-09-19T09:27:10.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.945+0000 7feab87e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2d9d0 0x559c45b48e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.945+0000 7feab87e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2e2f0 0x559c45b4a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feab87e9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feab87e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559c45b48d00 con 0x559c45b2e2f0 2024-09-19T09:27:10.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feab87e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559c45b493d0 con 0x559c45b2d9d0 2024-09-19T09:27:10.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feab87e9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559c45b49aa0 con 0x559c45b38bb0 2024-09-19T09:27:10.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feaaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2e2f0 0x559c45b4a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feaaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2e2f0 0x559c45b4a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53198/0 (socket says 172.21.15.16:53198) 2024-09-19T09:27:10.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feaaf7fe640 1 -- 172.21.15.16:0/1966003409 learned_addr learned my addr 172.21.15.16:0/1966003409 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:10.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feab49ad640 1 --2- 172.21.15.16:0/1966003409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feaaffff640 1 --2- 172.21.15.16:0/1966003409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2d9d0 0x559c45b48e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feaaf7fe640 1 -- 172.21.15.16:0/1966003409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 msgr2=0x559c45b356f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feaaf7fe640 1 --2- 172.21.15.16:0/1966003409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b356f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feaaf7fe640 1 -- 172.21.15.16:0/1966003409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2d9d0 msgr2=0x559c45b48e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feaaf7fe640 1 --2- 172.21.15.16:0/1966003409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2d9d0 0x559c45b48e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feaaf7fe640 1 -- 172.21.15.16:0/1966003409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c45b29d90 con 0x559c45b2e2f0 2024-09-19T09:27:10.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feab49ad640 1 --2- 172.21.15.16:0/1966003409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b356f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:10.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7feaaf7fe640 1 --2- 172.21.15.16:0/1966003409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2e2f0 0x559c45b4a2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feaa4005940 tx=0x7feaa40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bea3d6ecf26605 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7fea95ffb640 1 -- 172.21.15.16:0/1966003409 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4035040 con 0x559c45b2e2f0 2024-09-19T09:27:10.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7fea95ffb640 1 -- 172.21.15.16:0/1966003409 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa4048070 con 0x559c45b2e2f0 2024-09-19T09:27:10.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.946+0000 7fea95ffb640 1 -- 172.21.15.16:0/1966003409 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4004d90 con 0x559c45b2e2f0 2024-09-19T09:27:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.947+0000 7feab87e9740 1 -- 172.21.15.16:0/1966003409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2e2f0 msgr2=0x559c45b4a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.947+0000 7feab87e9740 1 --2- 172.21.15.16:0/1966003409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2e2f0 0x559c45b4a2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feaa4005940 tx=0x7feaa40058d0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.947+0000 7feab87e9740 1 -- 172.21.15.16:0/1966003409 shutdown_connections 2024-09-19T09:27:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.947+0000 7feab87e9740 1 --2- 172.21.15.16:0/1966003409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b38bb0 0x559c45b356f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.947+0000 7feab87e9740 1 --2- 172.21.15.16:0/1966003409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b2e2f0 0x559c45b4a2f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.947+0000 7feab87e9740 1 --2- 172.21.15.16:0/1966003409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2d9d0 0x559c45b48e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.947+0000 7feab87e9740 1 -- 172.21.15.16:0/1966003409 >> 172.21.15.16:0/1966003409 conn(0x559c45b29f60 msgr2=0x559c45b2a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.947+0000 7feab87e9740 1 -- 172.21.15.16:0/1966003409 shutdown_connections 2024-09-19T09:27:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.947+0000 7feab87e9740 1 -- 172.21.15.16:0/1966003409 wait complete. 2024-09-19T09:27:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.947+0000 7feab87e9740 1 -- start start 2024-09-19T09:27:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab87e9740 1 Processor -- start 2024-09-19T09:27:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab87e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2d9d0 0x559c45b534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab87e9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b2e2f0 0x559c45b4dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab87e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b38bb0 0x559c45b4e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab87e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559c45b4a840 con 0x559c45b38bb0 2024-09-19T09:27:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab87e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559c45b4a170 con 0x559c45b2d9d0 2024-09-19T09:27:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab87e9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559c45b4af10 con 0x559c45b2e2f0 2024-09-19T09:27:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab87e9740 10 client.0 _ll_register_callbacks cb 0x7feaa00056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:10.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab49ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b38bb0 0x559c45b4e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab49ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b38bb0 0x559c45b4e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53212/0 (socket says 172.21.15.16:53212) 2024-09-19T09:27:10.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab49ad640 1 -- 172.21.15.16:0/3732166520 learned_addr learned my addr 172.21.15.16:0/3732166520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:10.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feaaffff640 1 --2- 172.21.15.16:0/3732166520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2d9d0 0x559c45b534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feaaf7fe640 1 --2- 172.21.15.16:0/3732166520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b2e2f0 0x559c45b4dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab49ad640 1 -- 172.21.15.16:0/3732166520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b2e2f0 msgr2=0x559c45b4dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab49ad640 1 --2- 172.21.15.16:0/3732166520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b2e2f0 0x559c45b4dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab49ad640 1 -- 172.21.15.16:0/3732166520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2d9d0 msgr2=0x559c45b534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:10.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab49ad640 1 --2- 172.21.15.16:0/3732166520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2d9d0 0x559c45b534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab49ad640 1 -- 172.21.15.16:0/3732166520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c45bb1c50 con 0x559c45b38bb0 2024-09-19T09:27:10.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7feab49ad640 1 --2- 172.21.15.16:0/3732166520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b38bb0 0x559c45b4e120 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7feaa800d7d0 tx=0x7feaa800d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6674e443723a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:10.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.948+0000 7fea8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:10.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7fea8f7fe640 1 -- 172.21.15.16:0/3732166520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa80094a0 con 0x559c45b38bb0 2024-09-19T09:27:10.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7fea8f7fe640 1 -- 172.21.15.16:0/3732166520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7feaa800d8f0 con 0x559c45b38bb0 2024-09-19T09:27:10.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7feab87e9740 1 -- 172.21.15.16:0/3732166520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaa4048070 con 0x559c45b38bb0 2024-09-19T09:27:10.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7fea8f7fe640 1 -- 172.21.15.16:0/3732166520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feaa800de60 con 0x559c45b38bb0 2024-09-19T09:27:10.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7feab87e9740 10 client.15141 fetch_fsmap learned FSMap version 27 2024-09-19T09:27:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7feab87e9740 1 -- 172.21.15.16:0/3732166520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c45bb2d40 con 0x559c45b38bb0 2024-09-19T09:27:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7fea8f7fe640 1 -- 172.21.15.16:0/3732166520 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa4048070 con 0x559c45b38bb0 2024-09-19T09:27:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7fea8f7fe640 1 -- 172.21.15.16:0/3732166520 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feaa802e070 con 0x559c45b38bb0 2024-09-19T09:27:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7feab87e9740 10 client.15141 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:27:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7feab87e9740 10 client.15141 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:27:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7feab87e9740 20 client.15141 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7feab87e9740 1 --2- 172.21.15.16:0/3732166520 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x559c45bb7640 0x559c45bb6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7feab87e9740 4 client.15141 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:27:10.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.949+0000 7feab87e9740 1 -- 172.21.15.16:0/3732166520 --> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] -- command(tid 0: {"prefix": "status"}) -- 0x559c45a9bb10 con 0x559c45bb7640 2024-09-19T09:27:10.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.950+0000 7feaaffff640 1 --2- 172.21.15.16:0/3732166520 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x559c45bb7640 0x559c45bb6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:10.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.950+0000 7feaaffff640 1 --2- 172.21.15.16:0/3732166520 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x559c45bb7640 0x559c45bb6350 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f904fb578825396 server_cookie=d89922f9f92d78ee in_seq=0 out_seq=0 2024-09-19T09:27:10.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.950+0000 7fea8f7fe640 10 client.15141 ms_handle_connect on v2:172.21.15.97:6836/2018284957 2024-09-19T09:27:10.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.950+0000 7fea8f7fe640 1 -- 172.21.15.16:0/3732166520 <== mds.2 v2:172.21.15.97:6836/2018284957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559c45a9bb10 con 0x559c45bb7640 2024-09-19T09:27:10.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.950+0000 7fea8f7fe640 10 client.15141 handle_command_reply: tid=0 2024-09-19T09:27:10.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.950+0000 7feab87e9740 1 client.15141 shutdown 2024-09-19T09:27:10.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.950+0000 7feab87e9740 1 -- 172.21.15.16:0/3732166520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b38bb0 msgr2=0x559c45b4e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.950+0000 7feab87e9740 1 --2- 172.21.15.16:0/3732166520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b38bb0 0x559c45b4e120 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7feaa800d7d0 tx=0x7feaa800d650 comp rx=0 tx=0).stop 2024-09-19T09:27:10.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.950+0000 7feab87e9740 1 -- 172.21.15.16:0/3732166520 shutdown_connections 2024-09-19T09:27:10.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab87e9740 1 --2- 172.21.15.16:0/3732166520 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x559c45bb7640 0x559c45bb6350 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab87e9740 1 --2- 172.21.15.16:0/3732166520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559c45b2e2f0 0x559c45b4dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab87e9740 1 --2- 172.21.15.16:0/3732166520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c45b38bb0 0x559c45b4e120 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab87e9740 1 --2- 172.21.15.16:0/3732166520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c45b2d9d0 0x559c45b534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab87e9740 1 -- 172.21.15.16:0/3732166520 >> 172.21.15.16:0/3732166520 conn(0x559c45b29f60 msgr2=0x559c45b2a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7fea8f7fe640 0 client.15141 ms_handle_reset on v2:172.21.15.97:6836/2018284957 2024-09-19T09:27:10.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab87e9740 1 -- 172.21.15.16:0/3732166520 shutdown_connections 2024-09-19T09:27:10.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab87e9740 1 -- 172.21.15.16:0/3732166520 wait complete. 2024-09-19T09:27:10.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab87e9740 20 client.15141 trim_cache size 0 max 16384 2024-09-19T09:27:10.960 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:27:10.960 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:27:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-19T09:27:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24470, 2024-09-19T09:27:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:27:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:27:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:27:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 92.624823829999997, 2024-09-19T09:27:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-19T09:27:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:27:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:27:10.961 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 122.474203976 2024-09-19T09:27:10.961 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:27:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab6410640 1 -- 172.21.15.16:0/921106019 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fea980a32d0 msgr2=0x7fea980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab6410640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fea980a32d0 0x7fea980a5790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feaa8020000 tx=0x7feaa801d000 comp rx=0 tx=0).stop 2024-09-19T09:27:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab6410640 1 -- 172.21.15.16:0/921106019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab0100e20 msgr2=0x7feab01a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab6410640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab0100e20 0x7feab01a0400 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feaa000e9f0 tx=0x7feaa000edf0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab6410640 1 -- 172.21.15.16:0/921106019 shutdown_connections 2024-09-19T09:27:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab6410640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fea980a32d0 0x7fea980a5790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab6410640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feab0102570 0x7feab01a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab6410640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feab0102ef0 0x7feab019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab6410640 1 --2- 172.21.15.16:0/921106019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab0100e20 0x7feab01a0400 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab6410640 1 -- 172.21.15.16:0/921106019 >> 172.21.15.16:0/921106019 conn(0x7feab00fc7d0 msgr2=0x7feab006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.951+0000 7feab6410640 1 -- 172.21.15.16:0/921106019 shutdown_connections 2024-09-19T09:27:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:10.952+0000 7feab6410640 1 -- 172.21.15.16:0/921106019 wait complete. 2024-09-19T09:27:11.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:10 smithi142 ceph-mon[35032]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:27:11.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:10 smithi097 ceph-mon[34864]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:27:11.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:10 smithi016 ceph-mon[30979]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T09:27:11.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T09:27:12.000 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.486+0000 7f4608906640 1 -- 172.21.15.16:0/372357161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46041036e0 msgr2=0x7f4604107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.486+0000 7f4608906640 1 --2- 172.21.15.16:0/372357161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46041036e0 0x7f4604107b90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009930 tx=0x7f45ec02a420 comp rx=0 tx=0).stop 2024-09-19T09:27:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.486+0000 7f4608906640 1 -- 172.21.15.16:0/372357161 shutdown_connections 2024-09-19T09:27:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.486+0000 7f4608906640 1 --2- 172.21.15.16:0/372357161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46041082a0 0x7f460410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.486+0000 7f4608906640 1 --2- 172.21.15.16:0/372357161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46041036e0 0x7f4604107b90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.486+0000 7f4608906640 1 --2- 172.21.15.16:0/372357161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4604102d30 0x7f4604103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.486+0000 7f4608906640 1 -- 172.21.15.16:0/372357161 >> 172.21.15.16:0/372357161 conn(0x7f46040fc7d0 msgr2=0x7f46040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.486+0000 7f4608906640 1 -- 172.21.15.16:0/372357161 shutdown_connections 2024-09-19T09:27:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.486+0000 7f4608906640 1 -- 172.21.15.16:0/372357161 wait complete. 2024-09-19T09:27:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4608906640 1 Processor -- start 2024-09-19T09:27:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4608906640 1 -- start start 2024-09-19T09:27:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4608906640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4604102d30 0x7f4604198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4608906640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46041036e0 0x7f4604198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4608906640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46041082a0 0x7f460419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4608906640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4604112090 con 0x7f46041082a0 2024-09-19T09:27:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4608906640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4604111f10 con 0x7f46041036e0 2024-09-19T09:27:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4608906640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4604112210 con 0x7f4604102d30 2024-09-19T09:27:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4602d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46041082a0 0x7f460419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4602d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46041082a0 0x7f460419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53228/0 (socket says 172.21.15.16:53228) 2024-09-19T09:27:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4602d9d640 1 -- 172.21.15.16:0/2357555259 learned_addr learned my addr 172.21.15.16:0/2357555259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f460259c640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4604102d30 0x7f4604198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f45fbfff640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46041036e0 0x7f4604198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4602d9d640 1 -- 172.21.15.16:0/2357555259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4604102d30 msgr2=0x7f4604198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4602d9d640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4604102d30 0x7f4604198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4602d9d640 1 -- 172.21.15.16:0/2357555259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46041036e0 msgr2=0x7f4604198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4602d9d640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46041036e0 0x7f4604198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.487+0000 7f4602d9d640 1 -- 172.21.15.16:0/2357555259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460419d540 con 0x7f46041082a0 2024-09-19T09:27:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.488+0000 7f4602d9d640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46041082a0 0x7f460419ce60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00eaa0 tx=0x7f45fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.488+0000 7f45fb7fe640 1 -- 172.21.15.16:0/2357555259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc00cdb0 con 0x7f46041082a0 2024-09-19T09:27:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.488+0000 7f4608906640 1 -- 172.21.15.16:0/2357555259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460419d830 con 0x7f46041082a0 2024-09-19T09:27:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.488+0000 7f45fb7fe640 1 -- 172.21.15.16:0/2357555259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45fc005020 con 0x7f46041082a0 2024-09-19T09:27:12.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.488+0000 7f45fb7fe640 1 -- 172.21.15.16:0/2357555259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc0116b0 con 0x7f46041082a0 2024-09-19T09:27:12.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.488+0000 7f4608906640 1 -- 172.21.15.16:0/2357555259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46041a5110 con 0x7f46041082a0 2024-09-19T09:27:12.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.490+0000 7f45fb7fe640 1 -- 172.21.15.16:0/2357555259 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f45fc005560 con 0x7f46041082a0 2024-09-19T09:27:12.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.491+0000 7f460acdf740 1 Processor -- start 2024-09-19T09:27:12.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.491+0000 7f460acdf740 1 -- start start 2024-09-19T09:27:12.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.491+0000 7f460acdf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 0x55edadde8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.491+0000 7f460acdf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 0x55edaddf3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.491+0000 7f460acdf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 0x55edaddf5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.491+0000 7f460acdf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55edadd4f090 con 0x55edadde89d0 2024-09-19T09:27:12.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.491+0000 7f460acdf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55edadcf6a70 con 0x55edaddf3bb0 2024-09-19T09:27:12.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.491+0000 7f460acdf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55edadd4b050 con 0x55edadde92f0 2024-09-19T09:27:12.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.491+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 0x55edadde8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.491+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 0x55edadde8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53238/0 (socket says 172.21.15.16:53238) 2024-09-19T09:27:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.491+0000 7f45fbfff640 1 -- 172.21.15.16:0/3483476778 learned_addr learned my addr 172.21.15.16:0/3483476778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.491+0000 7f460259c640 1 --2- 172.21.15.16:0/3483476778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 0x55edaddf3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.492+0000 7f45fbfff640 1 --2- 172.21.15.16:0/3483476778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 0x55edaddf5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.492+0000 7f45fb7fe640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f45f00a3240 0x7f45f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.492+0000 7f45fbfff640 1 -- 172.21.15.16:0/3483476778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 msgr2=0x55edaddf3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.492+0000 7f45fbfff640 1 --2- 172.21.15.16:0/3483476778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 0x55edaddf3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.492+0000 7f45fbfff640 1 -- 172.21.15.16:0/3483476778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 msgr2=0x55edaddf5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.492+0000 7f45fbfff640 1 --2- 172.21.15.16:0/3483476778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 0x55edaddf5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.492+0000 7f45fbfff640 1 -- 172.21.15.16:0/3483476778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edadd03130 con 0x55edadde89d0 2024-09-19T09:27:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.492+0000 7f4602d9d640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f45f00a3240 0x7f45f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.492+0000 7f45fbfff640 1 --2- 172.21.15.16:0/3483476778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 0x55edadde8db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009900 tx=0x7f45ec005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e1bc7ab6b47930 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.492+0000 7f45fb7fe640 1 -- 172.21.15.16:0/2357555259 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f45fc0cfb40 con 0x7f46041082a0 2024-09-19T09:27:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.492+0000 7f45f8ff9640 1 -- 172.21.15.16:0/3483476778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec035070 con 0x55edadde89d0 2024-09-19T09:27:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.492+0000 7f45f8ff9640 1 -- 172.21.15.16:0/3483476778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ec005520 con 0x55edadde89d0 2024-09-19T09:27:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.492+0000 7f45f8ff9640 1 -- 172.21.15.16:0/3483476778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec003850 con 0x55edadde89d0 2024-09-19T09:27:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.493+0000 7f460acdf740 1 -- 172.21.15.16:0/3483476778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 msgr2=0x55edadde8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.493+0000 7f4602d9d640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f45f00a3240 0x7f45f00a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f45fc098580 tx=0x7f45fc066b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.493+0000 7f460acdf740 1 --2- 172.21.15.16:0/3483476778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 0x55edadde8db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009900 tx=0x7f45ec005be0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.493+0000 7f460acdf740 1 -- 172.21.15.16:0/3483476778 shutdown_connections 2024-09-19T09:27:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.493+0000 7f460acdf740 1 --2- 172.21.15.16:0/3483476778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 0x55edaddf3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.493+0000 7f460acdf740 1 --2- 172.21.15.16:0/3483476778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 0x55edaddf5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.493+0000 7f460acdf740 1 --2- 172.21.15.16:0/3483476778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 0x55edadde8db0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.493+0000 7f460acdf740 1 -- 172.21.15.16:0/3483476778 >> 172.21.15.16:0/3483476778 conn(0x55edadde4f60 msgr2=0x55edadde5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.493+0000 7f460acdf740 1 -- 172.21.15.16:0/3483476778 shutdown_connections 2024-09-19T09:27:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.493+0000 7f460acdf740 1 -- 172.21.15.16:0/3483476778 wait complete. 2024-09-19T09:27:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f460acdf740 1 -- start start 2024-09-19T09:27:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f460acdf740 1 Processor -- start 2024-09-19T09:27:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f460acdf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edadde89d0 0x55edaddf34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f460acdf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 0x55edaddf07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f460acdf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edaddf3bb0 0x55edaddf0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f460acdf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55edaddf6780 con 0x55edaddf3bb0 2024-09-19T09:27:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f460acdf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55edadd09f20 con 0x55edadde89d0 2024-09-19T09:27:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f460acdf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55edaddf6900 con 0x55edadde92f0 2024-09-19T09:27:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f460acdf740 10 client.0 _ll_register_callbacks cb 0x55edadcd3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edaddf3bb0 0x55edaddf0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edaddf3bb0 0x55edaddf0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53252/0 (socket says 172.21.15.16:53252) 2024-09-19T09:27:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f45fbfff640 1 -- 172.21.15.16:0/1016694667 learned_addr learned my addr 172.21.15.16:0/1016694667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f460259c640 1 --2- 172.21.15.16:0/1016694667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 0x55edaddf07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f45fbfff640 1 --2- 172.21.15.16:0/1016694667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edadde89d0 0x55edaddf34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f45fbfff640 1 -- 172.21.15.16:0/1016694667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 msgr2=0x55edaddf07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f45fbfff640 1 --2- 172.21.15.16:0/1016694667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 0x55edaddf07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f45fbfff640 1 -- 172.21.15.16:0/1016694667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edadde89d0 msgr2=0x55edaddf34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f45fbfff640 1 --2- 172.21.15.16:0/1016694667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edadde89d0 0x55edaddf34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.494+0000 7f45fbfff640 1 -- 172.21.15.16:0/1016694667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edadac02b0 con 0x55edaddf3bb0 2024-09-19T09:27:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f45fbfff640 1 --2- 172.21.15.16:0/1016694667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edaddf3bb0 0x55edaddf0d30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f45ec000c00 tx=0x7f45ec004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1565806722d553 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f45e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f45e9ffb640 1 -- 172.21.15.16:0/1016694667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec035070 con 0x55edaddf3bb0 2024-09-19T09:27:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f45e9ffb640 1 -- 172.21.15.16:0/1016694667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f45ec0048c0 con 0x55edaddf3bb0 2024-09-19T09:27:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f460acdf740 1 -- 172.21.15.16:0/1016694667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45ec005520 con 0x55edaddf3bb0 2024-09-19T09:27:12.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f45e9ffb640 1 -- 172.21.15.16:0/1016694667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f45ec004ac0 con 0x55edaddf3bb0 2024-09-19T09:27:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f460acdf740 10 client.15153 fetch_fsmap learned FSMap version 27 2024-09-19T09:27:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f460acdf740 1 -- 172.21.15.16:0/1016694667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55edade020f0 con 0x55edaddf3bb0 2024-09-19T09:27:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f45e9ffb640 1 -- 172.21.15.16:0/1016694667 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ec005520 con 0x55edaddf3bb0 2024-09-19T09:27:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f45e9ffb640 1 -- 172.21.15.16:0/1016694667 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f45ec0039c0 con 0x55edaddf3bb0 2024-09-19T09:27:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f460acdf740 10 client.15153 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:27:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f460acdf740 10 client.15153 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:27:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f460acdf740 20 client.15153 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f460acdf740 1 --2- 172.21.15.16:0/1016694667 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55edade03440 0x55edade055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f460acdf740 4 client.15153 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.495+0000 7f460acdf740 1 -- 172.21.15.16:0/1016694667 --> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edadd5c600 con 0x55edade03440 2024-09-19T09:27:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.496+0000 7f460259c640 1 --2- 172.21.15.16:0/1016694667 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55edade03440 0x55edade055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.496+0000 7f45e9ffb640 10 client.15153 ms_handle_connect on v2:172.21.15.142:6838/3822134597 2024-09-19T09:27:12.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.496+0000 7f460259c640 1 --2- 172.21.15.16:0/1016694667 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55edade03440 0x55edade055f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f32e814d179f92d6 server_cookie=60a4968ea8a01c11 in_seq=0 out_seq=0 2024-09-19T09:27:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f45e9ffb640 1 -- 172.21.15.16:0/1016694667 <== mds.1 v2:172.21.15.142:6838/3822134597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edadd5c600 con 0x55edade03440 2024-09-19T09:27:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f45e9ffb640 10 client.15153 handle_command_reply: tid=0 2024-09-19T09:27:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f460acdf740 1 client.15153 shutdown 2024-09-19T09:27:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f460acdf740 1 -- 172.21.15.16:0/1016694667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edaddf3bb0 msgr2=0x55edaddf0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f460acdf740 1 --2- 172.21.15.16:0/1016694667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edaddf3bb0 0x55edaddf0d30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f45ec000c00 tx=0x7f45ec004c90 comp rx=0 tx=0).stop 2024-09-19T09:27:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f460acdf740 1 -- 172.21.15.16:0/1016694667 shutdown_connections 2024-09-19T09:27:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f460acdf740 1 --2- 172.21.15.16:0/1016694667 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55edade03440 0x55edade055f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f460acdf740 1 --2- 172.21.15.16:0/1016694667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 0x55edaddf07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f460acdf740 1 --2- 172.21.15.16:0/1016694667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edaddf3bb0 0x55edaddf0d30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f460acdf740 1 --2- 172.21.15.16:0/1016694667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edadde89d0 0x55edaddf34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f460acdf740 1 -- 172.21.15.16:0/1016694667 >> 172.21.15.16:0/1016694667 conn(0x55edadde4f60 msgr2=0x55edaddeac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f45e9ffb640 0 client.15153 ms_handle_reset on v2:172.21.15.142:6838/3822134597 2024-09-19T09:27:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f460acdf740 1 -- 172.21.15.16:0/1016694667 shutdown_connections 2024-09-19T09:27:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f460acdf740 1 -- 172.21.15.16:0/1016694667 wait complete. 2024-09-19T09:27:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.497+0000 7f460acdf740 20 client.15153 trim_cache size 0 max 16384 2024-09-19T09:27:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.515+0000 7f460acdf740 1 Processor -- start 2024-09-19T09:27:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.515+0000 7f460acdf740 1 -- start start 2024-09-19T09:27:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.515+0000 7f460acdf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 0x55edade055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.515+0000 7f460acdf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 0x55edaddf00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.515+0000 7f460acdf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 0x55edade0dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.515+0000 7f460acdf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55edade03930 con 0x55edadde89d0 2024-09-19T09:27:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.515+0000 7f460acdf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55edade04000 con 0x55edaddf3bb0 2024-09-19T09:27:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.515+0000 7f460acdf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55edade046d0 con 0x55edadde92f0 2024-09-19T09:27:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.516+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 0x55edade055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.516+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 0x55edade055f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53254/0 (socket says 172.21.15.16:53254) 2024-09-19T09:27:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.516+0000 7f45fbfff640 1 -- 172.21.15.16:0/4077803196 learned_addr learned my addr 172.21.15.16:0/4077803196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.516+0000 7f460259c640 1 --2- 172.21.15.16:0/4077803196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 0x55edaddf00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.516+0000 7f45fbfff640 1 -- 172.21.15.16:0/4077803196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 msgr2=0x55edaddf00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.516+0000 7f45fbfff640 1 --2- 172.21.15.16:0/4077803196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 0x55edaddf00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.516+0000 7f45fbfff640 1 -- 172.21.15.16:0/4077803196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 msgr2=0x55edade0dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.516+0000 7f45fbfff640 1 --2- 172.21.15.16:0/4077803196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 0x55edade0dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.516+0000 7f45fbfff640 1 -- 172.21.15.16:0/4077803196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edaddf30c0 con 0x55edadde89d0 2024-09-19T09:27:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.516+0000 7f45fbfff640 1 --2- 172.21.15.16:0/4077803196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 0x55edade055f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f45ec005c00 tx=0x7f45ec009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bafc1e97143a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.516+0000 7f45ebfff640 1 -- 172.21.15.16:0/4077803196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec035070 con 0x55edadde89d0 2024-09-19T09:27:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.516+0000 7f45ebfff640 1 -- 172.21.15.16:0/4077803196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ec009780 con 0x55edadde89d0 2024-09-19T09:27:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.516+0000 7f45ebfff640 1 -- 172.21.15.16:0/4077803196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec03d6b0 con 0x55edadde89d0 2024-09-19T09:27:12.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.517+0000 7f460acdf740 1 -- 172.21.15.16:0/4077803196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 msgr2=0x55edade055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.517+0000 7f460acdf740 1 --2- 172.21.15.16:0/4077803196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 0x55edade055f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f45ec005c00 tx=0x7f45ec009650 comp rx=0 tx=0).stop 2024-09-19T09:27:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.517+0000 7f460acdf740 1 -- 172.21.15.16:0/4077803196 shutdown_connections 2024-09-19T09:27:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.517+0000 7f460acdf740 1 --2- 172.21.15.16:0/4077803196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde92f0 0x55edaddf00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.517+0000 7f460acdf740 1 --2- 172.21.15.16:0/4077803196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 0x55edade0dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.517+0000 7f460acdf740 1 --2- 172.21.15.16:0/4077803196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde89d0 0x55edade055f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.517+0000 7f460acdf740 1 -- 172.21.15.16:0/4077803196 >> 172.21.15.16:0/4077803196 conn(0x55edadde4f60 msgr2=0x55edaddef0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.517+0000 7f460acdf740 1 -- 172.21.15.16:0/4077803196 shutdown_connections 2024-09-19T09:27:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.517+0000 7f460acdf740 1 -- 172.21.15.16:0/4077803196 wait complete. 2024-09-19T09:27:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f460acdf740 1 -- start start 2024-09-19T09:27:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f460acdf740 1 Processor -- start 2024-09-19T09:27:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f460acdf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde89d0 0x55edade07e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f460acdf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde92f0 0x55edaddeb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f460acdf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 0x55edaddeb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f460acdf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55edade05470 con 0x55edadde92f0 2024-09-19T09:27:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f460acdf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55edade04da0 con 0x55edaddf3bb0 2024-09-19T09:27:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f460acdf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55edade05b40 con 0x55edadde89d0 2024-09-19T09:27:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f460acdf740 10 client.0 _ll_register_callbacks cb 0x7f45f400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde89d0 0x55edade07e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f460259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde92f0 0x55edaddeb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f45fbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde89d0 0x55edade07e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:45942/0 (socket says 172.21.15.16:45942) 2024-09-19T09:27:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f45fbfff640 1 -- 172.21.15.16:0/389331313 learned_addr learned my addr 172.21.15.16:0/389331313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.518+0000 7f45fbfff640 1 --2- 172.21.15.16:0/389331313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 0x55edaddeb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f45fbfff640 1 -- 172.21.15.16:0/389331313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 msgr2=0x55edaddeb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f45fbfff640 1 --2- 172.21.15.16:0/389331313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 0x55edaddeb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f45fbfff640 1 -- 172.21.15.16:0/389331313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde92f0 msgr2=0x55edaddeb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f45fbfff640 1 --2- 172.21.15.16:0/389331313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde92f0 0x55edaddeb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f45fbfff640 1 -- 172.21.15.16:0/389331313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edaddebcd0 con 0x55edadde89d0 2024-09-19T09:27:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f460259c640 1 --2- 172.21.15.16:0/389331313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde92f0 0x55edaddeb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f45fbfff640 1 --2- 172.21.15.16:0/389331313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde89d0 0x55edade07e40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f45ec001eb0 tx=0x7f45ec031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e74b0a1441e768c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f45e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-19T09:27:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f45e9ffb640 1 -- 172.21.15.16:0/389331313 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec035040 con 0x55edadde89d0 2024-09-19T09:27:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f45e9ffb640 1 -- 172.21.15.16:0/389331313 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f45ec0037f0 con 0x55edadde89d0 2024-09-19T09:27:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f460acdf740 1 -- 172.21.15.16:0/389331313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45ec009780 con 0x55edadde89d0 2024-09-19T09:27:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f45e9ffb640 1 -- 172.21.15.16:0/389331313 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f45ec0039f0 con 0x55edadde89d0 2024-09-19T09:27:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f460acdf740 10 client.24847 fetch_fsmap learned FSMap version 27 2024-09-19T09:27:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.519+0000 7f460acdf740 1 -- 172.21.15.16:0/389331313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55edade72590 con 0x55edadde89d0 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.520+0000 7f45e9ffb640 1 -- 172.21.15.16:0/389331313 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ec009780 con 0x55edadde89d0 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.520+0000 7f45e9ffb640 1 -- 172.21.15.16:0/389331313 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f45ec03a440 con 0x55edadde89d0 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.520+0000 7f460acdf740 10 client.24847 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.520+0000 7f460acdf740 10 client.24847 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.520+0000 7f460acdf740 20 client.24847 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.520+0000 7f460acdf740 1 --2- 172.21.15.16:0/389331313 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55edade76aa0 0x55edade757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.520+0000 7f460acdf740 4 client.24847 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.520+0000 7f460acdf740 1 -- 172.21.15.16:0/389331313 --> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] -- command(tid 0: {"prefix": "status"}) -- 0x55edadd56b10 con 0x55edade76aa0 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.521+0000 7f460259c640 1 --2- 172.21.15.16:0/389331313 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55edade76aa0 0x55edade757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.521+0000 7f460259c640 1 --2- 172.21.15.16:0/389331313 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55edade76aa0 0x55edade757b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9a37b8d117e4d7d server_cookie=4474162c30e83988 in_seq=0 out_seq=0 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.521+0000 7f45e9ffb640 10 client.24847 ms_handle_connect on v2:172.21.15.142:6838/3822134597 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.521+0000 7f45e9ffb640 1 -- 172.21.15.16:0/389331313 <== mds.1 v2:172.21.15.142:6838/3822134597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55edadd56b10 con 0x55edade76aa0 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.521+0000 7f45e9ffb640 10 client.24847 handle_command_reply: tid=0 2024-09-19T09:27:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.521+0000 7f460acdf740 1 client.24847 shutdown 2024-09-19T09:27:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f460acdf740 1 -- 172.21.15.16:0/389331313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde89d0 msgr2=0x55edade07e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f460acdf740 1 --2- 172.21.15.16:0/389331313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde89d0 0x55edade07e40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f45ec001eb0 tx=0x7f45ec031400 comp rx=0 tx=0).stop 2024-09-19T09:27:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f460acdf740 1 -- 172.21.15.16:0/389331313 shutdown_connections 2024-09-19T09:27:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f460acdf740 1 --2- 172.21.15.16:0/389331313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55edaddf3bb0 0x55edaddeb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f460acdf740 1 --2- 172.21.15.16:0/389331313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55edadde92f0 0x55edaddeb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f460acdf740 1 --2- 172.21.15.16:0/389331313 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55edade76aa0 0x55edade757b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f460acdf740 1 --2- 172.21.15.16:0/389331313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55edadde89d0 0x55edade07e40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f460acdf740 1 -- 172.21.15.16:0/389331313 >> 172.21.15.16:0/389331313 conn(0x55edadde4f60 msgr2=0x55edadde37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f45e9ffb640 0 client.24847 ms_handle_reset on v2:172.21.15.142:6838/3822134597 2024-09-19T09:27:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f460acdf740 1 -- 172.21.15.16:0/389331313 shutdown_connections 2024-09-19T09:27:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f460acdf740 1 -- 172.21.15.16:0/389331313 wait complete. 2024-09-19T09:27:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f460acdf740 20 client.24847 trim_cache size 0 max 16384 2024-09-19T09:27:12.529 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:27:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:27:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-19T09:27:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24484, 2024-09-19T09:27:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:27:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:27:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:27:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 95.203902424999995, 2024-09-19T09:27:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-19T09:27:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:27:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:27:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 121.551495515 2024-09-19T09:27:12.530 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:27:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f4608906640 1 -- 172.21.15.16:0/2357555259 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f45f00a3240 msgr2=0x7f45f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f4608906640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f45f00a3240 0x7f45f00a5700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f45fc098580 tx=0x7f45fc066b60 comp rx=0 tx=0).stop 2024-09-19T09:27:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f4608906640 1 -- 172.21.15.16:0/2357555259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46041082a0 msgr2=0x7f460419ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.522+0000 7f4608906640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46041082a0 0x7f460419ce60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00eaa0 tx=0x7f45fc00ef60 comp rx=0 tx=0).stop 2024-09-19T09:27:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.523+0000 7f4608906640 1 -- 172.21.15.16:0/2357555259 shutdown_connections 2024-09-19T09:27:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.523+0000 7f4608906640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46041082a0 0x7f460419ce60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.523+0000 7f4608906640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46041036e0 0x7f4604198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.523+0000 7f4608906640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f45f00a3240 0x7f45f00a5700 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.523+0000 7f4608906640 1 --2- 172.21.15.16:0/2357555259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4604102d30 0x7f4604198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.523+0000 7f4608906640 1 -- 172.21.15.16:0/2357555259 >> 172.21.15.16:0/2357555259 conn(0x7f46040fc7d0 msgr2=0x7f4604106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.523+0000 7f4608906640 1 -- 172.21.15.16:0/2357555259 shutdown_connections 2024-09-19T09:27:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:12.523+0000 7f4608906640 1 -- 172.21.15.16:0/2357555259 wait complete. 2024-09-19T09:27:13.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:12 smithi142 ceph-mon[35032]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:27:13.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:12 smithi142 ceph-mon[35032]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T09:27:13.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:12 smithi097 ceph-mon[34864]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:27:13.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:12 smithi097 ceph-mon[34864]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T09:27:13.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:12 smithi016 ceph-mon[30979]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:27:13.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:12 smithi016 ceph-mon[30979]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T09:27:13.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T09:27:13.632 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.062+0000 7f73c1268640 1 -- 172.21.15.16:0/927714165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73bc102790 msgr2=0x7f73bc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.062+0000 7f73c1268640 1 --2- 172.21.15.16:0/927714165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73bc102790 0x7f73bc1067e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009900 tx=0x7f73a402a410 comp rx=0 tx=0).stop 2024-09-19T09:27:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.063+0000 7f73c1268640 1 -- 172.21.15.16:0/927714165 shutdown_connections 2024-09-19T09:27:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.063+0000 7f73c1268640 1 --2- 172.21.15.16:0/927714165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73bc106f10 0x7f73bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.063+0000 7f73c1268640 1 --2- 172.21.15.16:0/927714165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73bc102790 0x7f73bc1067e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.063+0000 7f73c1268640 1 --2- 172.21.15.16:0/927714165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73bc101de0 0x7f73bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.063+0000 7f73c1268640 1 -- 172.21.15.16:0/927714165 >> 172.21.15.16:0/927714165 conn(0x7f73bc0fd8c0 msgr2=0x7f73bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.063+0000 7f73c1268640 1 -- 172.21.15.16:0/927714165 shutdown_connections 2024-09-19T09:27:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.063+0000 7f73c1268640 1 -- 172.21.15.16:0/927714165 wait complete. 2024-09-19T09:27:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.063+0000 7f73c1268640 1 Processor -- start 2024-09-19T09:27:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.063+0000 7f73c1268640 1 -- start start 2024-09-19T09:27:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.063+0000 7f73c1268640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73bc101de0 0x7f73bc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73c1268640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73bc102790 0x7f73bc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73c1268640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73bc106f10 0x7f73bc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73c1268640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc111f90 con 0x7f73bc102790 2024-09-19T09:27:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73c1268640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc111e10 con 0x7f73bc106f10 2024-09-19T09:27:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73c1268640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc112110 con 0x7f73bc101de0 2024-09-19T09:27:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73bc102790 0x7f73bc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73bc102790 0x7f73bc198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53292/0 (socket says 172.21.15.16:53292) 2024-09-19T09:27:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73b3fff640 1 -- 172.21.15.16:0/2230184475 learned_addr learned my addr 172.21.15.16:0/2230184475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73bb7fe640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73bc106f10 0x7f73bc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73baffd640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73bc101de0 0x7f73bc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73b3fff640 1 -- 172.21.15.16:0/2230184475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73bc101de0 msgr2=0x7f73bc198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73b3fff640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73bc101de0 0x7f73bc198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73b3fff640 1 -- 172.21.15.16:0/2230184475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73bc106f10 msgr2=0x7f73bc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73b3fff640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73bc106f10 0x7f73bc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73b3fff640 1 -- 172.21.15.16:0/2230184475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73bc19d3e0 con 0x7f73bc102790 2024-09-19T09:27:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73bb7fe640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73bc106f10 0x7f73bc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73baffd640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73bc101de0 0x7f73bc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73b3fff640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73bc102790 0x7f73bc198970 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f73a4005ab0 tx=0x7f73a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73b8ff9640 1 -- 172.21.15.16:0/2230184475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a403f030 con 0x7f73bc102790 2024-09-19T09:27:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73c1268640 1 -- 172.21.15.16:0/2230184475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73bc19d670 con 0x7f73bc102790 2024-09-19T09:27:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73b8ff9640 1 -- 172.21.15.16:0/2230184475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a4003ce0 con 0x7f73bc102790 2024-09-19T09:27:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73b8ff9640 1 -- 172.21.15.16:0/2230184475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4035070 con 0x7f73bc102790 2024-09-19T09:27:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.064+0000 7f73c1268640 1 -- 172.21.15.16:0/2230184475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73bc1a4fb0 con 0x7f73bc102790 2024-09-19T09:27:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.067+0000 7f73b8ff9640 1 -- 172.21.15.16:0/2230184475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f73a4042030 con 0x7f73bc102790 2024-09-19T09:27:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.067+0000 7f73c3641740 1 Processor -- start 2024-09-19T09:27:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.067+0000 7f73c3641740 1 -- start start 2024-09-19T09:27:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.067+0000 7f73c3641740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0878b0 0x5635de087c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73c3641740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0881d0 0x5635de096a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73c3641740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 0x5635de0993b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73c3641740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5635ddfee090 con 0x5635de096fc0 2024-09-19T09:27:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73c3641740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5635ddf95a70 con 0x5635de0878b0 2024-09-19T09:27:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73c3641740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5635ddfea050 con 0x5635de0881d0 2024-09-19T09:27:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 0x5635de0993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73b8ff9640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f73a80a3240 0x7f73a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 0x5635de0993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53302/0 (socket says 172.21.15.16:53302) 2024-09-19T09:27:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73b3fff640 1 -- 172.21.15.16:0/2691703042 learned_addr learned my addr 172.21.15.16:0/2691703042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:14.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73baffd640 1 --2- 172.21.15.16:0/2691703042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0881d0 0x5635de096a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73bb7fe640 1 --2- 172.21.15.16:0/2691703042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0878b0 0x5635de087c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73bb7fe640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f73a80a3240 0x7f73a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73b3fff640 1 -- 172.21.15.16:0/2691703042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0881d0 msgr2=0x5635de096a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73b3fff640 1 --2- 172.21.15.16:0/2691703042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0881d0 0x5635de096a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73b3fff640 1 -- 172.21.15.16:0/2691703042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0878b0 msgr2=0x5635de087c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73b3fff640 1 --2- 172.21.15.16:0/2691703042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0878b0 0x5635de087c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73b8ff9640 1 -- 172.21.15.16:0/2230184475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f73a40f0290 con 0x7f73bc102790 2024-09-19T09:27:14.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73b3fff640 1 -- 172.21.15.16:0/2691703042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ddfa2130 con 0x5635de096fc0 2024-09-19T09:27:14.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73bb7fe640 1 --2- 172.21.15.16:0/2691703042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0878b0 0x5635de087c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:14.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73b3fff640 1 --2- 172.21.15.16:0/2691703042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 0x5635de0993b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f73a40b8590 tx=0x7f73a4047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47db9838245ff87e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:14.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.068+0000 7f73bb7fe640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f73a80a3240 0x7f73a80a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f73b4007310 tx=0x7f73b40072a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:14.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.069+0000 7f73b17fa640 1 -- 172.21.15.16:0/2691703042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4049070 con 0x5635de096fc0 2024-09-19T09:27:14.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.069+0000 7f73b17fa640 1 -- 172.21.15.16:0/2691703042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a40c0c60 con 0x5635de096fc0 2024-09-19T09:27:14.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.069+0000 7f73b17fa640 1 -- 172.21.15.16:0/2691703042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a40edea0 con 0x5635de096fc0 2024-09-19T09:27:14.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.069+0000 7f73c3641740 1 -- 172.21.15.16:0/2691703042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 msgr2=0x5635de0993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.069+0000 7f73c3641740 1 --2- 172.21.15.16:0/2691703042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 0x5635de0993b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f73a40b8590 tx=0x7f73a4047000 comp rx=0 tx=0).stop 2024-09-19T09:27:14.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.069+0000 7f73c3641740 1 -- 172.21.15.16:0/2691703042 shutdown_connections 2024-09-19T09:27:14.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.069+0000 7f73c3641740 1 --2- 172.21.15.16:0/2691703042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0881d0 0x5635de096a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.069+0000 7f73c3641740 1 --2- 172.21.15.16:0/2691703042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 0x5635de0993b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.069+0000 7f73c3641740 1 --2- 172.21.15.16:0/2691703042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0878b0 0x5635de087c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.069+0000 7f73c3641740 1 -- 172.21.15.16:0/2691703042 >> 172.21.15.16:0/2691703042 conn(0x5635de083f60 msgr2=0x5635de084340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:14.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.070+0000 7f73c3641740 1 -- 172.21.15.16:0/2691703042 shutdown_connections 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.070+0000 7f73c3641740 1 -- 172.21.15.16:0/2691703042 wait complete. 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.070+0000 7f73c3641740 1 -- start start 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.070+0000 7f73c3641740 1 Processor -- start 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.070+0000 7f73c3641740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0878b0 0x5635de096840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.070+0000 7f73c3641740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0881d0 0x5635de093b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.070+0000 7f73c3641740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 0x5635de0940d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.070+0000 7f73c3641740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5635de099950 con 0x5635de096fc0 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.070+0000 7f73c3641740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5635ddfa8f20 con 0x5635de0878b0 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.070+0000 7f73c3641740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5635de099ad0 con 0x5635de0881d0 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.070+0000 7f73c3641740 10 client.0 _ll_register_callbacks cb 0x5635ddf72030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.070+0000 7f73bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 0x5635de0940d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 0x5635de0940d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53316/0 (socket says 172.21.15.16:53316) 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73bb7fe640 1 -- 172.21.15.16:0/1557541004 learned_addr learned my addr 172.21.15.16:0/1557541004 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73baffd640 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0878b0 0x5635de096840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73b3fff640 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0881d0 0x5635de093b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73bb7fe640 1 -- 172.21.15.16:0/1557541004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0881d0 msgr2=0x5635de093b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73bb7fe640 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0881d0 0x5635de093b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73bb7fe640 1 -- 172.21.15.16:0/1557541004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0878b0 msgr2=0x5635de096840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73bb7fe640 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0878b0 0x5635de096840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73bb7fe640 1 -- 172.21.15.16:0/1557541004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ddd5f2b0 con 0x5635de096fc0 2024-09-19T09:27:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73baffd640 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0878b0 0x5635de096840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73b3fff640 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0881d0 0x5635de093b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73bb7fe640 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 0x5635de0940d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f73b4010ef0 tx=0x7f73b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c785ff0f3e43c47c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73a27fc640 1 -- 172.21.15.16:0/1557541004 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b4023030 con 0x5635de096fc0 2024-09-19T09:27:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73a27fc640 1 -- 172.21.15.16:0/1557541004 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f73b4004b20 con 0x5635de096fc0 2024-09-19T09:27:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73c3641740 1 -- 172.21.15.16:0/1557541004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5635ddffb600 con 0x5635de096fc0 2024-09-19T09:27:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73a27fc640 1 -- 172.21.15.16:0/1557541004 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f73b402f050 con 0x5635de096fc0 2024-09-19T09:27:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73c3641740 10 client.15171 fetch_fsmap learned FSMap version 27 2024-09-19T09:27:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.071+0000 7f73c3641740 1 -- 172.21.15.16:0/1557541004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635de0a51f0 con 0x5635de096fc0 2024-09-19T09:27:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.072+0000 7f73a27fc640 1 -- 172.21.15.16:0/1557541004 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5635ddffb600 con 0x5635de096fc0 2024-09-19T09:27:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.072+0000 7f73a27fc640 1 -- 172.21.15.16:0/1557541004 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f73b401c050 con 0x5635de096fc0 2024-09-19T09:27:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.072+0000 7f73c3641740 10 client.15171 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:27:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.072+0000 7f73c3641740 10 client.15171 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:27:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.072+0000 7f73c3641740 20 client.15171 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.072+0000 7f73c3641740 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5635de0a7830 0x5635de0a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.072+0000 7f73c3641740 4 client.15171 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.072+0000 7f73c3641740 1 -- 172.21.15.16:0/1557541004 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635ddff5b10 con 0x5635de0a7830 2024-09-19T09:27:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.072+0000 7f73baffd640 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5635de0a7830 0x5635de0a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.072+0000 7f73baffd640 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5635de0a7830 0x5635de0a8ca0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed2e5dc34965673 server_cookie=81e86227f768e6b6 in_seq=0 out_seq=0 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.073+0000 7f73a27fc640 10 client.15171 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.073+0000 7f73a27fc640 1 -- 172.21.15.16:0/1557541004 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635ddff5b10 con 0x5635de0a7830 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.073+0000 7f73a27fc640 10 client.15171 handle_command_reply: tid=0 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.073+0000 7f73c3641740 1 client.15171 shutdown 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.073+0000 7f73c3641740 1 -- 172.21.15.16:0/1557541004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 msgr2=0x5635de0940d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.073+0000 7f73c3641740 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 0x5635de0940d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f73b4010ef0 tx=0x7f73b4004a40 comp rx=0 tx=0).stop 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.073+0000 7f73c3641740 1 -- 172.21.15.16:0/1557541004 shutdown_connections 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.073+0000 7f73c3641740 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5635de0a7830 0x5635de0a8ca0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.073+0000 7f73c3641740 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0881d0 0x5635de093b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.073+0000 7f73c3641740 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de096fc0 0x5635de0940d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.073+0000 7f73c3641740 1 --2- 172.21.15.16:0/1557541004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0878b0 0x5635de096840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.073+0000 7f73a27fc640 0 client.15171 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.073+0000 7f73c3641740 1 -- 172.21.15.16:0/1557541004 >> 172.21.15.16:0/1557541004 conn(0x5635de083f60 msgr2=0x5635de089cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:14.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.074+0000 7f73c3641740 1 -- 172.21.15.16:0/1557541004 shutdown_connections 2024-09-19T09:27:14.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.074+0000 7f73c3641740 1 -- 172.21.15.16:0/1557541004 wait complete. 2024-09-19T09:27:14.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.074+0000 7f73c3641740 20 client.15171 trim_cache size 0 max 16384 2024-09-19T09:27:14.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.091+0000 7f73c3641740 1 Processor -- start 2024-09-19T09:27:14.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.091+0000 7f73c3641740 1 -- start start 2024-09-19T09:27:14.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73c3641740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0878b0 0x5635de0a2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73c3641740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0881d0 0x5635de0a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73c3641740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de096fc0 0x5635de092460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73c3641740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5635de0a6fe0 con 0x5635de0881d0 2024-09-19T09:27:14.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73c3641740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5635de0a76b0 con 0x5635de096fc0 2024-09-19T09:27:14.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73c3641740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5635de0a7d80 con 0x5635de0878b0 2024-09-19T09:27:14.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0881d0 0x5635de0a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0881d0 0x5635de0a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53322/0 (socket says 172.21.15.16:53322) 2024-09-19T09:27:14.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73b3fff640 1 -- 172.21.15.16:0/3967984983 learned_addr learned my addr 172.21.15.16:0/3967984983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:14.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73baffd640 1 --2- 172.21.15.16:0/3967984983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0878b0 0x5635de0a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73b3fff640 1 -- 172.21.15.16:0/3967984983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0878b0 msgr2=0x5635de0a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73b3fff640 1 --2- 172.21.15.16:0/3967984983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0878b0 0x5635de0a2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73b3fff640 1 -- 172.21.15.16:0/3967984983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de096fc0 msgr2=0x5635de092460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73b3fff640 1 --2- 172.21.15.16:0/3967984983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de096fc0 0x5635de092460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73b3fff640 1 -- 172.21.15.16:0/3967984983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635de0850d0 con 0x5635de0881d0 2024-09-19T09:27:14.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73b3fff640 1 --2- 172.21.15.16:0/3967984983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0881d0 0x5635de0a2ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f73a4000c00 tx=0x7f73a4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4833b31f573ed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:14.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73b0ff9640 1 -- 172.21.15.16:0/3967984983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4049040 con 0x5635de0881d0 2024-09-19T09:27:14.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73b0ff9640 1 -- 172.21.15.16:0/3967984983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a40bcfb0 con 0x5635de0881d0 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.092+0000 7f73b0ff9640 1 -- 172.21.15.16:0/3967984983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a40edf90 con 0x5635de0881d0 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.093+0000 7f73c3641740 1 -- 172.21.15.16:0/3967984983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0881d0 msgr2=0x5635de0a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.093+0000 7f73c3641740 1 --2- 172.21.15.16:0/3967984983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0881d0 0x5635de0a2ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f73a4000c00 tx=0x7f73a4003f90 comp rx=0 tx=0).stop 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.093+0000 7f73c3641740 1 -- 172.21.15.16:0/3967984983 shutdown_connections 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.093+0000 7f73c3641740 1 --2- 172.21.15.16:0/3967984983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de096fc0 0x5635de092460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.093+0000 7f73c3641740 1 --2- 172.21.15.16:0/3967984983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0881d0 0x5635de0a2ab0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.093+0000 7f73c3641740 1 --2- 172.21.15.16:0/3967984983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de0878b0 0x5635de0a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.093+0000 7f73c3641740 1 -- 172.21.15.16:0/3967984983 >> 172.21.15.16:0/3967984983 conn(0x5635de083f60 msgr2=0x5635de084340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.093+0000 7f73c3641740 1 -- 172.21.15.16:0/3967984983 shutdown_connections 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.093+0000 7f73c3641740 1 -- 172.21.15.16:0/3967984983 wait complete. 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73c3641740 1 -- start start 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73c3641740 1 Processor -- start 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73c3641740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0878b0 0x5635de0af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73c3641740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0881d0 0x5635de0afd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73c3641740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de096fc0 0x5635de0a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73c3641740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5635de0a8b20 con 0x5635de0878b0 2024-09-19T09:27:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73c3641740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5635de0a8450 con 0x5635de0881d0 2024-09-19T09:27:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73c3641740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5635de0a91f0 con 0x5635de096fc0 2024-09-19T09:27:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73c3641740 10 client.0 _ll_register_callbacks cb 0x7f73b4013520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0878b0 0x5635de0af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0878b0 0x5635de0af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53336/0 (socket says 172.21.15.16:53336) 2024-09-19T09:27:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73baffd640 1 -- 172.21.15.16:0/3221066282 learned_addr learned my addr 172.21.15.16:0/3221066282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73b3fff640 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0881d0 0x5635de0afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73baffd640 1 -- 172.21.15.16:0/3221066282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de096fc0 msgr2=0x5635de0a9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.094+0000 7f73bb7fe640 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de096fc0 0x5635de0a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73baffd640 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de096fc0 0x5635de0a9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73baffd640 1 -- 172.21.15.16:0/3221066282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0881d0 msgr2=0x5635de0afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73baffd640 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0881d0 0x5635de0afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73baffd640 1 -- 172.21.15.16:0/3221066282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635de0b1950 con 0x5635de0878b0 2024-09-19T09:27:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73b3fff640 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0881d0 0x5635de0afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73bb7fe640 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de096fc0 0x5635de0a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73baffd640 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0878b0 0x5635de0af7c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5635de093420 tx=0x7f73ac0158b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0eb46c00939762a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73a27fc640 1 -- 172.21.15.16:0/3221066282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac0145c0 con 0x5635de0878b0 2024-09-19T09:27:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73a27fc640 1 -- 172.21.15.16:0/3221066282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f73ac014760 con 0x5635de0878b0 2024-09-19T09:27:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73c3641740 1 -- 172.21.15.16:0/3221066282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a40bcfb0 con 0x5635de0878b0 2024-09-19T09:27:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73a27fc640 1 -- 172.21.15.16:0/3221066282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f73ac014960 con 0x5635de0878b0 2024-09-19T09:27:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73c3641740 10 client.15177 fetch_fsmap learned FSMap version 27 2024-09-19T09:27:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73c3641740 1 -- 172.21.15.16:0/3221066282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635de0ae8d0 con 0x5635de0878b0 2024-09-19T09:27:14.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73a27fc640 1 -- 172.21.15.16:0/3221066282 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a40bcfb0 con 0x5635de0878b0 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.095+0000 7f73a27fc640 1 -- 172.21.15.16:0/3221066282 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f73ac024c90 con 0x5635de0878b0 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.096+0000 7f73c3641740 10 client.15177 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.096+0000 7f73c3641740 10 client.15177 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.096+0000 7f73c3641740 20 client.15177 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.096+0000 7f73c3641740 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5635de115f30 0x5635de114c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.096+0000 7f73c3641740 4 client.15177 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.096+0000 7f73c3641740 1 -- 172.21.15.16:0/3221066282 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "status"}) -- 0x5635ddf720e0 con 0x5635de115f30 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.096+0000 7f73b3fff640 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5635de115f30 0x5635de114c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.096+0000 7f73a27fc640 10 client.15177 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.096+0000 7f73b3fff640 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5635de115f30 0x5635de114c40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b498cdbe659a603 server_cookie=2ed5cdbb72151e0a in_seq=0 out_seq=0 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.096+0000 7f73a27fc640 1 -- 172.21.15.16:0/3221066282 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5635ddf720e0 con 0x5635de115f30 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.096+0000 7f73a27fc640 10 client.15177 handle_command_reply: tid=0 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.096+0000 7f73c3641740 1 client.15177 shutdown 2024-09-19T09:27:14.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c3641740 1 -- 172.21.15.16:0/3221066282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0878b0 msgr2=0x5635de0af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c3641740 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0878b0 0x5635de0af7c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5635de093420 tx=0x7f73ac0158b0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c3641740 1 -- 172.21.15.16:0/3221066282 shutdown_connections 2024-09-19T09:27:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c3641740 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5635de115f30 0x5635de114c40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c3641740 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635de096fc0 0x5635de0a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c3641740 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635de0881d0 0x5635de0afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73a27fc640 0 client.15177 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c3641740 1 --2- 172.21.15.16:0/3221066282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635de0878b0 0x5635de0af7c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c3641740 1 -- 172.21.15.16:0/3221066282 >> 172.21.15.16:0/3221066282 conn(0x5635de083f60 msgr2=0x5635de092370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c3641740 1 -- 172.21.15.16:0/3221066282 shutdown_connections 2024-09-19T09:27:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c3641740 1 -- 172.21.15.16:0/3221066282 wait complete. 2024-09-19T09:27:14.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c3641740 20 client.15177 trim_cache size 0 max 16384 2024-09-19T09:27:14.112 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:27:14.112 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:27:14.112 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T09:27:14.112 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24500, 2024-09-19T09:27:14.112 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:27:14.112 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:27:14.112 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:27:14.112 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 103.81059329599999, 2024-09-19T09:27:14.112 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-19T09:27:14.112 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:27:14.112 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 79, 2024-09-19T09:27:14.112 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 114.766726752 2024-09-19T09:27:14.113 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:27:14.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c1268640 1 -- 172.21.15.16:0/2230184475 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f73a80a3240 msgr2=0x7f73a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c1268640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f73a80a3240 0x7f73a80a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f73b4007310 tx=0x7f73b40072a0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c1268640 1 -- 172.21.15.16:0/2230184475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73bc102790 msgr2=0x7f73bc198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c1268640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73bc102790 0x7f73bc198970 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f73a4005ab0 tx=0x7f73a40042f0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c1268640 1 -- 172.21.15.16:0/2230184475 shutdown_connections 2024-09-19T09:27:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c1268640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73bc106f10 0x7f73bc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c1268640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73bc102790 0x7f73bc198970 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c1268640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f73a80a3240 0x7f73a80a5700 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c1268640 1 --2- 172.21.15.16:0/2230184475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73bc101de0 0x7f73bc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.097+0000 7f73c1268640 1 -- 172.21.15.16:0/2230184475 >> 172.21.15.16:0/2230184475 conn(0x7f73bc0fd8c0 msgr2=0x7f73bc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.098+0000 7f73c1268640 1 -- 172.21.15.16:0/2230184475 shutdown_connections 2024-09-19T09:27:14.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:14.098+0000 7f73c1268640 1 -- 172.21.15.16:0/2230184475 wait complete. 2024-09-19T09:27:14.789 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T09:27:14.799 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T09:27:14.799 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T09:27:14.800 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T09:27:14.800 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T09:27:14.828 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T09:27:14.828 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T09:27:14.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:27:14.829 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T09:27:14.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:27:14.847 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-19T09:27:14.917 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-09-19T09:27:14.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:27:14.919 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-19T09:27:14.919 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T09:27:14.919 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T09:27:14.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T09:27:15.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:14 smithi142 ceph-mon[35032]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T09:27:15.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:14 smithi097 ceph-mon[34864]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T09:27:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:15.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:14 smithi016 ceph-mon[30979]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-19T09:27:15.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.705+0000 7efea69dc640 1 -- 172.21.15.16:0/215902850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea01082a0 msgr2=0x7efea010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.705+0000 7efea69dc640 1 --2- 172.21.15.16:0/215902850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea01082a0 0x7efea010ec40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efe98009cd0 tx=0x7efe9802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.705+0000 7efea69dc640 1 -- 172.21.15.16:0/215902850 shutdown_connections 2024-09-19T09:27:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.705+0000 7efea69dc640 1 --2- 172.21.15.16:0/215902850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea01082a0 0x7efea010ec40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.705+0000 7efea69dc640 1 --2- 172.21.15.16:0/215902850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea01036e0 0x7efea0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.705+0000 7efea69dc640 1 --2- 172.21.15.16:0/215902850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea0102d30 0x7efea0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.705+0000 7efea69dc640 1 -- 172.21.15.16:0/215902850 >> 172.21.15.16:0/215902850 conn(0x7efea00fc7d0 msgr2=0x7efea00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.705+0000 7efea69dc640 1 -- 172.21.15.16:0/215902850 shutdown_connections 2024-09-19T09:27:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.705+0000 7efea69dc640 1 -- 172.21.15.16:0/215902850 wait complete. 2024-09-19T09:27:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.705+0000 7efea69dc640 1 Processor -- start 2024-09-19T09:27:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.705+0000 7efea69dc640 1 -- start start 2024-09-19T09:27:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efea69dc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea0102d30 0x7efea01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efea69dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea01036e0 0x7efea0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efea69dc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea01082a0 0x7efea019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efea69dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efea0112100 con 0x7efea01036e0 2024-09-19T09:27:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efea69dc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efea0111f80 con 0x7efea0102d30 2024-09-19T09:27:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efea69dc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efea0112280 con 0x7efea01082a0 2024-09-19T09:27:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efe9f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea01036e0 0x7efea0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efe9f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea01036e0 0x7efea0198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39546/0 (socket says 172.21.15.16:39546) 2024-09-19T09:27:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efe9f7fe640 1 -- 172.21.15.16:0/3470852386 learned_addr learned my addr 172.21.15.16:0/3470852386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efe9ffff640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea0102d30 0x7efea01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efea4f79640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea01082a0 0x7efea019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efe9f7fe640 1 -- 172.21.15.16:0/3470852386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea01082a0 msgr2=0x7efea019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efe9f7fe640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea01082a0 0x7efea019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efe9f7fe640 1 -- 172.21.15.16:0/3470852386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea0102d30 msgr2=0x7efea01985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efe9f7fe640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea0102d30 0x7efea01985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efe9f7fe640 1 -- 172.21.15.16:0/3470852386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea019d580 con 0x7efea01036e0 2024-09-19T09:27:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efe9ffff640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea0102d30 0x7efea01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efea4f79640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea01082a0 0x7efea019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:15.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.706+0000 7efe9f7fe640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea01036e0 0x7efea0198b10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efe9400d740 tx=0x7efe9400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.707+0000 7efe9d7fa640 1 -- 172.21.15.16:0/3470852386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94017070 con 0x7efea01036e0 2024-09-19T09:27:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.707+0000 7efe9d7fa640 1 -- 172.21.15.16:0/3470852386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe9400de50 con 0x7efea01036e0 2024-09-19T09:27:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.707+0000 7efea69dc640 1 -- 172.21.15.16:0/3470852386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea019d870 con 0x7efea01036e0 2024-09-19T09:27:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.707+0000 7efe9d7fa640 1 -- 172.21.15.16:0/3470852386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9400f6b0 con 0x7efea01036e0 2024-09-19T09:27:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.707+0000 7efea69dc640 1 -- 172.21.15.16:0/3470852386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea01a5150 con 0x7efea01036e0 2024-09-19T09:27:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.708+0000 7efea69dc640 1 -- 172.21.15.16:0/3470852386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea0103110 con 0x7efea01036e0 2024-09-19T09:27:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.709+0000 7efe9d7fa640 1 -- 172.21.15.16:0/3470852386 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7efe9401a070 con 0x7efea01036e0 2024-09-19T09:27:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.710+0000 7efe9d7fa640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7efe880a3240 0x7efe880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.710+0000 7efe9d7fa640 1 -- 172.21.15.16:0/3470852386 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7efe940cf3e0 con 0x7efea01036e0 2024-09-19T09:27:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.710+0000 7efe9ffff640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7efe880a3240 0x7efe880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.710+0000 7efe9ffff640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7efe880a3240 0x7efe880a5700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efe90009930 tx=0x7efe90008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.714+0000 7efe9d7fa640 1 -- 172.21.15.16:0/3470852386 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efe9409c100 con 0x7efea01036e0 2024-09-19T09:27:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.927+0000 7efea69dc640 1 -- 172.21.15.16:0/3470852386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efea010a2e0 con 0x7efea01036e0 2024-09-19T09:27:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.927+0000 7efe9d7fa640 1 -- 172.21.15.16:0/3470852386 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7efe9409ba00 con 0x7efea01036e0 2024-09-19T09:27:15.928 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T09:27:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.929+0000 7efea69dc640 1 -- 172.21.15.16:0/3470852386 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7efe880a3240 msgr2=0x7efe880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.929+0000 7efea69dc640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7efe880a3240 0x7efe880a5700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efe90009930 tx=0x7efe90008040 comp rx=0 tx=0).stop 2024-09-19T09:27:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.929+0000 7efea69dc640 1 -- 172.21.15.16:0/3470852386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea01036e0 msgr2=0x7efea0198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.929+0000 7efea69dc640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea01036e0 0x7efea0198b10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efe9400d740 tx=0x7efe9400dc00 comp rx=0 tx=0).stop 2024-09-19T09:27:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.929+0000 7efea69dc640 1 -- 172.21.15.16:0/3470852386 shutdown_connections 2024-09-19T09:27:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.929+0000 7efea69dc640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7efe880a3240 0x7efe880a5700 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.929+0000 7efea69dc640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efea01082a0 0x7efea019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.929+0000 7efea69dc640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efea01036e0 0x7efea0198b10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.929+0000 7efea69dc640 1 --2- 172.21.15.16:0/3470852386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efea0102d30 0x7efea01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.929+0000 7efea69dc640 1 -- 172.21.15.16:0/3470852386 >> 172.21.15.16:0/3470852386 conn(0x7efea00fc7d0 msgr2=0x7efea0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.929+0000 7efea69dc640 1 -- 172.21.15.16:0/3470852386 shutdown_connections 2024-09-19T09:27:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:15.929+0000 7efea69dc640 1 -- 172.21.15.16:0/3470852386 wait complete. 2024-09-19T09:27:16.589 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T09:27:16.589 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T09:27:16.589 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-19T09:27:16.589 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T09:27:16.589 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T09:27:16.589 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T09:27:16.589 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T09:27:16.589 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T09:27:16.590 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T09:27:16.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:27:16.590 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-19T09:27:16.609 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T09:27:16.609 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T09:27:16.609 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T09:27:16.609 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T09:27:16.609 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-19T09:27:16.609 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T09:27:16.609 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T09:27:16.609 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T09:27:16.609 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-19T09:27:16.609 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T09:27:16.610 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T09:27:16.610 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-19T09:27:16.610 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T09:27:16.610 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-19T09:27:16.610 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-19T09:27:16.610 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T09:27:16.610 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42205sec preferred_lft 42205sec 2024-09-19T09:27:16.610 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-19T09:27:16.610 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-19T09:27:16.610 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T09:27:16.610 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-19T09:27:16.610 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-19T09:27:16.611 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T09:27:16.611 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:27:16.611 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T09:27:16.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-19T09:27:16.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-19T09:27:16.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-19T09:27:16.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T09:27:16.611 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T09:27:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:27:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:27:16.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:27:16.832 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T09:27:16.859 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-19T09:27:16.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:27:16.860 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-19T09:27:16.933 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-19T09:27:16.933 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T09:27:16.934 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T09:27:16.934 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T09:27:16.934 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T09:27:16.934 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:27:16.934 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T09:27:16.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T09:27:16.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T09:27:16.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T09:27:16.935 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T09:27:17.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:16 smithi016 ceph-mon[30979]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T09:27:17.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:16 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3470852386' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T09:27:17.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:27:17.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:16 smithi142 ceph-mon[35032]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T09:27:17.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:16 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3470852386' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T09:27:17.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:16 smithi097 ceph-mon[34864]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T09:27:17.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:16 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3470852386' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T09:27:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:27:17.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:27:17.253 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T09:27:17.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:27:17.311 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-19T09:27:17.367 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:27:17.367 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T09:27:17.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T09:27:17.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T09:27:17.368 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T09:27:17.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:27:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:27:17.489 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T09:27:17.489 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:27:17.489 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T09:27:17.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T09:27:17.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T09:27:17.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T09:27:17.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T09:27:17.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T09:27:17.490 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T09:27:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:27:17.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:27:17.616 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T09:27:17.616 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-19T09:27:17.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-19T09:27:17.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T09:27:17.617 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-19T09:27:17.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:27:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:27:17.696 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T09:27:17.696 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T09:27:17.712 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T09:27:17.713 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T09:27:17.713 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T09:27:17.769 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T09:27:17.769 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T09:27:17.770 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/fb0e7bdc-edce-464b-b5ca-12d056277b39 2024-09-19T09:27:17.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:27:17.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/fb0e7bdc-edce-464b-b5ca-12d056277b39 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-19T09:27:17.881 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-19T09:27:17.881 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-19T09:27:18.007 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-19T09:27:18.007 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-19T09:27:18.007 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.97:3300,172.21.15.142:3300" 2024-09-19T09:27:18.007 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@ee1ab676-7667-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/fb0e7bdc-edce-464b-b5ca-12d056277b39 2024-09-19T09:27:18.007 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.16:3300/172.21.15.97:3300/172.21.15.142:3300" will pass to kernel 2024-09-19T09:27:18.008 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T09:27:18.008 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-09-19T09:27:18.009 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T09:27:18.009 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T09:27:18.009 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T09:27:18.011 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T09:27:18.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:27:18.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T09:27:18.038 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T09:27:18.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T09:27:18.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:27:18.146 INFO:teuthology.orchestra.run.smithi016.stdout:client24869 2024-09-19T09:27:18.147 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:27:18.147 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T09:27:18.165 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T09:27:18.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T09:27:18.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:27:18.263 INFO:teuthology.orchestra.run.smithi016.stdout:client24869 2024-09-19T09:27:18.264 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T09:27:18.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T09:27:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:27:18.321 INFO:teuthology.orchestra.run.smithi016.stdout:client24869 2024-09-19T09:27:18.322 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ee1ab676-7667-11ef-bceb-c7b262605968.client24869/status') 2024-09-19T09:27:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T09:27:18.381 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.24869 (3)172.21.15.16:0/3688225227 2024-09-19T09:27:18.381 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-09-19T09:27:18.382 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-09-19T09:27:18.382 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-09-19T09:27:18.382 INFO:teuthology.orchestra.run.smithi016.stderr:71 bytes copied, 2.3094e-05 s, 3.1 MB/s 2024-09-19T09:27:18.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T09:27:18.390 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T09:27:18.399 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T09:27:18.400 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T09:27:18.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-19T09:27:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.503+0000 7fa02771a640 1 -- 172.21.15.16:0/3982221340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa020107890 msgr2=0x7fa020109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.503+0000 7fa02771a640 1 --2- 172.21.15.16:0/3982221340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa020107890 0x7fa020109ca0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa018009ca0 tx=0x7fa01802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.503+0000 7fa02771a640 1 -- 172.21.15.16:0/3982221340 shutdown_connections 2024-09-19T09:27:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.503+0000 7fa02771a640 1 --2- 172.21.15.16:0/3982221340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa020107890 0x7fa020109ca0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.503+0000 7fa02771a640 1 --2- 172.21.15.16:0/3982221340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa020104f60 0x7fa020107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.503+0000 7fa02771a640 1 --2- 172.21.15.16:0/3982221340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa020102630 0x7fa020104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.503+0000 7fa02771a640 1 -- 172.21.15.16:0/3982221340 >> 172.21.15.16:0/3982221340 conn(0x7fa0200fc1b0 msgr2=0x7fa0200fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.503+0000 7fa02771a640 1 -- 172.21.15.16:0/3982221340 shutdown_connections 2024-09-19T09:27:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.503+0000 7fa02771a640 1 -- 172.21.15.16:0/3982221340 wait complete. 2024-09-19T09:27:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa02771a640 1 Processor -- start 2024-09-19T09:27:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa02771a640 1 -- start start 2024-09-19T09:27:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa02771a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa020102630 0x7fa020197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa02771a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa020104f60 0x7fa020198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa02771a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa020107890 0x7fa02019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa02771a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa02010d7f0 con 0x7fa020107890 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa02771a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa02010d670 con 0x7fa020102630 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa02771a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa02010d970 con 0x7fa020104f60 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa025cb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa020107890 0x7fa02019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa025cb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa020107890 0x7fa02019f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39564/0 (socket says 172.21.15.16:39564) 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa025cb7640 1 -- 172.21.15.16:0/850019119 learned_addr learned my addr 172.21.15.16:0/850019119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa0254b6640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa020102630 0x7fa020197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa024cb5640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa020104f60 0x7fa020198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa025cb7640 1 -- 172.21.15.16:0/850019119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa020104f60 msgr2=0x7fa020198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa025cb7640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa020104f60 0x7fa020198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa025cb7640 1 -- 172.21.15.16:0/850019119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa020102630 msgr2=0x7fa020197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa025cb7640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa020102630 0x7fa020197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa025cb7640 1 -- 172.21.15.16:0/850019119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02019f9e0 con 0x7fa020107890 2024-09-19T09:27:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.504+0000 7fa024cb5640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa020104f60 0x7fa020198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.505+0000 7fa025cb7640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa020107890 0x7fa02019f370 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa01802f540 tx=0x7fa018031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.505+0000 7fa0167fc640 1 -- 172.21.15.16:0/850019119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018003c30 con 0x7fa020107890 2024-09-19T09:27:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.505+0000 7fa0167fc640 1 -- 172.21.15.16:0/850019119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa018003dd0 con 0x7fa020107890 2024-09-19T09:27:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.505+0000 7fa02771a640 1 -- 172.21.15.16:0/850019119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa02019fc70 con 0x7fa020107890 2024-09-19T09:27:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.505+0000 7fa0167fc640 1 -- 172.21.15.16:0/850019119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01802fd90 con 0x7fa020107890 2024-09-19T09:27:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.505+0000 7fa02771a640 1 -- 172.21.15.16:0/850019119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0201a01c0 con 0x7fa020107890 2024-09-19T09:27:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.506+0000 7fa02771a640 1 -- 172.21.15.16:0/850019119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0200fdca0 con 0x7fa020107890 2024-09-19T09:27:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.507+0000 7fa0167fc640 1 -- 172.21.15.16:0/850019119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa018004050 con 0x7fa020107890 2024-09-19T09:27:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.508+0000 7fa0167fc640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa00c0a3370 0x7fa00c0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.508+0000 7fa0167fc640 1 -- 172.21.15.16:0/850019119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa0180f0a40 con 0x7fa020107890 2024-09-19T09:27:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.508+0000 7fa0254b6640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa00c0a3370 0x7fa00c0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.508+0000 7fa0254b6640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa00c0a3370 0x7fa00c0a5830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00bbe0 tx=0x7fa01c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.512+0000 7fa0167fc640 1 -- 172.21.15.16:0/850019119 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa0180f2e20 con 0x7fa020107890 2024-09-19T09:27:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.728+0000 7fa02771a640 1 -- 172.21.15.16:0/850019119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fa020062e20 con 0x7fa020107890 2024-09-19T09:27:18.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.868+0000 7fa0167fc640 1 -- 172.21.15.16:0/850019119 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7fa0180bd880 con 0x7fa020107890 2024-09-19T09:27:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.870+0000 7fa02771a640 1 -- 172.21.15.16:0/850019119 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa00c0a3370 msgr2=0x7fa00c0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.870+0000 7fa02771a640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa00c0a3370 0x7fa00c0a5830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00bbe0 tx=0x7fa01c00b390 comp rx=0 tx=0).stop 2024-09-19T09:27:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.870+0000 7fa02771a640 1 -- 172.21.15.16:0/850019119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa020107890 msgr2=0x7fa02019f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.870+0000 7fa02771a640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa020107890 0x7fa02019f370 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa01802f540 tx=0x7fa018031ee0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.870+0000 7fa02771a640 1 -- 172.21.15.16:0/850019119 shutdown_connections 2024-09-19T09:27:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.870+0000 7fa02771a640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa00c0a3370 0x7fa00c0a5830 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.870+0000 7fa02771a640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa020104f60 0x7fa020198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.870+0000 7fa02771a640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa020107890 0x7fa02019f370 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.870+0000 7fa02771a640 1 --2- 172.21.15.16:0/850019119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa020102630 0x7fa020197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.870+0000 7fa02771a640 1 -- 172.21.15.16:0/850019119 >> 172.21.15.16:0/850019119 conn(0x7fa0200fc1b0 msgr2=0x7fa020102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.870+0000 7fa02771a640 1 -- 172.21.15.16:0/850019119 shutdown_connections 2024-09-19T09:27:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:18.870+0000 7fa02771a640 1 -- 172.21.15.16:0/850019119 wait complete. 2024-09-19T09:27:18.884 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T09:27:18.895 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T09:27:18.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T09:27:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.013+0000 7f294b11f640 1 -- 172.21.15.16:0/1353389864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29441025e0 msgr2=0x7f2944108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.013+0000 7f294b11f640 1 --2- 172.21.15.16:0/1353389864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29441025e0 0x7f2944108e70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f293c009ca0 tx=0x7f293c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.013+0000 7f294b11f640 1 -- 172.21.15.16:0/1353389864 shutdown_connections 2024-09-19T09:27:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.013+0000 7f294b11f640 1 --2- 172.21.15.16:0/1353389864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2944101c20 0x7f29441020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.013+0000 7f294b11f640 1 --2- 172.21.15.16:0/1353389864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29441025e0 0x7f2944108e70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.013+0000 7f294b11f640 1 --2- 172.21.15.16:0/1353389864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2944100a30 0x7f2944100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.014+0000 7f294b11f640 1 -- 172.21.15.16:0/1353389864 >> 172.21.15.16:0/1353389864 conn(0x7f29440fc1b0 msgr2=0x7f29440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.014+0000 7f294b11f640 1 -- 172.21.15.16:0/1353389864 shutdown_connections 2024-09-19T09:27:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.014+0000 7f294b11f640 1 -- 172.21.15.16:0/1353389864 wait complete. 2024-09-19T09:27:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.014+0000 7f294b11f640 1 Processor -- start 2024-09-19T09:27:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.014+0000 7f294b11f640 1 -- start start 2024-09-19T09:27:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.014+0000 7f294b11f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2944100a30 0x7f294410b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.014+0000 7f294b11f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944101c20 0x7f29441099f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.014+0000 7f294b11f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29441025e0 0x7f2944109f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.014+0000 7f294b11f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f294410d860 con 0x7f2944101c20 2024-09-19T09:27:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.014+0000 7f294b11f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f294410d6e0 con 0x7f2944100a30 2024-09-19T09:27:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.014+0000 7f294b11f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f294410d9e0 con 0x7f29441025e0 2024-09-19T09:27:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f2943fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944101c20 0x7f29441099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f2943fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944101c20 0x7f29441099f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39576/0 (socket says 172.21.15.16:39576) 2024-09-19T09:27:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f2948ebb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2944100a30 0x7f294410b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f2943fff640 1 -- 172.21.15.16:0/950829107 learned_addr learned my addr 172.21.15.16:0/950829107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f29496bc640 1 --2- 172.21.15.16:0/950829107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29441025e0 0x7f2944109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f2943fff640 1 -- 172.21.15.16:0/950829107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29441025e0 msgr2=0x7f2944109f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f2943fff640 1 --2- 172.21.15.16:0/950829107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29441025e0 0x7f2944109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f2943fff640 1 -- 172.21.15.16:0/950829107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2944100a30 msgr2=0x7f294410b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f2943fff640 1 --2- 172.21.15.16:0/950829107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2944100a30 0x7f294410b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f2943fff640 1 -- 172.21.15.16:0/950829107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294410a7c0 con 0x7f2944101c20 2024-09-19T09:27:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f2943fff640 1 --2- 172.21.15.16:0/950829107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944101c20 0x7f29441099f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f293000d950 tx=0x7f293000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f2941ffb640 1 -- 172.21.15.16:0/950829107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930017030 con 0x7f2944101c20 2024-09-19T09:27:19.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f294b11f640 1 -- 172.21.15.16:0/950829107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29441a1ef0 con 0x7f2944101c20 2024-09-19T09:27:19.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f2941ffb640 1 -- 172.21.15.16:0/950829107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2930004020 con 0x7f2944101c20 2024-09-19T09:27:19.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.015+0000 7f2941ffb640 1 -- 172.21.15.16:0/950829107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29300047a0 con 0x7f2944101c20 2024-09-19T09:27:19.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.016+0000 7f294b11f640 1 -- 172.21.15.16:0/950829107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29441a2360 con 0x7f2944101c20 2024-09-19T09:27:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.017+0000 7f294b11f640 1 -- 172.21.15.16:0/950829107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29280052c0 con 0x7f2944101c20 2024-09-19T09:27:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.018+0000 7f2941ffb640 1 -- 172.21.15.16:0/950829107 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f29300041d0 con 0x7f2944101c20 2024-09-19T09:27:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.018+0000 7f2941ffb640 1 --2- 172.21.15.16:0/950829107 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f29340a3250 0x7f29340a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.018+0000 7f2948ebb640 1 --2- 172.21.15.16:0/950829107 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f29340a3250 0x7f29340a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.019+0000 7f2941ffb640 1 -- 172.21.15.16:0/950829107 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f29300cf260 con 0x7f2944101c20 2024-09-19T09:27:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.019+0000 7f2948ebb640 1 --2- 172.21.15.16:0/950829107 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f29340a3250 0x7f29340a5710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2944101a80 tx=0x7f293800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.023+0000 7f2941ffb640 1 -- 172.21.15.16:0/950829107 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f293009bf80 con 0x7f2944101c20 2024-09-19T09:27:19.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:18 smithi142 ceph-mon[35032]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T09:27:19.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:18 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/850019119' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T09:27:19.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:18 smithi097 ceph-mon[34864]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T09:27:19.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:18 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/850019119' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T09:27:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.200+0000 7f294b11f640 1 -- 172.21.15.16:0/950829107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29280055b0 con 0x7f2944101c20 2024-09-19T09:27:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.201+0000 7f2941ffb640 1 -- 172.21.15.16:0/950829107 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22665 (secure 0 0 0) 0x7f293009b880 con 0x7f2944101c20 2024-09-19T09:27:19.203 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","created":"2024-09-19T09:17:48.242238+0000","modified":"2024-09-19T09:25:42.982671+0000","last_up_change":"2024-09-19T09:24:40.190289+0000","last_in_change":"2024-09-19T09:24:17.190380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:20:57.748686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:25:25.694451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:25:28.262997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbf3b28b-2282-4057-9e65-19ceb78ff19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6803","nonce":1578957623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6805","nonce":1578957623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6809","nonce":1578957623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6807","nonce":1578957623}]},"public_addr":"172.21.15.16:6803/1578957623","cluster_addr":"172.21.15.16:6805/1578957623","heartbeat_back_addr":"172.21.15.16:6809/1578957623","heartbeat_front_addr":"172.21.15.16:6807/1578957623","state":["exists","up"]},{"osd":1,"uuid":"c2517aeb-1d79-41ad-a180-369eecb9732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6801","nonce":3804152349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6803","nonce":3804152349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6807","nonce":3804152349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6805","nonce":3804152349}]},"public_addr":"172.21.15.97:6801/3804152349","cluster_addr":"172.21.15.97:6803/3804152349","heartbeat_back_addr":"172.21.15.97:6807/3804152349","heartbeat_front_addr":"172.21.15.97:6805/3804152349","state":["exists","up"]},{"osd":2,"uuid":"3c367518-5085-471a-98cb-ec5bf5facdf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6801","nonce":2692613951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6803","nonce":2692613951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6807","nonce":2692613951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6805","nonce":2692613951}]},"public_addr":"172.21.15.142:6801/2692613951","cluster_addr":"172.21.15.142:6803/2692613951","heartbeat_back_addr":"172.21.15.142:6807/2692613951","heartbeat_front_addr":"172.21.15.142:6805/2692613951","state":["exists","up"]},{"osd":3,"uuid":"e7118925-f0fa-46b3-8552-3e3bb76a6489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6811","nonce":3169290004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6813","nonce":3169290004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6817","nonce":3169290004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6815","nonce":3169290004}]},"public_addr":"172.21.15.16:6811/3169290004","cluster_addr":"172.21.15.16:6813/3169290004","heartbeat_back_addr":"172.21.15.16:6817/3169290004","heartbeat_front_addr":"172.21.15.16:6815/3169290004","state":["exists","up"]},{"osd":4,"uuid":"dc1964f7-af9c-4853-843b-9818f87c7768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6809","nonce":3491353825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6811","nonce":3491353825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6815","nonce":3491353825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6813","nonce":3491353825}]},"public_addr":"172.21.15.97:6809/3491353825","cluster_addr":"172.21.15.97:6811/3491353825","heartbeat_back_addr":"172.21.15.97:6815/3491353825","heartbeat_front_addr":"172.21.15.97:6813/3491353825","state":["exists","up"]},{"osd":5,"uuid":"4d7fbafc-d9ba-4036-a89d-aa295b8a23a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6809","nonce":577099986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6811","nonce":577099986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6815","nonce":577099986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6813","nonce":577099986}]},"public_addr":"172.21.15.142:6809/577099986","cluster_addr":"172.21.15.142:6811/577099986","heartbeat_back_addr":"172.21.15.142:6815/577099986","heartbeat_front_addr":"172.21.15.142:6813/577099986","state":["exists","up"]},{"osd":6,"uuid":"e3f346fc-c027-44f8-bd37-bccf6ec09e3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6819","nonce":4053539536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6821","nonce":4053539536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6825","nonce":4053539536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6823","nonce":4053539536}]},"public_addr":"172.21.15.16:6819/4053539536","cluster_addr":"172.21.15.16:6821/4053539536","heartbeat_back_addr":"172.21.15.16:6825/4053539536","heartbeat_front_addr":"172.21.15.16:6823/4053539536","state":["exists","up"]},{"osd":7,"uuid":"d390b547-8949-4d4b-8481-c49d6ae7e424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6817","nonce":1157122670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6819","nonce":1157122670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6823","nonce":1157122670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6821","nonce":1157122670}]},"public_addr":"172.21.15.97:6817/1157122670","cluster_addr":"172.21.15.97:6819/1157122670","heartbeat_back_addr":"172.21.15.97:6823/1157122670","heartbeat_front_addr":"172.21.15.97:6821/1157122670","state":["exists","up"]},{"osd":8,"uuid":"78a476c4-5deb-4f9e-a266-db4493850732","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6817","nonce":643795004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6819","nonce":643795004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6823","nonce":643795004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6821","nonce":643795004}]},"public_addr":"172.21.15.142:6817/643795004","cluster_addr":"172.21.15.142:6819/643795004","heartbeat_back_addr":"172.21.15.142:6823/643795004","heartbeat_front_addr":"172.21.15.142:6821/643795004","state":["exists","up"]},{"osd":9,"uuid":"0415d187-4a77-42f7-b8bb-2d0b52e9cb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6827","nonce":2651132360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6829","nonce":2651132360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2651132360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6831","nonce":2651132360}]},"public_addr":"172.21.15.16:6827/2651132360","cluster_addr":"172.21.15.16:6829/2651132360","heartbeat_back_addr":"172.21.15.16:6833/2651132360","heartbeat_front_addr":"172.21.15.16:6831/2651132360","state":["exists","up"]},{"osd":10,"uuid":"6fa78f96-cdfe-42a0-8d27-0705a0c7df8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6825","nonce":1848371486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6827","nonce":1848371486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6831","nonce":1848371486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6829","nonce":1848371486}]},"public_addr":"172.21.15.97:6825/1848371486","cluster_addr":"172.21.15.97:6827/1848371486","heartbeat_back_addr":"172.21.15.97:6831/1848371486","heartbeat_front_addr":"172.21.15.97:6829/1848371486","state":["exists","up"]},{"osd":11,"uuid":"d27f20e3-84ce-470c-9889-cd86906b158d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6825","nonce":155083878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6827","nonce":155083878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6831","nonce":155083878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6829","nonce":155083878}]},"public_addr":"172.21.15.142:6825/155083878","cluster_addr":"172.21.15.142:6827/155083878","heartbeat_back_addr":"172.21.15.142:6831/155083878","heartbeat_front_addr":"172.21.15.142:6829/155083878","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:05.088764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:29.928529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:55.236930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:20.379471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:44.984151+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:10.410074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:35.233553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:59.928700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:24.503069+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:49.853073+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:14.006841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:38.546467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3710267992":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3740275561":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/2755421444":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/357390846":"2024-09-20T09:18:17.070034+0000","172.21.15.16:6800/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3104087402":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3546947161":"2024-09-20T09:18:17.070034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:27:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.203+0000 7f294b11f640 1 -- 172.21.15.16:0/950829107 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f29340a3250 msgr2=0x7f29340a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.203+0000 7f294b11f640 1 --2- 172.21.15.16:0/950829107 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f29340a3250 0x7f29340a5710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2944101a80 tx=0x7f293800b390 comp rx=0 tx=0).stop 2024-09-19T09:27:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.203+0000 7f294b11f640 1 -- 172.21.15.16:0/950829107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944101c20 msgr2=0x7f29441099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.203+0000 7f294b11f640 1 --2- 172.21.15.16:0/950829107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944101c20 0x7f29441099f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f293000d950 tx=0x7f293000de10 comp rx=0 tx=0).stop 2024-09-19T09:27:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.203+0000 7f294b11f640 1 -- 172.21.15.16:0/950829107 shutdown_connections 2024-09-19T09:27:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.203+0000 7f294b11f640 1 --2- 172.21.15.16:0/950829107 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f29340a3250 0x7f29340a5710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.203+0000 7f294b11f640 1 --2- 172.21.15.16:0/950829107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29441025e0 0x7f2944109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.203+0000 7f294b11f640 1 --2- 172.21.15.16:0/950829107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2944101c20 0x7f29441099f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.203+0000 7f294b11f640 1 --2- 172.21.15.16:0/950829107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2944100a30 0x7f294410b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.203+0000 7f294b11f640 1 -- 172.21.15.16:0/950829107 >> 172.21.15.16:0/950829107 conn(0x7f29440fc1b0 msgr2=0x7f2944104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.204+0000 7f294b11f640 1 -- 172.21.15.16:0/950829107 shutdown_connections 2024-09-19T09:27:19.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.204+0000 7f294b11f640 1 -- 172.21.15.16:0/950829107 wait complete. 2024-09-19T09:27:19.214 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:18 smithi016 ceph-mon[30979]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-19T09:27:19.214 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:18 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/850019119' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T09:27:19.216 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T09:20:57.748686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T09:25:25.694451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T09:25:28.262997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T09:27:19.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T09:27:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.322+0000 7fb994933640 1 -- 172.21.15.16:0/1172018623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9901025b0 msgr2=0x7fb9901049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.322+0000 7fb994933640 1 --2- 172.21.15.16:0/1172018623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9901025b0 0x7fb9901049a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb980009900 tx=0x7fb98002a410 comp rx=0 tx=0).stop 2024-09-19T09:27:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.322+0000 7fb994933640 1 -- 172.21.15.16:0/1172018623 shutdown_connections 2024-09-19T09:27:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.322+0000 7fb994933640 1 --2- 172.21.15.16:0/1172018623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb990107810 0x7fb990109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.322+0000 7fb994933640 1 --2- 172.21.15.16:0/1172018623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb990104ee0 0x7fb9901072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.322+0000 7fb994933640 1 --2- 172.21.15.16:0/1172018623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9901025b0 0x7fb9901049a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.322+0000 7fb994933640 1 -- 172.21.15.16:0/1172018623 >> 172.21.15.16:0/1172018623 conn(0x7fb9900fc1b0 msgr2=0x7fb9900fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.322+0000 7fb994933640 1 -- 172.21.15.16:0/1172018623 shutdown_connections 2024-09-19T09:27:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.322+0000 7fb994933640 1 -- 172.21.15.16:0/1172018623 wait complete. 2024-09-19T09:27:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.323+0000 7fb994933640 1 Processor -- start 2024-09-19T09:27:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.323+0000 7fb994933640 1 -- start start 2024-09-19T09:27:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.323+0000 7fb994933640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9901025b0 0x7fb990101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.323+0000 7fb994933640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb990104ee0 0x7fb990100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.323+0000 7fb994933640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb990107810 0x7fb990100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.323+0000 7fb994933640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb99010d820 con 0x7fb990104ee0 2024-09-19T09:27:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.323+0000 7fb994933640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb99010d6a0 con 0x7fb990107810 2024-09-19T09:27:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.323+0000 7fb994933640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb99010d9a0 con 0x7fb9901025b0 2024-09-19T09:27:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.323+0000 7fb98dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb990104ee0 0x7fb990100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.323+0000 7fb98dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb990104ee0 0x7fb990100560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39596/0 (socket says 172.21.15.16:39596) 2024-09-19T09:27:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.323+0000 7fb98dd9b640 1 -- 172.21.15.16:0/3735082953 learned_addr learned my addr 172.21.15.16:0/3735082953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.323+0000 7fb98ed9d640 1 --2- 172.21.15.16:0/3735082953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb990107810 0x7fb990100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.324+0000 7fb98dd9b640 1 -- 172.21.15.16:0/3735082953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9901025b0 msgr2=0x7fb990101f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.324+0000 7fb98dd9b640 1 --2- 172.21.15.16:0/3735082953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9901025b0 0x7fb990101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.324+0000 7fb98dd9b640 1 -- 172.21.15.16:0/3735082953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb990107810 msgr2=0x7fb990100aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.324+0000 7fb98dd9b640 1 --2- 172.21.15.16:0/3735082953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb990107810 0x7fb990100aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.324+0000 7fb98dd9b640 1 -- 172.21.15.16:0/3735082953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb990101330 con 0x7fb990104ee0 2024-09-19T09:27:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.324+0000 7fb98dd9b640 1 --2- 172.21.15.16:0/3735082953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb990104ee0 0x7fb990100560 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb98400e9c0 tx=0x7fb98400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.324+0000 7fb97f7fe640 1 -- 172.21.15.16:0/3735082953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98400cd80 con 0x7fb990104ee0 2024-09-19T09:27:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.324+0000 7fb97f7fe640 1 -- 172.21.15.16:0/3735082953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb984005500 con 0x7fb990104ee0 2024-09-19T09:27:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.324+0000 7fb97f7fe640 1 -- 172.21.15.16:0/3735082953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984010690 con 0x7fb990104ee0 2024-09-19T09:27:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.324+0000 7fb994933640 1 -- 172.21.15.16:0/3735082953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9901a2080 con 0x7fb990104ee0 2024-09-19T09:27:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.324+0000 7fb994933640 1 -- 172.21.15.16:0/3735082953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9901a2620 con 0x7fb990104ee0 2024-09-19T09:27:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.325+0000 7fb994933640 1 -- 172.21.15.16:0/3735082953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9900fdca0 con 0x7fb990104ee0 2024-09-19T09:27:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.327+0000 7fb97f7fe640 1 -- 172.21.15.16:0/3735082953 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb9840046c0 con 0x7fb990104ee0 2024-09-19T09:27:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.327+0000 7fb97f7fe640 1 --2- 172.21.15.16:0/3735082953 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9780a32d0 0x7fb9780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.327+0000 7fb97f7fe640 1 -- 172.21.15.16:0/3735082953 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb9840cfd40 con 0x7fb990104ee0 2024-09-19T09:27:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.327+0000 7fb98e59c640 1 --2- 172.21.15.16:0/3735082953 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9780a32d0 0x7fb9780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.328+0000 7fb98e59c640 1 --2- 172.21.15.16:0/3735082953 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9780a32d0 0x7fb9780a5790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb98002a930 tx=0x7fb980000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:19.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.332+0000 7fb97f7fe640 1 -- 172.21.15.16:0/3735082953 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb98409ca60 con 0x7fb990104ee0 2024-09-19T09:27:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.508+0000 7fb994933640 1 -- 172.21.15.16:0/3735082953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb990062df0 con 0x7fb990104ee0 2024-09-19T09:27:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.508+0000 7fb97f7fe640 1 -- 172.21.15.16:0/3735082953 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fb98409c360 con 0x7fb990104ee0 2024-09-19T09:27:19.510 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-19T09:27:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.510+0000 7fb994933640 1 -- 172.21.15.16:0/3735082953 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9780a32d0 msgr2=0x7fb9780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.510+0000 7fb994933640 1 --2- 172.21.15.16:0/3735082953 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9780a32d0 0x7fb9780a5790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb98002a930 tx=0x7fb980000bc0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.510+0000 7fb994933640 1 -- 172.21.15.16:0/3735082953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb990104ee0 msgr2=0x7fb990100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.510+0000 7fb994933640 1 --2- 172.21.15.16:0/3735082953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb990104ee0 0x7fb990100560 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb98400e9c0 tx=0x7fb98400ee80 comp rx=0 tx=0).stop 2024-09-19T09:27:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.510+0000 7fb994933640 1 -- 172.21.15.16:0/3735082953 shutdown_connections 2024-09-19T09:27:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.510+0000 7fb994933640 1 --2- 172.21.15.16:0/3735082953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb990107810 0x7fb990100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.510+0000 7fb994933640 1 --2- 172.21.15.16:0/3735082953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb990104ee0 0x7fb990100560 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.510+0000 7fb994933640 1 --2- 172.21.15.16:0/3735082953 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb9780a32d0 0x7fb9780a5790 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.510+0000 7fb994933640 1 --2- 172.21.15.16:0/3735082953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9901025b0 0x7fb990101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.510+0000 7fb994933640 1 -- 172.21.15.16:0/3735082953 >> 172.21.15.16:0/3735082953 conn(0x7fb9900fc1b0 msgr2=0x7fb990102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.510+0000 7fb994933640 1 -- 172.21.15.16:0/3735082953 shutdown_connections 2024-09-19T09:27:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.510+0000 7fb994933640 1 -- 172.21.15.16:0/3735082953 wait complete. 2024-09-19T09:27:19.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T09:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.628+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1155197451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e41025b0 msgr2=0x7fb4e4108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.628+0000 7fb4e88c3640 1 --2- 172.21.15.16:0/1155197451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e41025b0 0x7fb4e4108e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009900 tx=0x7fb4dc02a410 comp rx=0 tx=0).stop 2024-09-19T09:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.629+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1155197451 shutdown_connections 2024-09-19T09:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.629+0000 7fb4e88c3640 1 --2- 172.21.15.16:0/1155197451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e41025b0 0x7fb4e4108e40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.629+0000 7fb4e88c3640 1 --2- 172.21.15.16:0/1155197451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4e4101bf0 0x7fb4e4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.629+0000 7fb4e88c3640 1 --2- 172.21.15.16:0/1155197451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4e4100a00 0x7fb4e4100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.629+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1155197451 >> 172.21.15.16:0/1155197451 conn(0x7fb4e40fc140 msgr2=0x7fb4e40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.629+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1155197451 shutdown_connections 2024-09-19T09:27:19.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.629+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1155197451 wait complete. 2024-09-19T09:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.629+0000 7fb4e88c3640 1 Processor -- start 2024-09-19T09:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.629+0000 7fb4e88c3640 1 -- start start 2024-09-19T09:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.629+0000 7fb4e88c3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4e4100a00 0x7fb4e4197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e88c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e4101bf0 0x7fb4e4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e88c3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4e41025b0 0x7fb4e419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e88c3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e410c660 con 0x7fb4e4101bf0 2024-09-19T09:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e88c3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e410c4e0 con 0x7fb4e4100a00 2024-09-19T09:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e88c3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e410c7e0 con 0x7fb4e41025b0 2024-09-19T09:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e4101bf0 0x7fb4e4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e4101bf0 0x7fb4e4198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39610/0 (socket says 172.21.15.16:39610) 2024-09-19T09:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e1d9b640 1 -- 172.21.15.16:0/1396992812 learned_addr learned my addr 172.21.15.16:0/1396992812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e2d9d640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4e41025b0 0x7fb4e419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e259c640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4e4100a00 0x7fb4e4197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e1d9b640 1 -- 172.21.15.16:0/1396992812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4e41025b0 msgr2=0x7fb4e419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e1d9b640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4e41025b0 0x7fb4e419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e1d9b640 1 -- 172.21.15.16:0/1396992812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4e4100a00 msgr2=0x7fb4e4197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e1d9b640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4e4100a00 0x7fb4e4197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e1d9b640 1 -- 172.21.15.16:0/1396992812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e419fa20 con 0x7fb4e4101bf0 2024-09-19T09:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e1d9b640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e4101bf0 0x7fb4e4198340 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800d950 tx=0x7fb4d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4d37fe640 1 -- 172.21.15.16:0/1396992812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8017030 con 0x7fb4e4101bf0 2024-09-19T09:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4d37fe640 1 -- 172.21.15.16:0/1396992812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d8004020 con 0x7fb4e4101bf0 2024-09-19T09:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1396992812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e419fd10 con 0x7fb4e4101bf0 2024-09-19T09:27:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4d37fe640 1 -- 172.21.15.16:0/1396992812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80047a0 con 0x7fb4e4101bf0 2024-09-19T09:27:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.630+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1396992812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e41a0250 con 0x7fb4e4101bf0 2024-09-19T09:27:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.631+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1396992812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e41a04e0 con 0x7fb4e4101bf0 2024-09-19T09:27:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.633+0000 7fb4d37fe640 1 -- 172.21.15.16:0/1396992812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb4d80041d0 con 0x7fb4e4101bf0 2024-09-19T09:27:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.633+0000 7fb4d37fe640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb4cc0a32d0 0x7fb4cc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.633+0000 7fb4d37fe640 1 -- 172.21.15.16:0/1396992812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb4d80cf8e0 con 0x7fb4e4101bf0 2024-09-19T09:27:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.634+0000 7fb4e259c640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb4cc0a32d0 0x7fb4cc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.634+0000 7fb4e259c640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb4cc0a32d0 0x7fb4cc0a5790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400a300 tx=0x7fb4d400a290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:19.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.637+0000 7fb4d37fe640 1 -- 172.21.15.16:0/1396992812 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb4d809c680 con 0x7fb4e4101bf0 2024-09-19T09:27:19.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.816+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1396992812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb4e41a06b0 con 0x7fb4e4101bf0 2024-09-19T09:27:19.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.817+0000 7fb4d37fe640 1 -- 172.21.15.16:0/1396992812 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7fb4d809bf80 con 0x7fb4e4101bf0 2024-09-19T09:27:19.818 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-19T09:27:19.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.819+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1396992812 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb4cc0a32d0 msgr2=0x7fb4cc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.819+0000 7fb4e88c3640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb4cc0a32d0 0x7fb4cc0a5790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400a300 tx=0x7fb4d400a290 comp rx=0 tx=0).stop 2024-09-19T09:27:19.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.819+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1396992812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e4101bf0 msgr2=0x7fb4e4198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.819+0000 7fb4e88c3640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e4101bf0 0x7fb4e4198340 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800d950 tx=0x7fb4d800de10 comp rx=0 tx=0).stop 2024-09-19T09:27:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.819+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1396992812 shutdown_connections 2024-09-19T09:27:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.819+0000 7fb4e88c3640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb4cc0a32d0 0x7fb4cc0a5790 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.819+0000 7fb4e88c3640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4e41025b0 0x7fb4e419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.819+0000 7fb4e88c3640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e4101bf0 0x7fb4e4198340 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.819+0000 7fb4e88c3640 1 --2- 172.21.15.16:0/1396992812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4e4100a00 0x7fb4e4197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.819+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1396992812 >> 172.21.15.16:0/1396992812 conn(0x7fb4e40fc140 msgr2=0x7fb4e40693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.819+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1396992812 shutdown_connections 2024-09-19T09:27:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.819+0000 7fb4e88c3640 1 -- 172.21.15.16:0/1396992812 wait complete. 2024-09-19T09:27:19.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T09:27:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.937+0000 7f8aacac9640 1 -- 172.21.15.16:0/466757765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8101bf0 msgr2=0x7f8aa8102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.937+0000 7f8aacac9640 1 --2- 172.21.15.16:0/466757765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8101bf0 0x7f8aa8102070 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009900 tx=0x7f8a9002a410 comp rx=0 tx=0).stop 2024-09-19T09:27:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.938+0000 7f8aacac9640 1 -- 172.21.15.16:0/466757765 shutdown_connections 2024-09-19T09:27:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.938+0000 7f8aacac9640 1 --2- 172.21.15.16:0/466757765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa81025b0 0x7f8aa8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.938+0000 7f8aacac9640 1 --2- 172.21.15.16:0/466757765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8101bf0 0x7f8aa8102070 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.938+0000 7f8aacac9640 1 --2- 172.21.15.16:0/466757765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa8100a00 0x7f8aa8100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.938+0000 7f8aacac9640 1 -- 172.21.15.16:0/466757765 >> 172.21.15.16:0/466757765 conn(0x7f8aa80fc140 msgr2=0x7f8aa80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.938+0000 7f8aacac9640 1 -- 172.21.15.16:0/466757765 shutdown_connections 2024-09-19T09:27:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.938+0000 7f8aacac9640 1 -- 172.21.15.16:0/466757765 wait complete. 2024-09-19T09:27:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.938+0000 7f8aacac9640 1 Processor -- start 2024-09-19T09:27:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.938+0000 7f8aacac9640 1 -- start start 2024-09-19T09:27:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.938+0000 7f8aacac9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa8100a00 0x7f8aa8197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aacac9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8101bf0 0x7f8aa8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aacac9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa81025b0 0x7f8aa819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aacac9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa810d7d0 con 0x7f8aa8101bf0 2024-09-19T09:27:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aacac9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa810d650 con 0x7f8aa81025b0 2024-09-19T09:27:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aacac9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa810d950 con 0x7f8aa8100a00 2024-09-19T09:27:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aa5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8101bf0 0x7f8aa8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aa5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8101bf0 0x7f8aa8198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39622/0 (socket says 172.21.15.16:39622) 2024-09-19T09:27:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aa5d9b640 1 -- 172.21.15.16:0/1785383750 learned_addr learned my addr 172.21.15.16:0/1785383750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aa659c640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa8100a00 0x7f8aa8197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aa6d9d640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa81025b0 0x7f8aa819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aa5d9b640 1 -- 172.21.15.16:0/1785383750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa8100a00 msgr2=0x7f8aa8197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aa5d9b640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa8100a00 0x7f8aa8197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aa5d9b640 1 -- 172.21.15.16:0/1785383750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa81025b0 msgr2=0x7f8aa819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aa5d9b640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa81025b0 0x7f8aa819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aa5d9b640 1 -- 172.21.15.16:0/1785383750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa819fa20 con 0x7f8aa8101bf0 2024-09-19T09:27:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aa5d9b640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8101bf0 0x7f8aa8198340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009230 tx=0x7f8a90031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8a9f7fe640 1 -- 172.21.15.16:0/1785383750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9003f030 con 0x7f8aa8101bf0 2024-09-19T09:27:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8a9f7fe640 1 -- 172.21.15.16:0/1785383750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a90003ad0 con 0x7f8aa8101bf0 2024-09-19T09:27:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aacac9640 1 -- 172.21.15.16:0/1785383750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa819fcb0 con 0x7f8aa8101bf0 2024-09-19T09:27:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8a9f7fe640 1 -- 172.21.15.16:0/1785383750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90035070 con 0x7f8aa8101bf0 2024-09-19T09:27:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.939+0000 7f8aacac9640 1 -- 172.21.15.16:0/1785383750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa81a01a0 con 0x7f8aa8101bf0 2024-09-19T09:27:19.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.940+0000 7f8aacac9640 1 -- 172.21.15.16:0/1785383750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa8107de0 con 0x7f8aa8101bf0 2024-09-19T09:27:19.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.942+0000 7f8a9f7fe640 1 -- 172.21.15.16:0/1785383750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8a90042030 con 0x7f8aa8101bf0 2024-09-19T09:27:19.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.942+0000 7f8a9f7fe640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8a940a32d0 0x7f8a940a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:19.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.943+0000 7f8aa659c640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8a940a32d0 0x7f8a940a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.943+0000 7f8a9f7fe640 1 -- 172.21.15.16:0/1785383750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8a900efa30 con 0x7f8aa8101bf0 2024-09-19T09:27:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.943+0000 7f8aa659c640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8a940a32d0 0x7f8a940a5790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8101a50 tx=0x7f8aa0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:19.947+0000 7f8a9f7fe640 1 -- 172.21.15.16:0/1785383750 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8a900bc0c0 con 0x7f8aa8101bf0 2024-09-19T09:27:20.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.121+0000 7f8aacac9640 1 -- 172.21.15.16:0/1785383750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8aa8062df0 con 0x7f8aa8101bf0 2024-09-19T09:27:20.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.122+0000 7f8a9f7fe640 1 -- 172.21.15.16:0/1785383750 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7f8a900c03c0 con 0x7f8aa8101bf0 2024-09-19T09:27:20.123 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-19T09:27:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.124+0000 7f8aacac9640 1 -- 172.21.15.16:0/1785383750 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8a940a32d0 msgr2=0x7f8a940a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.124+0000 7f8aacac9640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8a940a32d0 0x7f8a940a5790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8101a50 tx=0x7f8aa0005f90 comp rx=0 tx=0).stop 2024-09-19T09:27:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.124+0000 7f8aacac9640 1 -- 172.21.15.16:0/1785383750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8101bf0 msgr2=0x7f8aa8198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.124+0000 7f8aacac9640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8101bf0 0x7f8aa8198340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009230 tx=0x7f8a90031b60 comp rx=0 tx=0).stop 2024-09-19T09:27:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.124+0000 7f8aacac9640 1 -- 172.21.15.16:0/1785383750 shutdown_connections 2024-09-19T09:27:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.124+0000 7f8aacac9640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8aa81025b0 0x7f8aa819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.124+0000 7f8aacac9640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa8101bf0 0x7f8aa8198340 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.124+0000 7f8aacac9640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8a940a32d0 0x7f8a940a5790 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.124+0000 7f8aacac9640 1 --2- 172.21.15.16:0/1785383750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa8100a00 0x7f8aa8197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.124+0000 7f8aacac9640 1 -- 172.21.15.16:0/1785383750 >> 172.21.15.16:0/1785383750 conn(0x7f8aa80fc140 msgr2=0x7f8aa8104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.124+0000 7f8aacac9640 1 -- 172.21.15.16:0/1785383750 shutdown_connections 2024-09-19T09:27:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.124+0000 7f8aacac9640 1 -- 172.21.15.16:0/1785383750 wait complete. 2024-09-19T09:27:20.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:19 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/850019119' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T09:27:20.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:19 smithi016 ceph-mon[30979]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:27:20.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:19 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/950829107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:27:20.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:19 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3735082953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:27:20.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:19 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1396992812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:27:20.136 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T09:27:20.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:27:20.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:19 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/850019119' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T09:27:20.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:19 smithi097 ceph-mon[34864]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:27:20.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:19 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/950829107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:27:20.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:19 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3735082953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:27:20.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:19 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1396992812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:27:20.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:19 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/850019119' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T09:27:20.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:19 smithi142 ceph-mon[35032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T09:27:20.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:19 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/950829107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:27:20.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:19 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3735082953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:27:20.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:19 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1396992812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:27:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.817+0000 7f6f8919f640 1 -- 172.21.15.16:0/2282468875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f84103740 msgr2=0x7f6f84107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.817+0000 7f6f8919f640 1 --2- 172.21.15.16:0/2282468875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f84103740 0x7f6f84107b80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009930 tx=0x7f6f7802a420 comp rx=0 tx=0).stop 2024-09-19T09:27:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.817+0000 7f6f8919f640 1 -- 172.21.15.16:0/2282468875 shutdown_connections 2024-09-19T09:27:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.817+0000 7f6f8919f640 1 --2- 172.21.15.16:0/2282468875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f84108290 0x7f6f8410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.817+0000 7f6f8919f640 1 --2- 172.21.15.16:0/2282468875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f84103740 0x7f6f84107b80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.817+0000 7f6f8919f640 1 --2- 172.21.15.16:0/2282468875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f84102d90 0x7f6f84103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.817+0000 7f6f8919f640 1 -- 172.21.15.16:0/2282468875 >> 172.21.15.16:0/2282468875 conn(0x7f6f840fc7d0 msgr2=0x7f6f840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.817+0000 7f6f8919f640 1 -- 172.21.15.16:0/2282468875 shutdown_connections 2024-09-19T09:27:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.817+0000 7f6f8919f640 1 -- 172.21.15.16:0/2282468875 wait complete. 2024-09-19T09:27:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.818+0000 7f6f8919f640 1 Processor -- start 2024-09-19T09:27:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.818+0000 7f6f8919f640 1 -- start start 2024-09-19T09:27:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.818+0000 7f6f8919f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f84102d90 0x7f6f841985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.818+0000 7f6f8919f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f84103740 0x7f6f84198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.818+0000 7f6f8919f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f84108290 0x7f6f8419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.818+0000 7f6f8919f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84112050 con 0x7f6f84103740 2024-09-19T09:27:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.818+0000 7f6f8919f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84111ed0 con 0x7f6f84108290 2024-09-19T09:27:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.818+0000 7f6f8919f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841121d0 con 0x7f6f84102d90 2024-09-19T09:27:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.818+0000 7f6f8259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f84103740 0x7f6f84198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.818+0000 7f6f8259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f84103740 0x7f6f84198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39648/0 (socket says 172.21.15.16:39648) 2024-09-19T09:27:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.818+0000 7f6f8259c640 1 -- 172.21.15.16:0/3038066765 learned_addr learned my addr 172.21.15.16:0/3038066765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f82d9d640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f84102d90 0x7f6f841985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f8359e640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f84108290 0x7f6f8419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f8259c640 1 -- 172.21.15.16:0/3038066765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f84102d90 msgr2=0x7f6f841985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f8259c640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f84102d90 0x7f6f841985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f8259c640 1 -- 172.21.15.16:0/3038066765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f84108290 msgr2=0x7f6f8419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f8259c640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f84108290 0x7f6f8419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f8259c640 1 -- 172.21.15.16:0/3038066765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8419d550 con 0x7f6f84103740 2024-09-19T09:27:20.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f82d9d640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f84102d90 0x7f6f841985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:20.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f8259c640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f84103740 0x7f6f84198ae0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6f78005940 tx=0x7f6f780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:20.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f6bfff640 1 -- 172.21.15.16:0/3038066765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7803f030 con 0x7f6f84103740 2024-09-19T09:27:20.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f8919f640 1 -- 172.21.15.16:0/3038066765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8419d7e0 con 0x7f6f84103740 2024-09-19T09:27:20.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f6bfff640 1 -- 172.21.15.16:0/3038066765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f78003ce0 con 0x7f6f84103740 2024-09-19T09:27:20.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f6bfff640 1 -- 172.21.15.16:0/3038066765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78035070 con 0x7f6f84103740 2024-09-19T09:27:20.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.819+0000 7f6f8919f640 1 -- 172.21.15.16:0/3038066765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f841a5120 con 0x7f6f84103740 2024-09-19T09:27:20.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.820+0000 7f6f8919f640 1 -- 172.21.15.16:0/3038066765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f84103170 con 0x7f6f84103740 2024-09-19T09:27:20.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.821+0000 7f6f6bfff640 1 -- 172.21.15.16:0/3038066765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f6f78042030 con 0x7f6f84103740 2024-09-19T09:27:20.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.822+0000 7f6f6bfff640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6f6c0a3240 0x7f6f6c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:20.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.822+0000 7f6f82d9d640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6f6c0a3240 0x7f6f6c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:20.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.822+0000 7f6f6bfff640 1 -- 172.21.15.16:0/3038066765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6f780efef0 con 0x7f6f84103740 2024-09-19T09:27:20.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.823+0000 7f6f82d9d640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6f6c0a3240 0x7f6f6c0a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6f740058c0 tx=0x7f6f74005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:20.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:20.826+0000 7f6f6bfff640 1 -- 172.21.15.16:0/3038066765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6f780bcc10 con 0x7f6f84103740 2024-09-19T09:27:21.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.038+0000 7f6f8919f640 1 -- 172.21.15.16:0/3038066765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f8410a2a0 con 0x7f6f84103740 2024-09-19T09:27:21.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.039+0000 7f6f6bfff640 1 -- 172.21.15.16:0/3038066765 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f6f780bc510 con 0x7f6f84103740 2024-09-19T09:27:21.040 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:21.040 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:27:21.040 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:27:21.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.041+0000 7f6f8919f640 1 -- 172.21.15.16:0/3038066765 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6f6c0a3240 msgr2=0x7f6f6c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.041+0000 7f6f8919f640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6f6c0a3240 0x7f6f6c0a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6f740058c0 tx=0x7f6f74005850 comp rx=0 tx=0).stop 2024-09-19T09:27:21.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.041+0000 7f6f8919f640 1 -- 172.21.15.16:0/3038066765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f84103740 msgr2=0x7f6f84198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.041+0000 7f6f8919f640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f84103740 0x7f6f84198ae0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6f78005940 tx=0x7f6f780042f0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.041+0000 7f6f8919f640 1 -- 172.21.15.16:0/3038066765 shutdown_connections 2024-09-19T09:27:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.041+0000 7f6f8919f640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f84108290 0x7f6f8419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.041+0000 7f6f8919f640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f84103740 0x7f6f84198ae0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.041+0000 7f6f8919f640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f6f6c0a3240 0x7f6f6c0a5700 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.041+0000 7f6f8919f640 1 --2- 172.21.15.16:0/3038066765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f84102d90 0x7f6f841985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.041+0000 7f6f8919f640 1 -- 172.21.15.16:0/3038066765 >> 172.21.15.16:0/3038066765 conn(0x7f6f840fc7d0 msgr2=0x7f6f84105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.041+0000 7f6f8919f640 1 -- 172.21.15.16:0/3038066765 shutdown_connections 2024-09-19T09:27:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.041+0000 7f6f8919f640 1 -- 172.21.15.16:0/3038066765 wait complete. 2024-09-19T09:27:21.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:20 smithi097 ceph-mon[34864]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T09:27:21.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:20 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1785383750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:27:21.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:20 smithi016 ceph-mon[30979]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T09:27:21.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:20 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1785383750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:27:21.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:20 smithi142 ceph-mon[35032]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-19T09:27:21.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:20 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1785383750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:27:21.830 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T09:27:21.830 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T09:27:21.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T09:27:21.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.940+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/925235443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e8104f80 msgr2=0x7fc2e8107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.940+0000 7fc2ee4ae640 1 --2- 172.21.15.16:0/925235443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e8104f80 0x7fc2e8107370 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009900 tx=0x7fc2dc02a410 comp rx=0 tx=0).stop 2024-09-19T09:27:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.940+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/925235443 shutdown_connections 2024-09-19T09:27:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.940+0000 7fc2ee4ae640 1 --2- 172.21.15.16:0/925235443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e81078b0 0x7fc2e8109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.940+0000 7fc2ee4ae640 1 --2- 172.21.15.16:0/925235443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e8104f80 0x7fc2e8107370 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.940+0000 7fc2ee4ae640 1 --2- 172.21.15.16:0/925235443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2e8102650 0x7fc2e8104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.940+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/925235443 >> 172.21.15.16:0/925235443 conn(0x7fc2e80fc170 msgr2=0x7fc2e80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.940+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/925235443 shutdown_connections 2024-09-19T09:27:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.940+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/925235443 wait complete. 2024-09-19T09:27:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.940+0000 7fc2ee4ae640 1 Processor -- start 2024-09-19T09:27:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.940+0000 7fc2ee4ae640 1 -- start start 2024-09-19T09:27:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2ee4ae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2e8102650 0x7fc2e8101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2ee4ae640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e8104f80 0x7fc2e8100600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2ee4ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e81078b0 0x7fc2e8100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2ee4ae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e810d770 con 0x7fc2e81078b0 2024-09-19T09:27:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2ee4ae640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e810d5f0 con 0x7fc2e8102650 2024-09-19T09:27:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2ee4ae640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e810d8f0 con 0x7fc2e8104f80 2024-09-19T09:27:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2eca4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e81078b0 0x7fc2e8100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2eca4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e81078b0 0x7fc2e8100b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39654/0 (socket says 172.21.15.16:39654) 2024-09-19T09:27:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2eca4b640 1 -- 172.21.15.16:0/3131516667 learned_addr learned my addr 172.21.15.16:0/3131516667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2e7fff640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2e8102650 0x7fc2e8101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2e77fe640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e8104f80 0x7fc2e8100600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2eca4b640 1 -- 172.21.15.16:0/3131516667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e8104f80 msgr2=0x7fc2e8100600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2eca4b640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e8104f80 0x7fc2e8100600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2eca4b640 1 -- 172.21.15.16:0/3131516667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2e8102650 msgr2=0x7fc2e8101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2eca4b640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2e8102650 0x7fc2e8101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2eca4b640 1 -- 172.21.15.16:0/3131516667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e81013d0 con 0x7fc2e81078b0 2024-09-19T09:27:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.941+0000 7fc2eca4b640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e81078b0 0x7fc2e8100b40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000cce0 tx=0x7fc2e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.942+0000 7fc2e57fa640 1 -- 172.21.15.16:0/3131516667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e000f450 con 0x7fc2e81078b0 2024-09-19T09:27:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.942+0000 7fc2e57fa640 1 -- 172.21.15.16:0/3131516667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e000fad0 con 0x7fc2e81078b0 2024-09-19T09:27:21.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.942+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/3131516667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e81a1f70 con 0x7fc2e81078b0 2024-09-19T09:27:21.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.942+0000 7fc2e57fa640 1 -- 172.21.15.16:0/3131516667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0018c10 con 0x7fc2e81078b0 2024-09-19T09:27:21.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.942+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/3131516667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e81a23d0 con 0x7fc2e81078b0 2024-09-19T09:27:21.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.943+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/3131516667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e80fdc60 con 0x7fc2e81078b0 2024-09-19T09:27:21.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.944+0000 7fc2e57fa640 1 -- 172.21.15.16:0/3131516667 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fc2e000f5f0 con 0x7fc2e81078b0 2024-09-19T09:27:21.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.945+0000 7fc2e57fa640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc2d00a32d0 0x7fc2d00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:21.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.945+0000 7fc2e7fff640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc2d00a32d0 0x7fc2d00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:21.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.945+0000 7fc2e57fa640 1 -- 172.21.15.16:0/3131516667 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc2e00cff10 con 0x7fc2e81078b0 2024-09-19T09:27:21.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.945+0000 7fc2e7fff640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc2d00a32d0 0x7fc2d00a5790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8009bd0 tx=0x7fc2d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:21.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:21.949+0000 7fc2e57fa640 1 -- 172.21.15.16:0/3131516667 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc2e0014030 con 0x7fc2e81078b0 2024-09-19T09:27:22.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.123+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/3131516667 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc2e8062630 con 0x7fc2d00a32d0 2024-09-19T09:27:22.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.129+0000 7fc2e57fa640 1 -- 172.21.15.16:0/3131516667 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258890 (secure 0 0 0) 0x7fc2e8062630 con 0x7fc2d00a32d0 2024-09-19T09:27:22.133 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:22.135 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T09:27:22.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.132+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/3131516667 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc2d00a32d0 msgr2=0x7fc2d00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:22.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.132+0000 7fc2ee4ae640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc2d00a32d0 0x7fc2d00a5790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8009bd0 tx=0x7fc2d8005f90 comp rx=0 tx=0).stop 2024-09-19T09:27:22.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.132+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/3131516667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e81078b0 msgr2=0x7fc2e8100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:22.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.132+0000 7fc2ee4ae640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e81078b0 0x7fc2e8100b40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000cce0 tx=0x7fc2e000a620 comp rx=0 tx=0).stop 2024-09-19T09:27:22.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.132+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/3131516667 shutdown_connections 2024-09-19T09:27:22.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.132+0000 7fc2ee4ae640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc2d00a32d0 0x7fc2d00a5790 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.132+0000 7fc2ee4ae640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e8104f80 0x7fc2e8100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.132+0000 7fc2ee4ae640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e81078b0 0x7fc2e8100b40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.132+0000 7fc2ee4ae640 1 --2- 172.21.15.16:0/3131516667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2e8102650 0x7fc2e8101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.132+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/3131516667 >> 172.21.15.16:0/3131516667 conn(0x7fc2e80fc170 msgr2=0x7fc2e8102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:22.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.132+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/3131516667 shutdown_connections 2024-09-19T09:27:22.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.132+0000 7fc2ee4ae640 1 -- 172.21.15.16:0/3131516667 wait complete. 2024-09-19T09:27:22.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:21 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3038066765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:22.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:21 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3038066765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:22.145 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-19T09:27:21.365506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659173,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1146,"num_read_kb":696,"num_write":441,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1182578,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391004,"kb_used_data":12636,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550820,"statfs":{"total":1151940427776,"available":1151540039680,"internally_reserved":0,"allocated":12939264,"data_stored":7123427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":813,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":13,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003207"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347860+0000","last_change":"2024-09-19T09:25:29.294965+0000","last_active":"2024-09-19T09:27:11.347860+0000","last_peered":"2024-09-19T09:27:11.347860+0000","last_clean":"2024-09-19T09:27:11.347860+0000","last_became_active":"2024-09-19T09:25:29.294157+0000","last_became_peered":"2024-09-19T09:25:29.294157+0000","last_unstale":"2024-09-19T09:27:11.347860+0000","last_undegraded":"2024-09-19T09:27:11.347860+0000","last_fullsized":"2024-09-19T09:27:11.347860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:24.643835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335438+0000","last_change":"2024-09-19T09:25:29.298888+0000","last_active":"2024-09-19T09:27:11.335438+0000","last_peered":"2024-09-19T09:27:11.335438+0000","last_clean":"2024-09-19T09:27:11.335438+0000","last_became_active":"2024-09-19T09:25:29.298245+0000","last_became_peered":"2024-09-19T09:25:29.298245+0000","last_unstale":"2024-09-19T09:27:11.335438+0000","last_undegraded":"2024-09-19T09:27:11.335438+0000","last_fullsized":"2024-09-19T09:27:11.335438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:03.162342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344990+0000","last_change":"2024-09-19T09:25:29.292300+0000","last_active":"2024-09-19T09:27:11.344990+0000","last_peered":"2024-09-19T09:27:11.344990+0000","last_clean":"2024-09-19T09:27:11.344990+0000","last_became_active":"2024-09-19T09:25:29.291973+0000","last_became_peered":"2024-09-19T09:25:29.291973+0000","last_unstale":"2024-09-19T09:27:11.344990+0000","last_undegraded":"2024-09-19T09:27:11.344990+0000","last_fullsized":"2024-09-19T09:27:11.344990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:19.672778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342109+0000","last_change":"2024-09-19T09:25:29.305695+0000","last_active":"2024-09-19T09:27:11.342109+0000","last_peered":"2024-09-19T09:27:11.342109+0000","last_clean":"2024-09-19T09:27:11.342109+0000","last_became_active":"2024-09-19T09:25:29.305244+0000","last_became_peered":"2024-09-19T09:25:29.305244+0000","last_unstale":"2024-09-19T09:27:11.342109+0000","last_undegraded":"2024-09-19T09:27:11.342109+0000","last_fullsized":"2024-09-19T09:27:11.342109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:38.054624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336846+0000","last_change":"2024-09-19T09:25:29.295259+0000","last_active":"2024-09-19T09:27:11.336846+0000","last_peered":"2024-09-19T09:27:11.336846+0000","last_clean":"2024-09-19T09:27:11.336846+0000","last_became_active":"2024-09-19T09:25:29.294936+0000","last_became_peered":"2024-09-19T09:25:29.294936+0000","last_unstale":"2024-09-19T09:27:11.336846+0000","last_undegraded":"2024-09-19T09:27:11.336846+0000","last_fullsized":"2024-09-19T09:27:11.336846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:59.193504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336175+0000","last_change":"2024-09-19T09:25:29.299650+0000","last_active":"2024-09-19T09:27:11.336175+0000","last_peered":"2024-09-19T09:27:11.336175+0000","last_clean":"2024-09-19T09:27:11.336175+0000","last_became_active":"2024-09-19T09:25:29.299182+0000","last_became_peered":"2024-09-19T09:25:29.299182+0000","last_unstale":"2024-09-19T09:27:11.336175+0000","last_undegraded":"2024-09-19T09:27:11.336175+0000","last_fullsized":"2024-09-19T09:27:11.336175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:44.637771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342329+0000","last_change":"2024-09-19T09:25:29.302501+0000","last_active":"2024-09-19T09:27:11.342329+0000","last_peered":"2024-09-19T09:27:11.342329+0000","last_clean":"2024-09-19T09:27:11.342329+0000","last_became_active":"2024-09-19T09:25:29.301419+0000","last_became_peered":"2024-09-19T09:25:29.301419+0000","last_unstale":"2024-09-19T09:27:11.342329+0000","last_undegraded":"2024-09-19T09:27:11.342329+0000","last_fullsized":"2024-09-19T09:27:11.342329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:50.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337306+0000","last_change":"2024-09-19T09:25:29.300212+0000","last_active":"2024-09-19T09:27:11.337306+0000","last_peered":"2024-09-19T09:27:11.337306+0000","last_clean":"2024-09-19T09:27:11.337306+0000","last_became_active":"2024-09-19T09:25:29.299873+0000","last_became_peered":"2024-09-19T09:25:29.299873+0000","last_unstale":"2024-09-19T09:27:11.337306+0000","last_undegraded":"2024-09-19T09:27:11.337306+0000","last_fullsized":"2024-09-19T09:27:11.337306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:34.620314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346891+0000","last_change":"2024-09-19T09:25:29.290214+0000","last_active":"2024-09-19T09:27:11.346891+0000","last_peered":"2024-09-19T09:27:11.346891+0000","last_clean":"2024-09-19T09:27:11.346891+0000","last_became_active":"2024-09-19T09:25:29.289707+0000","last_became_peered":"2024-09-19T09:25:29.289707+0000","last_unstale":"2024-09-19T09:27:11.346891+0000","last_undegraded":"2024-09-19T09:27:11.346891+0000","last_fullsized":"2024-09-19T09:27:11.346891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:16.341340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338183+0000","last_change":"2024-09-19T09:25:29.301774+0000","last_active":"2024-09-19T09:27:11.338183+0000","last_peered":"2024-09-19T09:27:11.338183+0000","last_clean":"2024-09-19T09:27:11.338183+0000","last_became_active":"2024-09-19T09:25:29.301398+0000","last_became_peered":"2024-09-19T09:25:29.301398+0000","last_unstale":"2024-09-19T09:27:11.338183+0000","last_undegraded":"2024-09-19T09:27:11.338183+0000","last_fullsized":"2024-09-19T09:27:11.338183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:36.058038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346172+0000","last_change":"2024-09-19T09:25:29.301475+0000","last_active":"2024-09-19T09:27:11.346172+0000","last_peered":"2024-09-19T09:27:11.346172+0000","last_clean":"2024-09-19T09:27:11.346172+0000","last_became_active":"2024-09-19T09:25:29.300908+0000","last_became_peered":"2024-09-19T09:25:29.300908+0000","last_unstale":"2024-09-19T09:27:11.346172+0000","last_undegraded":"2024-09-19T09:27:11.346172+0000","last_fullsized":"2024-09-19T09:27:11.346172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:44.774127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336219+0000","last_change":"2024-09-19T09:25:42.994395+0000","last_active":"2024-09-19T09:27:11.336219+0000","last_peered":"2024-09-19T09:27:11.336219+0000","last_clean":"2024-09-19T09:27:11.336219+0000","last_became_active":"2024-09-19T09:25:42.993992+0000","last_became_peered":"2024-09-19T09:25:42.993992+0000","last_unstale":"2024-09-19T09:27:11.336219+0000","last_undegraded":"2024-09-19T09:27:11.336219+0000","last_fullsized":"2024-09-19T09:27:11.336219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.486069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.340454+0000","last_change":"2024-09-19T09:25:42.989979+0000","last_active":"2024-09-19T09:27:11.340454+0000","last_peered":"2024-09-19T09:27:11.340454+0000","last_clean":"2024-09-19T09:27:11.340454+0000","last_became_active":"2024-09-19T09:25:42.989578+0000","last_became_peered":"2024-09-19T09:25:42.989578+0000","last_unstale":"2024-09-19T09:27:11.340454+0000","last_undegraded":"2024-09-19T09:27:11.340454+0000","last_fullsized":"2024-09-19T09:27:11.340454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:04.583023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336926+0000","last_change":"2024-09-19T09:25:29.299917+0000","last_active":"2024-09-19T09:27:11.336926+0000","last_peered":"2024-09-19T09:27:11.336926+0000","last_clean":"2024-09-19T09:27:11.336926+0000","last_became_active":"2024-09-19T09:25:29.299571+0000","last_became_peered":"2024-09-19T09:25:29.299571+0000","last_unstale":"2024-09-19T09:27:11.336926+0000","last_undegraded":"2024-09-19T09:27:11.336926+0000","last_fullsized":"2024-09-19T09:27:11.336926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:25.142252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335108+0000","last_change":"2024-09-19T09:25:29.299713+0000","last_active":"2024-09-19T09:27:11.335108+0000","last_peered":"2024-09-19T09:27:11.335108+0000","last_clean":"2024-09-19T09:27:11.335108+0000","last_became_active":"2024-09-19T09:25:29.299234+0000","last_became_peered":"2024-09-19T09:25:29.299234+0000","last_unstale":"2024-09-19T09:27:11.335108+0000","last_undegraded":"2024-09-19T09:27:11.335108+0000","last_fullsized":"2024-09-19T09:27:11.335108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:34.714714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346606+0000","last_change":"2024-09-19T09:25:29.293010+0000","last_active":"2024-09-19T09:27:11.346606+0000","last_peered":"2024-09-19T09:27:11.346606+0000","last_clean":"2024-09-19T09:27:11.346606+0000","last_became_active":"2024-09-19T09:25:29.292645+0000","last_became_peered":"2024-09-19T09:25:29.292645+0000","last_unstale":"2024-09-19T09:27:11.346606+0000","last_undegraded":"2024-09-19T09:27:11.346606+0000","last_fullsized":"2024-09-19T09:27:11.346606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:24.749075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347900+0000","last_change":"2024-09-19T09:25:29.294530+0000","last_active":"2024-09-19T09:27:11.347900+0000","last_peered":"2024-09-19T09:27:11.347900+0000","last_clean":"2024-09-19T09:27:11.347900+0000","last_became_active":"2024-09-19T09:25:29.293699+0000","last_became_peered":"2024-09-19T09:25:29.293699+0000","last_unstale":"2024-09-19T09:27:11.347900+0000","last_undegraded":"2024-09-19T09:27:11.347900+0000","last_fullsized":"2024-09-19T09:27:11.347900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:53.491258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336452+0000","last_change":"2024-09-19T09:25:29.290600+0000","last_active":"2024-09-19T09:27:11.336452+0000","last_peered":"2024-09-19T09:27:11.336452+0000","last_clean":"2024-09-19T09:27:11.336452+0000","last_became_active":"2024-09-19T09:25:29.290305+0000","last_became_peered":"2024-09-19T09:25:29.290305+0000","last_unstale":"2024-09-19T09:27:11.336452+0000","last_undegraded":"2024-09-19T09:27:11.336452+0000","last_fullsized":"2024-09-19T09:27:11.336452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:42.809659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338092+0000","last_change":"2024-09-19T09:25:29.300591+0000","last_active":"2024-09-19T09:27:11.338092+0000","last_peered":"2024-09-19T09:27:11.338092+0000","last_clean":"2024-09-19T09:27:11.338092+0000","last_became_active":"2024-09-19T09:25:29.300251+0000","last_became_peered":"2024-09-19T09:25:29.300251+0000","last_unstale":"2024-09-19T09:27:11.338092+0000","last_undegraded":"2024-09-19T09:27:11.338092+0000","last_fullsized":"2024-09-19T09:27:11.338092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:08.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342064+0000","last_change":"2024-09-19T09:25:29.298452+0000","last_active":"2024-09-19T09:27:11.342064+0000","last_peered":"2024-09-19T09:27:11.342064+0000","last_clean":"2024-09-19T09:27:11.342064+0000","last_became_active":"2024-09-19T09:25:29.298137+0000","last_became_peered":"2024-09-19T09:25:29.298137+0000","last_unstale":"2024-09-19T09:27:11.342064+0000","last_undegraded":"2024-09-19T09:27:11.342064+0000","last_fullsized":"2024-09-19T09:27:11.342064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:11.716228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337968+0000","last_change":"2024-09-19T09:25:29.294278+0000","last_active":"2024-09-19T09:27:11.337968+0000","last_peered":"2024-09-19T09:27:11.337968+0000","last_clean":"2024-09-19T09:27:11.337968+0000","last_became_active":"2024-09-19T09:25:29.293972+0000","last_became_peered":"2024-09-19T09:25:29.293972+0000","last_unstale":"2024-09-19T09:27:11.337968+0000","last_undegraded":"2024-09-19T09:27:11.337968+0000","last_fullsized":"2024-09-19T09:27:11.337968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:27.923839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346634+0000","last_change":"2024-09-19T09:25:29.294000+0000","last_active":"2024-09-19T09:27:11.346634+0000","last_peered":"2024-09-19T09:27:11.346634+0000","last_clean":"2024-09-19T09:27:11.346634+0000","last_became_active":"2024-09-19T09:25:29.293486+0000","last_became_peered":"2024-09-19T09:25:29.293486+0000","last_unstale":"2024-09-19T09:27:11.346634+0000","last_undegraded":"2024-09-19T09:27:11.346634+0000","last_fullsized":"2024-09-19T09:27:11.346634+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:44.053620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334040+0000","last_change":"2024-09-19T09:25:42.994951+0000","last_active":"2024-09-19T09:27:11.334040+0000","last_peered":"2024-09-19T09:27:11.334040+0000","last_clean":"2024-09-19T09:27:11.334040+0000","last_became_active":"2024-09-19T09:25:42.994588+0000","last_became_peered":"2024-09-19T09:25:42.994588+0000","last_unstale":"2024-09-19T09:27:11.334040+0000","last_undegraded":"2024-09-19T09:27:11.334040+0000","last_fullsized":"2024-09-19T09:27:11.334040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:10.141169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334658+0000","last_change":"2024-09-19T09:25:29.286395+0000","last_active":"2024-09-19T09:27:11.334658+0000","last_peered":"2024-09-19T09:27:11.334658+0000","last_clean":"2024-09-19T09:27:11.334658+0000","last_became_active":"2024-09-19T09:25:29.286048+0000","last_became_peered":"2024-09-19T09:25:29.286048+0000","last_unstale":"2024-09-19T09:27:11.334658+0000","last_undegraded":"2024-09-19T09:27:11.334658+0000","last_fullsized":"2024-09-19T09:27:11.334658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:43.121166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.327390+0000","last_change":"2024-09-19T09:25:29.298458+0000","last_active":"2024-09-19T09:27:11.327390+0000","last_peered":"2024-09-19T09:27:11.327390+0000","last_clean":"2024-09-19T09:27:11.327390+0000","last_became_active":"2024-09-19T09:25:29.297993+0000","last_became_peered":"2024-09-19T09:25:29.297993+0000","last_unstale":"2024-09-19T09:27:11.327390+0000","last_undegraded":"2024-09-19T09:27:11.327390+0000","last_fullsized":"2024-09-19T09:27:11.327390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:44.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344574+0000","last_change":"2024-09-19T09:25:29.297167+0000","last_active":"2024-09-19T09:27:11.344574+0000","last_peered":"2024-09-19T09:27:11.344574+0000","last_clean":"2024-09-19T09:27:11.344574+0000","last_became_active":"2024-09-19T09:25:29.296777+0000","last_became_peered":"2024-09-19T09:25:29.296777+0000","last_unstale":"2024-09-19T09:27:11.344574+0000","last_undegraded":"2024-09-19T09:27:11.344574+0000","last_fullsized":"2024-09-19T09:27:11.344574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:40.581643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.339599+0000","last_change":"2024-09-19T09:25:29.296169+0000","last_active":"2024-09-19T09:27:11.339599+0000","last_peered":"2024-09-19T09:27:11.339599+0000","last_clean":"2024-09-19T09:27:11.339599+0000","last_became_active":"2024-09-19T09:25:29.295806+0000","last_became_peered":"2024-09-19T09:25:29.295806+0000","last_unstale":"2024-09-19T09:27:11.339599+0000","last_undegraded":"2024-09-19T09:27:11.339599+0000","last_fullsized":"2024-09-19T09:27:11.339599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:39.639884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334590+0000","last_change":"2024-09-19T09:25:29.297960+0000","last_active":"2024-09-19T09:27:11.334590+0000","last_peered":"2024-09-19T09:27:11.334590+0000","last_clean":"2024-09-19T09:27:11.334590+0000","last_became_active":"2024-09-19T09:25:29.297577+0000","last_became_peered":"2024-09-19T09:25:29.297577+0000","last_unstale":"2024-09-19T09:27:11.334590+0000","last_undegraded":"2024-09-19T09:27:11.334590+0000","last_fullsized":"2024-09-19T09:27:11.334590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:27.035504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335188+0000","last_change":"2024-09-19T09:25:29.297896+0000","last_active":"2024-09-19T09:27:11.335188+0000","last_peered":"2024-09-19T09:27:11.335188+0000","last_clean":"2024-09-19T09:27:11.335188+0000","last_became_active":"2024-09-19T09:25:29.297594+0000","last_became_peered":"2024-09-19T09:25:29.297594+0000","last_unstale":"2024-09-19T09:27:11.335188+0000","last_undegraded":"2024-09-19T09:27:11.335188+0000","last_fullsized":"2024-09-19T09:27:11.335188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.366325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334560+0000","last_change":"2024-09-19T09:25:29.301851+0000","last_active":"2024-09-19T09:27:11.334560+0000","last_peered":"2024-09-19T09:27:11.334560+0000","last_clean":"2024-09-19T09:27:11.334560+0000","last_became_active":"2024-09-19T09:25:29.299282+0000","last_became_peered":"2024-09-19T09:25:29.299282+0000","last_unstale":"2024-09-19T09:27:11.334560+0000","last_undegraded":"2024-09-19T09:27:11.334560+0000","last_fullsized":"2024-09-19T09:27:11.334560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:00.382114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342990+0000","last_change":"2024-09-19T09:25:29.303316+0000","last_active":"2024-09-19T09:27:11.342990+0000","last_peered":"2024-09-19T09:27:11.342990+0000","last_clean":"2024-09-19T09:27:11.342990+0000","last_became_active":"2024-09-19T09:25:29.302994+0000","last_became_peered":"2024-09-19T09:25:29.302994+0000","last_unstale":"2024-09-19T09:27:11.342990+0000","last_undegraded":"2024-09-19T09:27:11.342990+0000","last_fullsized":"2024-09-19T09:27:11.342990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:46.233991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341767+0000","last_change":"2024-09-19T09:25:29.293004+0000","last_active":"2024-09-19T09:27:11.341767+0000","last_peered":"2024-09-19T09:27:11.341767+0000","last_clean":"2024-09-19T09:27:11.341767+0000","last_became_active":"2024-09-19T09:25:29.292657+0000","last_became_peered":"2024-09-19T09:25:29.292657+0000","last_unstale":"2024-09-19T09:27:11.341767+0000","last_undegraded":"2024-09-19T09:27:11.341767+0000","last_fullsized":"2024-09-19T09:27:11.341767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:50.966477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.332473+0000","last_change":"2024-09-19T09:25:29.300434+0000","last_active":"2024-09-19T09:27:11.332473+0000","last_peered":"2024-09-19T09:27:11.332473+0000","last_clean":"2024-09-19T09:27:11.332473+0000","last_became_active":"2024-09-19T09:25:29.300053+0000","last_became_peered":"2024-09-19T09:25:29.300053+0000","last_unstale":"2024-09-19T09:27:11.332473+0000","last_undegraded":"2024-09-19T09:27:11.332473+0000","last_fullsized":"2024-09-19T09:27:11.332473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:13.283298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":132,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.106724+0000","last_change":"2024-09-19T09:25:27.280792+0000","last_active":"2024-09-19T09:27:16.106724+0000","last_peered":"2024-09-19T09:27:16.106724+0000","last_clean":"2024-09-19T09:27:16.106724+0000","last_became_active":"2024-09-19T09:25:27.280443+0000","last_became_peered":"2024-09-19T09:25:27.280443+0000","last_unstale":"2024-09-19T09:27:16.106724+0000","last_undegraded":"2024-09-19T09:27:16.106724+0000","last_fullsized":"2024-09-19T09:27:16.106724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.599512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334165+0000","last_change":"2024-09-19T09:25:29.298975+0000","last_active":"2024-09-19T09:27:11.334165+0000","last_peered":"2024-09-19T09:27:11.334165+0000","last_clean":"2024-09-19T09:27:11.334165+0000","last_became_active":"2024-09-19T09:25:29.298476+0000","last_became_peered":"2024-09-19T09:25:29.298476+0000","last_unstale":"2024-09-19T09:27:11.334165+0000","last_undegraded":"2024-09-19T09:27:11.334165+0000","last_fullsized":"2024-09-19T09:27:11.334165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:09.658894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986623+0000","last_change":"2024-09-19T09:25:27.278257+0000","last_active":"2024-09-19T09:25:42.986623+0000","last_peered":"2024-09-19T09:25:42.986623+0000","last_clean":"2024-09-19T09:25:42.986623+0000","last_became_active":"2024-09-19T09:25:27.277897+0000","last_became_peered":"2024-09-19T09:25:27.277897+0000","last_unstale":"2024-09-19T09:25:42.986623+0000","last_undegraded":"2024-09-19T09:25:42.986623+0000","last_fullsized":"2024-09-19T09:25:42.986623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:47.550314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985617+0000","last_change":"2024-09-19T09:25:27.272215+0000","last_active":"2024-09-19T09:25:42.985617+0000","last_peered":"2024-09-19T09:25:42.985617+0000","last_clean":"2024-09-19T09:25:42.985617+0000","last_became_active":"2024-09-19T09:25:27.271821+0000","last_became_peered":"2024-09-19T09:25:27.271821+0000","last_unstale":"2024-09-19T09:25:42.985617+0000","last_undegraded":"2024-09-19T09:25:42.985617+0000","last_fullsized":"2024-09-19T09:25:42.985617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:30.615730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.329744+0000","last_change":"2024-09-19T09:25:29.289347+0000","last_active":"2024-09-19T09:27:11.329744+0000","last_peered":"2024-09-19T09:27:11.329744+0000","last_clean":"2024-09-19T09:27:11.329744+0000","last_became_active":"2024-09-19T09:25:29.289015+0000","last_became_peered":"2024-09-19T09:25:29.289015+0000","last_unstale":"2024-09-19T09:27:11.329744+0000","last_undegraded":"2024-09-19T09:27:11.329744+0000","last_fullsized":"2024-09-19T09:27:11.329744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:18.419334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995432+0000","last_change":"2024-09-19T09:25:27.277342+0000","last_active":"2024-09-19T09:25:42.995432+0000","last_peered":"2024-09-19T09:25:42.995432+0000","last_clean":"2024-09-19T09:25:42.995432+0000","last_became_active":"2024-09-19T09:25:27.277016+0000","last_became_peered":"2024-09-19T09:25:27.277016+0000","last_unstale":"2024-09-19T09:25:42.995432+0000","last_undegraded":"2024-09-19T09:25:42.995432+0000","last_fullsized":"2024-09-19T09:25:42.995432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:56.736156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336629+0000","last_change":"2024-09-19T09:25:29.290924+0000","last_active":"2024-09-19T09:27:11.336629+0000","last_peered":"2024-09-19T09:27:11.336629+0000","last_clean":"2024-09-19T09:27:11.336629+0000","last_became_active":"2024-09-19T09:25:29.290482+0000","last_became_peered":"2024-09-19T09:25:29.290482+0000","last_unstale":"2024-09-19T09:27:11.336629+0000","last_undegraded":"2024-09-19T09:27:11.336629+0000","last_fullsized":"2024-09-19T09:27:11.336629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:16.265040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'6","reported_seq":222,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.105891+0000","last_change":"2024-09-19T09:25:27.277604+0000","last_active":"2024-09-19T09:27:16.105891+0000","last_peered":"2024-09-19T09:27:16.105891+0000","last_clean":"2024-09-19T09:27:16.105891+0000","last_became_active":"2024-09-19T09:25:27.277271+0000","last_became_peered":"2024-09-19T09:25:27.277271+0000","last_unstale":"2024-09-19T09:27:16.105891+0000","last_undegraded":"2024-09-19T09:27:16.105891+0000","last_fullsized":"2024-09-19T09:27:16.105891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:32.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347371+0000","last_change":"2024-09-19T09:25:29.290537+0000","last_active":"2024-09-19T09:27:11.347371+0000","last_peered":"2024-09-19T09:27:11.347371+0000","last_clean":"2024-09-19T09:27:11.347371+0000","last_became_active":"2024-09-19T09:25:29.287984+0000","last_became_peered":"2024-09-19T09:25:29.287984+0000","last_unstale":"2024-09-19T09:27:11.347371+0000","last_undegraded":"2024-09-19T09:27:11.347371+0000","last_fullsized":"2024-09-19T09:27:11.347371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:26.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990018+0000","last_change":"2024-09-19T09:25:27.278560+0000","last_active":"2024-09-19T09:25:42.990018+0000","last_peered":"2024-09-19T09:25:42.990018+0000","last_clean":"2024-09-19T09:25:42.990018+0000","last_became_active":"2024-09-19T09:25:27.277950+0000","last_became_peered":"2024-09-19T09:25:27.277950+0000","last_unstale":"2024-09-19T09:25:42.990018+0000","last_undegraded":"2024-09-19T09:25:42.990018+0000","last_fullsized":"2024-09-19T09:25:42.990018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:28.387725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.339317+0000","last_change":"2024-09-19T09:25:29.298897+0000","last_active":"2024-09-19T09:27:11.339317+0000","last_peered":"2024-09-19T09:27:11.339317+0000","last_clean":"2024-09-19T09:27:11.339317+0000","last_became_active":"2024-09-19T09:25:29.298617+0000","last_became_peered":"2024-09-19T09:25:29.298617+0000","last_unstale":"2024-09-19T09:27:11.339317+0000","last_undegraded":"2024-09-19T09:27:11.339317+0000","last_fullsized":"2024-09-19T09:27:11.339317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:53.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994870+0000","last_change":"2024-09-19T09:25:27.324871+0000","last_active":"2024-09-19T09:25:42.994870+0000","last_peered":"2024-09-19T09:25:42.994870+0000","last_clean":"2024-09-19T09:25:42.994870+0000","last_became_active":"2024-09-19T09:25:27.324471+0000","last_became_peered":"2024-09-19T09:25:27.324471+0000","last_unstale":"2024-09-19T09:25:42.994870+0000","last_undegraded":"2024-09-19T09:25:42.994870+0000","last_fullsized":"2024-09-19T09:25:42.994870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:35.936413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347260+0000","last_change":"2024-09-19T09:25:29.298332+0000","last_active":"2024-09-19T09:27:11.347260+0000","last_peered":"2024-09-19T09:27:11.347260+0000","last_clean":"2024-09-19T09:27:11.347260+0000","last_became_active":"2024-09-19T09:25:29.298006+0000","last_became_peered":"2024-09-19T09:25:29.298006+0000","last_unstale":"2024-09-19T09:27:11.347260+0000","last_undegraded":"2024-09-19T09:27:11.347260+0000","last_fullsized":"2024-09-19T09:27:11.347260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:02.621821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'33","reported_seq":261,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.107843+0000","last_change":"2024-09-19T09:25:27.279528+0000","last_active":"2024-09-19T09:27:16.107843+0000","last_peered":"2024-09-19T09:27:16.107843+0000","last_clean":"2024-09-19T09:27:16.107843+0000","last_became_active":"2024-09-19T09:25:27.279149+0000","last_became_peered":"2024-09-19T09:25:27.279149+0000","last_unstale":"2024-09-19T09:27:16.107843+0000","last_undegraded":"2024-09-19T09:27:16.107843+0000","last_fullsized":"2024-09-19T09:27:16.107843+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:47.030736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60013,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":206,"num_read_kb":161,"num_write":35,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.324455+0000","last_change":"2024-09-19T09:25:29.297538+0000","last_active":"2024-09-19T09:27:11.324455+0000","last_peered":"2024-09-19T09:27:11.324455+0000","last_clean":"2024-09-19T09:27:11.324455+0000","last_became_active":"2024-09-19T09:25:29.297181+0000","last_became_peered":"2024-09-19T09:25:29.297181+0000","last_unstale":"2024-09-19T09:27:11.324455+0000","last_undegraded":"2024-09-19T09:27:11.324455+0000","last_fullsized":"2024-09-19T09:27:11.324455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:53.135438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992860+0000","last_change":"2024-09-19T09:25:27.277090+0000","last_active":"2024-09-19T09:25:42.992860+0000","last_peered":"2024-09-19T09:25:42.992860+0000","last_clean":"2024-09-19T09:25:42.992860+0000","last_became_active":"2024-09-19T09:25:27.276718+0000","last_became_peered":"2024-09-19T09:25:27.276718+0000","last_unstale":"2024-09-19T09:25:42.992860+0000","last_undegraded":"2024-09-19T09:25:42.992860+0000","last_fullsized":"2024-09-19T09:25:42.992860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:25.664261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.325840+0000","last_change":"2024-09-19T09:25:29.294393+0000","last_active":"2024-09-19T09:27:11.325840+0000","last_peered":"2024-09-19T09:27:11.325840+0000","last_clean":"2024-09-19T09:27:11.325840+0000","last_became_active":"2024-09-19T09:25:29.294099+0000","last_became_peered":"2024-09-19T09:25:29.294099+0000","last_unstale":"2024-09-19T09:27:11.325840+0000","last_undegraded":"2024-09-19T09:27:11.325840+0000","last_fullsized":"2024-09-19T09:27:11.325840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:17.333736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995398+0000","last_change":"2024-09-19T09:25:27.278593+0000","last_active":"2024-09-19T09:25:42.995398+0000","last_peered":"2024-09-19T09:25:42.995398+0000","last_clean":"2024-09-19T09:25:42.995398+0000","last_became_active":"2024-09-19T09:25:27.278232+0000","last_became_peered":"2024-09-19T09:25:27.278232+0000","last_unstale":"2024-09-19T09:25:42.995398+0000","last_undegraded":"2024-09-19T09:25:42.995398+0000","last_fullsized":"2024-09-19T09:25:42.995398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:05.852671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343581+0000","last_change":"2024-09-19T09:25:29.300024+0000","last_active":"2024-09-19T09:27:11.343581+0000","last_peered":"2024-09-19T09:27:11.343581+0000","last_clean":"2024-09-19T09:27:11.343581+0000","last_became_active":"2024-09-19T09:25:29.299683+0000","last_became_peered":"2024-09-19T09:25:29.299683+0000","last_unstale":"2024-09-19T09:27:11.343581+0000","last_undegraded":"2024-09-19T09:27:11.343581+0000","last_fullsized":"2024-09-19T09:27:11.343581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:42.249399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'7","reported_seq":131,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:15.107233+0000","last_change":"2024-09-19T09:25:27.272739+0000","last_active":"2024-09-19T09:27:15.107233+0000","last_peered":"2024-09-19T09:27:15.107233+0000","last_clean":"2024-09-19T09:27:15.107233+0000","last_became_active":"2024-09-19T09:25:27.272354+0000","last_became_peered":"2024-09-19T09:25:27.272354+0000","last_unstale":"2024-09-19T09:27:15.107233+0000","last_undegraded":"2024-09-19T09:27:15.107233+0000","last_fullsized":"2024-09-19T09:27:15.107233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:27.863840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343011+0000","last_change":"2024-09-19T09:25:29.303460+0000","last_active":"2024-09-19T09:27:11.343011+0000","last_peered":"2024-09-19T09:27:11.343011+0000","last_clean":"2024-09-19T09:27:11.343011+0000","last_became_active":"2024-09-19T09:25:29.303127+0000","last_became_peered":"2024-09-19T09:25:29.303127+0000","last_unstale":"2024-09-19T09:27:11.343011+0000","last_undegraded":"2024-09-19T09:27:11.343011+0000","last_fullsized":"2024-09-19T09:27:11.343011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:35.313924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990876+0000","last_change":"2024-09-19T09:25:27.323238+0000","last_active":"2024-09-19T09:25:42.990876+0000","last_peered":"2024-09-19T09:25:42.990876+0000","last_clean":"2024-09-19T09:25:42.990876+0000","last_became_active":"2024-09-19T09:25:27.322868+0000","last_became_peered":"2024-09-19T09:25:27.322868+0000","last_unstale":"2024-09-19T09:25:42.990876+0000","last_undegraded":"2024-09-19T09:25:42.990876+0000","last_fullsized":"2024-09-19T09:25:42.990876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:22.890280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338463+0000","last_change":"2024-09-19T09:25:43.005569+0000","last_active":"2024-09-19T09:27:11.338463+0000","last_peered":"2024-09-19T09:27:11.338463+0000","last_clean":"2024-09-19T09:27:11.338463+0000","last_became_active":"2024-09-19T09:25:43.005143+0000","last_became_peered":"2024-09-19T09:25:43.005143+0000","last_unstale":"2024-09-19T09:27:11.338463+0000","last_undegraded":"2024-09-19T09:27:11.338463+0000","last_fullsized":"2024-09-19T09:27:11.338463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:26.297013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991438+0000","last_change":"2024-09-19T09:25:27.282737+0000","last_active":"2024-09-19T09:25:42.991438+0000","last_peered":"2024-09-19T09:25:42.991438+0000","last_clean":"2024-09-19T09:25:42.991438+0000","last_became_active":"2024-09-19T09:25:27.281785+0000","last_became_peered":"2024-09-19T09:25:27.281785+0000","last_unstale":"2024-09-19T09:25:42.991438+0000","last_undegraded":"2024-09-19T09:25:42.991438+0000","last_fullsized":"2024-09-19T09:25:42.991438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:41.552206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.340095+0000","last_change":"2024-09-19T09:25:29.300168+0000","last_active":"2024-09-19T09:27:11.340095+0000","last_peered":"2024-09-19T09:27:11.340095+0000","last_clean":"2024-09-19T09:27:11.340095+0000","last_became_active":"2024-09-19T09:25:29.299811+0000","last_became_peered":"2024-09-19T09:25:29.299811+0000","last_unstale":"2024-09-19T09:27:11.340095+0000","last_undegraded":"2024-09-19T09:27:11.340095+0000","last_fullsized":"2024-09-19T09:27:11.340095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:06.691259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985783+0000","last_change":"2024-09-19T09:25:27.278433+0000","last_active":"2024-09-19T09:25:42.985783+0000","last_peered":"2024-09-19T09:25:42.985783+0000","last_clean":"2024-09-19T09:25:42.985783+0000","last_became_active":"2024-09-19T09:25:27.278076+0000","last_became_peered":"2024-09-19T09:25:27.278076+0000","last_unstale":"2024-09-19T09:25:42.985783+0000","last_undegraded":"2024-09-19T09:25:42.985783+0000","last_fullsized":"2024-09-19T09:25:42.985783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:53.799265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338515+0000","last_change":"2024-09-19T09:25:29.295650+0000","last_active":"2024-09-19T09:27:11.338515+0000","last_peered":"2024-09-19T09:27:11.338515+0000","last_clean":"2024-09-19T09:27:11.338515+0000","last_became_active":"2024-09-19T09:25:29.295277+0000","last_became_peered":"2024-09-19T09:25:29.295277+0000","last_unstale":"2024-09-19T09:27:11.338515+0000","last_undegraded":"2024-09-19T09:27:11.338515+0000","last_fullsized":"2024-09-19T09:27:11.338515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:57.690699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996123+0000","last_change":"2024-09-19T09:25:27.279239+0000","last_active":"2024-09-19T09:25:42.996123+0000","last_peered":"2024-09-19T09:25:42.996123+0000","last_clean":"2024-09-19T09:25:42.996123+0000","last_became_active":"2024-09-19T09:25:27.278868+0000","last_became_peered":"2024-09-19T09:25:27.278868+0000","last_unstale":"2024-09-19T09:25:42.996123+0000","last_undegraded":"2024-09-19T09:25:42.996123+0000","last_fullsized":"2024-09-19T09:25:42.996123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:41.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341911+0000","last_change":"2024-09-19T09:25:29.300833+0000","last_active":"2024-09-19T09:27:11.341911+0000","last_peered":"2024-09-19T09:27:11.341911+0000","last_clean":"2024-09-19T09:27:11.341911+0000","last_became_active":"2024-09-19T09:25:29.300472+0000","last_became_peered":"2024-09-19T09:25:29.300472+0000","last_unstale":"2024-09-19T09:27:11.341911+0000","last_undegraded":"2024-09-19T09:27:11.341911+0000","last_fullsized":"2024-09-19T09:27:11.341911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:50.591276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'7","reported_seq":117,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:15.140661+0000","last_change":"2024-09-19T09:25:43.001722+0000","last_active":"2024-09-19T09:27:15.140661+0000","last_peered":"2024-09-19T09:27:15.140661+0000","last_clean":"2024-09-19T09:27:15.140661+0000","last_became_active":"2024-09-19T09:25:42.992796+0000","last_became_peered":"2024-09-19T09:25:42.992796+0000","last_unstale":"2024-09-19T09:27:15.140661+0000","last_undegraded":"2024-09-19T09:27:15.140661+0000","last_fullsized":"2024-09-19T09:27:15.140661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:59.639010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346495+0000","last_change":"2024-09-19T09:25:29.295906+0000","last_active":"2024-09-19T09:27:11.346495+0000","last_peered":"2024-09-19T09:27:11.346495+0000","last_clean":"2024-09-19T09:27:11.346495+0000","last_became_active":"2024-09-19T09:25:29.295565+0000","last_became_peered":"2024-09-19T09:25:29.295565+0000","last_unstale":"2024-09-19T09:27:11.346495+0000","last_undegraded":"2024-09-19T09:27:11.346495+0000","last_fullsized":"2024-09-19T09:27:11.346495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:24.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989907+0000","last_change":"2024-09-19T09:24:19.256300+0000","last_active":"2024-09-19T09:25:42.989907+0000","last_peered":"2024-09-19T09:25:42.989907+0000","last_clean":"2024-09-19T09:25:42.989907+0000","last_became_active":"2024-09-19T09:24:19.255833+0000","last_became_peered":"2024-09-19T09:24:19.255833+0000","last_unstale":"2024-09-19T09:25:42.989907+0000","last_undegraded":"2024-09-19T09:25:42.989907+0000","last_fullsized":"2024-09-19T09:25:42.989907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_clean_scrub_stamp":"2024-09-19T09:20:57.798838+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:14:33.343748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989637+0000","last_change":"2024-09-19T09:25:27.272090+0000","last_active":"2024-09-19T09:25:42.989637+0000","last_peered":"2024-09-19T09:25:42.989637+0000","last_clean":"2024-09-19T09:25:42.989637+0000","last_became_active":"2024-09-19T09:25:27.271651+0000","last_became_peered":"2024-09-19T09:25:27.271651+0000","last_unstale":"2024-09-19T09:25:42.989637+0000","last_undegraded":"2024-09-19T09:25:42.989637+0000","last_fullsized":"2024-09-19T09:25:42.989637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:45.847082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344375+0000","last_change":"2024-09-19T09:25:29.300718+0000","last_active":"2024-09-19T09:27:11.344375+0000","last_peered":"2024-09-19T09:27:11.344375+0000","last_clean":"2024-09-19T09:27:11.344375+0000","last_became_active":"2024-09-19T09:25:29.299460+0000","last_became_peered":"2024-09-19T09:25:29.299460+0000","last_unstale":"2024-09-19T09:27:11.344375+0000","last_undegraded":"2024-09-19T09:27:11.344375+0000","last_fullsized":"2024-09-19T09:27:11.344375+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:43.063909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003299+0000","last_change":"2024-09-19T09:25:43.003299+0000","last_active":"2024-09-19T09:25:43.003299+0000","last_peered":"2024-09-19T09:25:43.003299+0000","last_clean":"2024-09-19T09:25:43.003299+0000","last_became_active":"2024-09-19T09:25:42.999686+0000","last_became_peered":"2024-09-19T09:25:42.999686+0000","last_unstale":"2024-09-19T09:25:43.003299+0000","last_undegraded":"2024-09-19T09:25:43.003299+0000","last_fullsized":"2024-09-19T09:25:43.003299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:05.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337734+0000","last_change":"2024-09-19T09:25:29.297833+0000","last_active":"2024-09-19T09:27:11.337734+0000","last_peered":"2024-09-19T09:27:11.337734+0000","last_clean":"2024-09-19T09:27:11.337734+0000","last_became_active":"2024-09-19T09:25:29.297534+0000","last_became_peered":"2024-09-19T09:25:29.297534+0000","last_unstale":"2024-09-19T09:27:11.337734+0000","last_undegraded":"2024-09-19T09:27:11.337734+0000","last_fullsized":"2024-09-19T09:27:11.337734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:47.156253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":129,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:17.109301+0000","last_change":"2024-09-19T09:25:27.322759+0000","last_active":"2024-09-19T09:27:17.109301+0000","last_peered":"2024-09-19T09:27:17.109301+0000","last_clean":"2024-09-19T09:27:17.109301+0000","last_became_active":"2024-09-19T09:25:27.322402+0000","last_became_peered":"2024-09-19T09:25:27.322402+0000","last_unstale":"2024-09-19T09:27:17.109301+0000","last_undegraded":"2024-09-19T09:27:17.109301+0000","last_fullsized":"2024-09-19T09:27:17.109301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:06.276957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335476+0000","last_change":"2024-09-19T09:25:42.999834+0000","last_active":"2024-09-19T09:27:11.335476+0000","last_peered":"2024-09-19T09:27:11.335476+0000","last_clean":"2024-09-19T09:27:11.335476+0000","last_became_active":"2024-09-19T09:25:42.999450+0000","last_became_peered":"2024-09-19T09:25:42.999450+0000","last_unstale":"2024-09-19T09:27:11.335476+0000","last_undegraded":"2024-09-19T09:27:11.335476+0000","last_fullsized":"2024-09-19T09:27:11.335476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:00.509844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'7","reported_seq":129,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.117110+0000","last_change":"2024-09-19T09:25:27.275212+0000","last_active":"2024-09-19T09:27:16.117110+0000","last_peered":"2024-09-19T09:27:16.117110+0000","last_clean":"2024-09-19T09:27:16.117110+0000","last_became_active":"2024-09-19T09:25:27.274802+0000","last_became_peered":"2024-09-19T09:25:27.274802+0000","last_unstale":"2024-09-19T09:27:16.117110+0000","last_undegraded":"2024-09-19T09:27:16.117110+0000","last_fullsized":"2024-09-19T09:27:16.117110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:48.392026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.333364+0000","last_change":"2024-09-19T09:25:29.304747+0000","last_active":"2024-09-19T09:27:11.333364+0000","last_peered":"2024-09-19T09:27:11.333364+0000","last_clean":"2024-09-19T09:27:11.333364+0000","last_became_active":"2024-09-19T09:25:29.304386+0000","last_became_peered":"2024-09-19T09:25:29.304386+0000","last_unstale":"2024-09-19T09:27:11.333364+0000","last_undegraded":"2024-09-19T09:27:11.333364+0000","last_fullsized":"2024-09-19T09:27:11.333364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.788697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988854+0000","last_change":"2024-09-19T09:25:27.323083+0000","last_active":"2024-09-19T09:25:42.988854+0000","last_peered":"2024-09-19T09:25:42.988854+0000","last_clean":"2024-09-19T09:25:42.988854+0000","last_became_active":"2024-09-19T09:25:27.322707+0000","last_became_peered":"2024-09-19T09:25:27.322707+0000","last_unstale":"2024-09-19T09:25:42.988854+0000","last_undegraded":"2024-09-19T09:25:42.988854+0000","last_fullsized":"2024-09-19T09:25:42.988854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:24.415116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334132+0000","last_change":"2024-09-19T09:25:29.292472+0000","last_active":"2024-09-19T09:27:11.334132+0000","last_peered":"2024-09-19T09:27:11.334132+0000","last_clean":"2024-09-19T09:27:11.334132+0000","last_became_active":"2024-09-19T09:25:29.292079+0000","last_became_peered":"2024-09-19T09:25:29.292079+0000","last_unstale":"2024-09-19T09:27:11.334132+0000","last_undegraded":"2024-09-19T09:27:11.334132+0000","last_fullsized":"2024-09-19T09:27:11.334132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:35.801594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991224+0000","last_change":"2024-09-19T09:25:27.324839+0000","last_active":"2024-09-19T09:25:42.991224+0000","last_peered":"2024-09-19T09:25:42.991224+0000","last_clean":"2024-09-19T09:25:42.991224+0000","last_became_active":"2024-09-19T09:25:27.324475+0000","last_became_peered":"2024-09-19T09:25:27.324475+0000","last_unstale":"2024-09-19T09:25:42.991224+0000","last_undegraded":"2024-09-19T09:25:42.991224+0000","last_fullsized":"2024-09-19T09:25:42.991224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:32.251753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338007+0000","last_change":"2024-09-19T09:25:29.293025+0000","last_active":"2024-09-19T09:27:11.338007+0000","last_peered":"2024-09-19T09:27:11.338007+0000","last_clean":"2024-09-19T09:27:11.338007+0000","last_became_active":"2024-09-19T09:25:29.292570+0000","last_became_peered":"2024-09-19T09:25:29.292570+0000","last_unstale":"2024-09-19T09:27:11.338007+0000","last_undegraded":"2024-09-19T09:27:11.338007+0000","last_fullsized":"2024-09-19T09:27:11.338007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:16.394524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:55.295924+0000","last_change":"2024-09-19T09:25:27.278100+0000","last_active":"2024-09-19T09:25:55.295924+0000","last_peered":"2024-09-19T09:25:55.295924+0000","last_clean":"2024-09-19T09:25:55.295924+0000","last_became_active":"2024-09-19T09:25:27.277769+0000","last_became_peered":"2024-09-19T09:25:27.277769+0000","last_unstale":"2024-09-19T09:25:55.295924+0000","last_undegraded":"2024-09-19T09:25:55.295924+0000","last_fullsized":"2024-09-19T09:25:55.295924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:31.885142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338230+0000","last_change":"2024-09-19T09:25:29.293651+0000","last_active":"2024-09-19T09:27:11.338230+0000","last_peered":"2024-09-19T09:27:11.338230+0000","last_clean":"2024-09-19T09:27:11.338230+0000","last_became_active":"2024-09-19T09:25:29.293317+0000","last_became_peered":"2024-09-19T09:25:29.293317+0000","last_unstale":"2024-09-19T09:27:11.338230+0000","last_undegraded":"2024-09-19T09:27:11.338230+0000","last_fullsized":"2024-09-19T09:27:11.338230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:40.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990030+0000","last_change":"2024-09-19T09:25:27.269367+0000","last_active":"2024-09-19T09:25:42.990030+0000","last_peered":"2024-09-19T09:25:42.990030+0000","last_clean":"2024-09-19T09:25:42.990030+0000","last_became_active":"2024-09-19T09:25:27.269011+0000","last_became_peered":"2024-09-19T09:25:27.269011+0000","last_unstale":"2024-09-19T09:25:42.990030+0000","last_undegraded":"2024-09-19T09:25:42.990030+0000","last_fullsized":"2024-09-19T09:25:42.990030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:00.930752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334968+0000","last_change":"2024-09-19T09:25:29.299134+0000","last_active":"2024-09-19T09:27:11.334968+0000","last_peered":"2024-09-19T09:27:11.334968+0000","last_clean":"2024-09-19T09:27:11.334968+0000","last_became_active":"2024-09-19T09:25:29.298648+0000","last_became_peered":"2024-09-19T09:25:29.298648+0000","last_unstale":"2024-09-19T09:27:11.334968+0000","last_undegraded":"2024-09-19T09:27:11.334968+0000","last_fullsized":"2024-09-19T09:27:11.334968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:39.627852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003686+0000","last_change":"2024-09-19T09:25:27.322861+0000","last_active":"2024-09-19T09:25:43.003686+0000","last_peered":"2024-09-19T09:25:43.003686+0000","last_clean":"2024-09-19T09:25:43.003686+0000","last_became_active":"2024-09-19T09:25:27.322560+0000","last_became_peered":"2024-09-19T09:25:27.322560+0000","last_unstale":"2024-09-19T09:25:43.003686+0000","last_undegraded":"2024-09-19T09:25:43.003686+0000","last_fullsized":"2024-09-19T09:25:43.003686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:35.993093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.301522+0000","last_change":"2024-09-19T09:25:29.301680+0000","last_active":"2024-09-19T09:27:11.301522+0000","last_peered":"2024-09-19T09:27:11.301522+0000","last_clean":"2024-09-19T09:27:11.301522+0000","last_became_active":"2024-09-19T09:25:29.301331+0000","last_became_peered":"2024-09-19T09:25:29.301331+0000","last_unstale":"2024-09-19T09:27:11.301522+0000","last_undegraded":"2024-09-19T09:27:11.301522+0000","last_fullsized":"2024-09-19T09:27:11.301522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:57.364245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:27.272609+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:27.272169+0000","last_became_peered":"2024-09-19T09:25:27.272169+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.879474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341200+0000","last_change":"2024-09-19T09:25:29.296847+0000","last_active":"2024-09-19T09:27:11.341200+0000","last_peered":"2024-09-19T09:27:11.341200+0000","last_clean":"2024-09-19T09:27:11.341200+0000","last_became_active":"2024-09-19T09:25:29.296465+0000","last_became_peered":"2024-09-19T09:25:29.296465+0000","last_unstale":"2024-09-19T09:27:11.341200+0000","last_undegraded":"2024-09-19T09:27:11.341200+0000","last_fullsized":"2024-09-19T09:27:11.341200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:05:48.553252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.012954+0000","last_change":"2024-09-19T09:25:43.012532+0000","last_active":"2024-09-19T09:25:43.012954+0000","last_peered":"2024-09-19T09:25:43.012954+0000","last_clean":"2024-09-19T09:25:43.012954+0000","last_became_active":"2024-09-19T09:25:42.994996+0000","last_became_peered":"2024-09-19T09:25:42.994996+0000","last_unstale":"2024-09-19T09:25:43.012954+0000","last_undegraded":"2024-09-19T09:25:43.012954+0000","last_fullsized":"2024-09-19T09:25:43.012954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:25.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337187+0000","last_change":"2024-09-19T09:25:29.297633+0000","last_active":"2024-09-19T09:27:11.337187+0000","last_peered":"2024-09-19T09:27:11.337187+0000","last_clean":"2024-09-19T09:27:11.337187+0000","last_became_active":"2024-09-19T09:25:29.297284+0000","last_became_peered":"2024-09-19T09:25:29.297284+0000","last_unstale":"2024-09-19T09:27:11.337187+0000","last_undegraded":"2024-09-19T09:27:11.337187+0000","last_fullsized":"2024-09-19T09:27:11.337187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:50.582546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991172+0000","last_change":"2024-09-19T09:25:27.281778+0000","last_active":"2024-09-19T09:25:42.991172+0000","last_peered":"2024-09-19T09:25:42.991172+0000","last_clean":"2024-09-19T09:25:42.991172+0000","last_became_active":"2024-09-19T09:25:27.281280+0000","last_became_peered":"2024-09-19T09:25:27.281280+0000","last_unstale":"2024-09-19T09:25:42.991172+0000","last_undegraded":"2024-09-19T09:25:42.991172+0000","last_fullsized":"2024-09-19T09:25:42.991172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:46.811968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343342+0000","last_change":"2024-09-19T09:25:29.295663+0000","last_active":"2024-09-19T09:27:11.343342+0000","last_peered":"2024-09-19T09:27:11.343342+0000","last_clean":"2024-09-19T09:27:11.343342+0000","last_became_active":"2024-09-19T09:25:29.295146+0000","last_became_peered":"2024-09-19T09:25:29.295146+0000","last_unstale":"2024-09-19T09:27:11.343342+0000","last_undegraded":"2024-09-19T09:27:11.343342+0000","last_fullsized":"2024-09-19T09:27:11.343342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:36.749339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.345435+0000","last_change":"2024-09-19T09:25:29.298004+0000","last_active":"2024-09-19T09:27:11.345435+0000","last_peered":"2024-09-19T09:27:11.345435+0000","last_clean":"2024-09-19T09:27:11.345435+0000","last_became_active":"2024-09-19T09:25:29.297600+0000","last_became_peered":"2024-09-19T09:25:29.297600+0000","last_unstale":"2024-09-19T09:27:11.345435+0000","last_undegraded":"2024-09-19T09:27:11.345435+0000","last_fullsized":"2024-09-19T09:27:11.345435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:09.738615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992323+0000","last_change":"2024-09-19T09:25:27.282876+0000","last_active":"2024-09-19T09:25:42.992323+0000","last_peered":"2024-09-19T09:25:42.992323+0000","last_clean":"2024-09-19T09:25:42.992323+0000","last_became_active":"2024-09-19T09:25:27.282201+0000","last_became_peered":"2024-09-19T09:25:27.282201+0000","last_unstale":"2024-09-19T09:25:42.992323+0000","last_undegraded":"2024-09-19T09:25:42.992323+0000","last_fullsized":"2024-09-19T09:25:42.992323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:08.508662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347692+0000","last_change":"2024-09-19T09:25:29.288379+0000","last_active":"2024-09-19T09:27:11.347692+0000","last_peered":"2024-09-19T09:27:11.347692+0000","last_clean":"2024-09-19T09:27:11.347692+0000","last_became_active":"2024-09-19T09:25:29.287917+0000","last_became_peered":"2024-09-19T09:25:29.287917+0000","last_unstale":"2024-09-19T09:27:11.347692+0000","last_undegraded":"2024-09-19T09:27:11.347692+0000","last_fullsized":"2024-09-19T09:27:11.347692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:03.628233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991076+0000","last_change":"2024-09-19T09:25:27.279551+0000","last_active":"2024-09-19T09:25:42.991076+0000","last_peered":"2024-09-19T09:25:42.991076+0000","last_clean":"2024-09-19T09:25:42.991076+0000","last_became_active":"2024-09-19T09:25:27.278956+0000","last_became_peered":"2024-09-19T09:25:27.278956+0000","last_unstale":"2024-09-19T09:25:42.991076+0000","last_undegraded":"2024-09-19T09:25:42.991076+0000","last_fullsized":"2024-09-19T09:25:42.991076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:17.186453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347269+0000","last_change":"2024-09-19T09:25:29.288657+0000","last_active":"2024-09-19T09:27:11.347269+0000","last_peered":"2024-09-19T09:27:11.347269+0000","last_clean":"2024-09-19T09:27:11.347269+0000","last_became_active":"2024-09-19T09:25:29.288355+0000","last_became_peered":"2024-09-19T09:25:29.288355+0000","last_unstale":"2024-09-19T09:27:11.347269+0000","last_undegraded":"2024-09-19T09:27:11.347269+0000","last_fullsized":"2024-09-19T09:27:11.347269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:46.237558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991315+0000","last_change":"2024-09-19T09:25:27.323312+0000","last_active":"2024-09-19T09:25:42.991315+0000","last_peered":"2024-09-19T09:25:42.991315+0000","last_clean":"2024-09-19T09:25:42.991315+0000","last_became_active":"2024-09-19T09:25:27.322984+0000","last_became_peered":"2024-09-19T09:25:27.322984+0000","last_unstale":"2024-09-19T09:25:42.991315+0000","last_undegraded":"2024-09-19T09:25:42.991315+0000","last_fullsized":"2024-09-19T09:25:42.991315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:48.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.327987+0000","last_change":"2024-09-19T09:25:29.301832+0000","last_active":"2024-09-19T09:27:11.327987+0000","last_peered":"2024-09-19T09:27:11.327987+0000","last_clean":"2024-09-19T09:27:11.327987+0000","last_became_active":"2024-09-19T09:25:29.301539+0000","last_became_peered":"2024-09-19T09:25:29.301539+0000","last_unstale":"2024-09-19T09:27:11.327987+0000","last_undegraded":"2024-09-19T09:27:11.327987+0000","last_fullsized":"2024-09-19T09:27:11.327987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:00.244948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'14","reported_seq":154,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.116536+0000","last_change":"2024-09-19T09:25:43.009668+0000","last_active":"2024-09-19T09:27:16.116536+0000","last_peered":"2024-09-19T09:27:16.116536+0000","last_clean":"2024-09-19T09:27:16.116536+0000","last_became_active":"2024-09-19T09:25:42.994966+0000","last_became_peered":"2024-09-19T09:25:42.994966+0000","last_unstale":"2024-09-19T09:27:16.116536+0000","last_undegraded":"2024-09-19T09:27:16.116536+0000","last_fullsized":"2024-09-19T09:27:16.116536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:04:05.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68571,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1020,"num_read_kb":587,"num_write":258,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1842,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612644,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":431506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841576,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1593344,"data_stored":1081568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005100,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":917504,"data_stored":429922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234033,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":430300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1023357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593788,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":901120,"data_stored":430173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1022639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":431738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215178,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":490149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":491099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":431512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:27:22.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T09:27:22.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.287+0000 7f567061e640 1 -- 172.21.15.16:0/1774902910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5668101b30 msgr2=0x7f5668101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.287+0000 7f567061e640 1 --2- 172.21.15.16:0/1774902910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5668101b30 0x7f5668101f30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5664009900 tx=0x7f566402a410 comp rx=0 tx=0).stop 2024-09-19T09:27:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.287+0000 7f567061e640 1 -- 172.21.15.16:0/1774902910 shutdown_connections 2024-09-19T09:27:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.287+0000 7f567061e640 1 --2- 172.21.15.16:0/1774902910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5668102d20 0x7f56681031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.287+0000 7f567061e640 1 --2- 172.21.15.16:0/1774902910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56681036e0 0x7f5668109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.287+0000 7f567061e640 1 --2- 172.21.15.16:0/1774902910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5668101b30 0x7f5668101f30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.287+0000 7f567061e640 1 -- 172.21.15.16:0/1774902910 >> 172.21.15.16:0/1774902910 conn(0x7f56680fd270 msgr2=0x7f56680ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.287+0000 7f567061e640 1 -- 172.21.15.16:0/1774902910 shutdown_connections 2024-09-19T09:27:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.287+0000 7f567061e640 1 -- 172.21.15.16:0/1774902910 wait complete. 2024-09-19T09:27:22.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.288+0000 7f567061e640 1 Processor -- start 2024-09-19T09:27:22.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.288+0000 7f567061e640 1 -- start start 2024-09-19T09:27:22.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.288+0000 7f567061e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5668101b30 0x7f5668197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:22.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.288+0000 7f567061e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5668102d20 0x7f5668198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.288+0000 7f567061e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56681036e0 0x7f566819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.288+0000 7f567061e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f566810d790 con 0x7f56681036e0 2024-09-19T09:27:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.288+0000 7f567061e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f566810d610 con 0x7f5668101b30 2024-09-19T09:27:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.288+0000 7f567061e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f566810d910 con 0x7f5668102d20 2024-09-19T09:27:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.288+0000 7f566ebbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56681036e0 0x7f566819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.288+0000 7f566ebbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56681036e0 0x7f566819f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39682/0 (socket says 172.21.15.16:39682) 2024-09-19T09:27:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.288+0000 7f566ebbb640 1 -- 172.21.15.16:0/1520091249 learned_addr learned my addr 172.21.15.16:0/1520091249 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.288+0000 7f566dbb9640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5668102d20 0x7f5668198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.288+0000 7f566e3ba640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5668101b30 0x7f5668197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.289+0000 7f566ebbb640 1 -- 172.21.15.16:0/1520091249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5668102d20 msgr2=0x7f5668198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.289+0000 7f566ebbb640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5668102d20 0x7f5668198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.289+0000 7f566ebbb640 1 -- 172.21.15.16:0/1520091249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5668101b30 msgr2=0x7f5668197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.289+0000 7f566ebbb640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5668101b30 0x7f5668197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.289+0000 7f566ebbb640 1 -- 172.21.15.16:0/1520091249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566819fa00 con 0x7f56681036e0 2024-09-19T09:27:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.289+0000 7f566ebbb640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56681036e0 0x7f566819f390 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f566000eaa0 tx=0x7f566000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.289+0000 7f565f7fe640 1 -- 172.21.15.16:0/1520091249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660004d20 con 0x7f56681036e0 2024-09-19T09:27:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.289+0000 7f565f7fe640 1 -- 172.21.15.16:0/1520091249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5660005020 con 0x7f56681036e0 2024-09-19T09:27:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.289+0000 7f567061e640 1 -- 172.21.15.16:0/1520091249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566819fcf0 con 0x7f56681036e0 2024-09-19T09:27:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.289+0000 7f565f7fe640 1 -- 172.21.15.16:0/1520091249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56600116d0 con 0x7f56681036e0 2024-09-19T09:27:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.289+0000 7f567061e640 1 -- 172.21.15.16:0/1520091249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56681a01e0 con 0x7f56681036e0 2024-09-19T09:27:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.290+0000 7f567061e640 1 -- 172.21.15.16:0/1520091249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5668108ed0 con 0x7f56681036e0 2024-09-19T09:27:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.291+0000 7f565f7fe640 1 -- 172.21.15.16:0/1520091249 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f56600051d0 con 0x7f56681036e0 2024-09-19T09:27:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.292+0000 7f565f7fe640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f56540a32d0 0x7f56540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.292+0000 7f566e3ba640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f56540a32d0 0x7f56540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.292+0000 7f565f7fe640 1 -- 172.21.15.16:0/1520091249 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f56600cfec0 con 0x7f56681036e0 2024-09-19T09:27:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.292+0000 7f566e3ba640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f56540a32d0 0x7f56540a5790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5664006010 tx=0x7f5664000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:22.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.295+0000 7f565f7fe640 1 -- 172.21.15.16:0/1520091249 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5660016030 con 0x7f56681036e0 2024-09-19T09:27:22.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:21 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3038066765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.468+0000 7f567061e640 1 -- 172.21.15.16:0/1520091249 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5668062630 con 0x7f56540a32d0 2024-09-19T09:27:22.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.475+0000 7f565f7fe640 1 -- 172.21.15.16:0/1520091249 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258890 (secure 0 0 0) 0x7f5668062630 con 0x7f56540a32d0 2024-09-19T09:27:22.478 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:22.480 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T09:27:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.478+0000 7f567061e640 1 -- 172.21.15.16:0/1520091249 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f56540a32d0 msgr2=0x7f56540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.478+0000 7f567061e640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f56540a32d0 0x7f56540a5790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5664006010 tx=0x7f5664000bc0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.478+0000 7f567061e640 1 -- 172.21.15.16:0/1520091249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56681036e0 msgr2=0x7f566819f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.478+0000 7f567061e640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56681036e0 0x7f566819f390 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f566000eaa0 tx=0x7f566000eea0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.478+0000 7f567061e640 1 -- 172.21.15.16:0/1520091249 shutdown_connections 2024-09-19T09:27:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.478+0000 7f567061e640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f56540a32d0 0x7f56540a5790 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.478+0000 7f567061e640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5668102d20 0x7f5668198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.478+0000 7f567061e640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56681036e0 0x7f566819f390 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.478+0000 7f567061e640 1 --2- 172.21.15.16:0/1520091249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5668101b30 0x7f5668197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.478+0000 7f567061e640 1 -- 172.21.15.16:0/1520091249 >> 172.21.15.16:0/1520091249 conn(0x7f56680fd270 msgr2=0x7f56681060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:22.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.478+0000 7f567061e640 1 -- 172.21.15.16:0/1520091249 shutdown_connections 2024-09-19T09:27:22.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.478+0000 7f567061e640 1 -- 172.21.15.16:0/1520091249 wait complete. 2024-09-19T09:27:22.495 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-19T09:27:21.365506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659173,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1146,"num_read_kb":696,"num_write":441,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1182578,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391004,"kb_used_data":12636,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550820,"statfs":{"total":1151940427776,"available":1151540039680,"internally_reserved":0,"allocated":12939264,"data_stored":7123427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":813,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":13,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003207"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347860+0000","last_change":"2024-09-19T09:25:29.294965+0000","last_active":"2024-09-19T09:27:11.347860+0000","last_peered":"2024-09-19T09:27:11.347860+0000","last_clean":"2024-09-19T09:27:11.347860+0000","last_became_active":"2024-09-19T09:25:29.294157+0000","last_became_peered":"2024-09-19T09:25:29.294157+0000","last_unstale":"2024-09-19T09:27:11.347860+0000","last_undegraded":"2024-09-19T09:27:11.347860+0000","last_fullsized":"2024-09-19T09:27:11.347860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:24.643835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335438+0000","last_change":"2024-09-19T09:25:29.298888+0000","last_active":"2024-09-19T09:27:11.335438+0000","last_peered":"2024-09-19T09:27:11.335438+0000","last_clean":"2024-09-19T09:27:11.335438+0000","last_became_active":"2024-09-19T09:25:29.298245+0000","last_became_peered":"2024-09-19T09:25:29.298245+0000","last_unstale":"2024-09-19T09:27:11.335438+0000","last_undegraded":"2024-09-19T09:27:11.335438+0000","last_fullsized":"2024-09-19T09:27:11.335438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:03.162342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344990+0000","last_change":"2024-09-19T09:25:29.292300+0000","last_active":"2024-09-19T09:27:11.344990+0000","last_peered":"2024-09-19T09:27:11.344990+0000","last_clean":"2024-09-19T09:27:11.344990+0000","last_became_active":"2024-09-19T09:25:29.291973+0000","last_became_peered":"2024-09-19T09:25:29.291973+0000","last_unstale":"2024-09-19T09:27:11.344990+0000","last_undegraded":"2024-09-19T09:27:11.344990+0000","last_fullsized":"2024-09-19T09:27:11.344990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:19.672778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342109+0000","last_change":"2024-09-19T09:25:29.305695+0000","last_active":"2024-09-19T09:27:11.342109+0000","last_peered":"2024-09-19T09:27:11.342109+0000","last_clean":"2024-09-19T09:27:11.342109+0000","last_became_active":"2024-09-19T09:25:29.305244+0000","last_became_peered":"2024-09-19T09:25:29.305244+0000","last_unstale":"2024-09-19T09:27:11.342109+0000","last_undegraded":"2024-09-19T09:27:11.342109+0000","last_fullsized":"2024-09-19T09:27:11.342109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:38.054624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336846+0000","last_change":"2024-09-19T09:25:29.295259+0000","last_active":"2024-09-19T09:27:11.336846+0000","last_peered":"2024-09-19T09:27:11.336846+0000","last_clean":"2024-09-19T09:27:11.336846+0000","last_became_active":"2024-09-19T09:25:29.294936+0000","last_became_peered":"2024-09-19T09:25:29.294936+0000","last_unstale":"2024-09-19T09:27:11.336846+0000","last_undegraded":"2024-09-19T09:27:11.336846+0000","last_fullsized":"2024-09-19T09:27:11.336846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:59.193504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336175+0000","last_change":"2024-09-19T09:25:29.299650+0000","last_active":"2024-09-19T09:27:11.336175+0000","last_peered":"2024-09-19T09:27:11.336175+0000","last_clean":"2024-09-19T09:27:11.336175+0000","last_became_active":"2024-09-19T09:25:29.299182+0000","last_became_peered":"2024-09-19T09:25:29.299182+0000","last_unstale":"2024-09-19T09:27:11.336175+0000","last_undegraded":"2024-09-19T09:27:11.336175+0000","last_fullsized":"2024-09-19T09:27:11.336175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:44.637771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342329+0000","last_change":"2024-09-19T09:25:29.302501+0000","last_active":"2024-09-19T09:27:11.342329+0000","last_peered":"2024-09-19T09:27:11.342329+0000","last_clean":"2024-09-19T09:27:11.342329+0000","last_became_active":"2024-09-19T09:25:29.301419+0000","last_became_peered":"2024-09-19T09:25:29.301419+0000","last_unstale":"2024-09-19T09:27:11.342329+0000","last_undegraded":"2024-09-19T09:27:11.342329+0000","last_fullsized":"2024-09-19T09:27:11.342329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:50.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337306+0000","last_change":"2024-09-19T09:25:29.300212+0000","last_active":"2024-09-19T09:27:11.337306+0000","last_peered":"2024-09-19T09:27:11.337306+0000","last_clean":"2024-09-19T09:27:11.337306+0000","last_became_active":"2024-09-19T09:25:29.299873+0000","last_became_peered":"2024-09-19T09:25:29.299873+0000","last_unstale":"2024-09-19T09:27:11.337306+0000","last_undegraded":"2024-09-19T09:27:11.337306+0000","last_fullsized":"2024-09-19T09:27:11.337306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:34.620314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346891+0000","last_change":"2024-09-19T09:25:29.290214+0000","last_active":"2024-09-19T09:27:11.346891+0000","last_peered":"2024-09-19T09:27:11.346891+0000","last_clean":"2024-09-19T09:27:11.346891+0000","last_became_active":"2024-09-19T09:25:29.289707+0000","last_became_peered":"2024-09-19T09:25:29.289707+0000","last_unstale":"2024-09-19T09:27:11.346891+0000","last_undegraded":"2024-09-19T09:27:11.346891+0000","last_fullsized":"2024-09-19T09:27:11.346891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:16.341340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338183+0000","last_change":"2024-09-19T09:25:29.301774+0000","last_active":"2024-09-19T09:27:11.338183+0000","last_peered":"2024-09-19T09:27:11.338183+0000","last_clean":"2024-09-19T09:27:11.338183+0000","last_became_active":"2024-09-19T09:25:29.301398+0000","last_became_peered":"2024-09-19T09:25:29.301398+0000","last_unstale":"2024-09-19T09:27:11.338183+0000","last_undegraded":"2024-09-19T09:27:11.338183+0000","last_fullsized":"2024-09-19T09:27:11.338183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:36.058038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346172+0000","last_change":"2024-09-19T09:25:29.301475+0000","last_active":"2024-09-19T09:27:11.346172+0000","last_peered":"2024-09-19T09:27:11.346172+0000","last_clean":"2024-09-19T09:27:11.346172+0000","last_became_active":"2024-09-19T09:25:29.300908+0000","last_became_peered":"2024-09-19T09:25:29.300908+0000","last_unstale":"2024-09-19T09:27:11.346172+0000","last_undegraded":"2024-09-19T09:27:11.346172+0000","last_fullsized":"2024-09-19T09:27:11.346172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:44.774127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336219+0000","last_change":"2024-09-19T09:25:42.994395+0000","last_active":"2024-09-19T09:27:11.336219+0000","last_peered":"2024-09-19T09:27:11.336219+0000","last_clean":"2024-09-19T09:27:11.336219+0000","last_became_active":"2024-09-19T09:25:42.993992+0000","last_became_peered":"2024-09-19T09:25:42.993992+0000","last_unstale":"2024-09-19T09:27:11.336219+0000","last_undegraded":"2024-09-19T09:27:11.336219+0000","last_fullsized":"2024-09-19T09:27:11.336219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.486069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.340454+0000","last_change":"2024-09-19T09:25:42.989979+0000","last_active":"2024-09-19T09:27:11.340454+0000","last_peered":"2024-09-19T09:27:11.340454+0000","last_clean":"2024-09-19T09:27:11.340454+0000","last_became_active":"2024-09-19T09:25:42.989578+0000","last_became_peered":"2024-09-19T09:25:42.989578+0000","last_unstale":"2024-09-19T09:27:11.340454+0000","last_undegraded":"2024-09-19T09:27:11.340454+0000","last_fullsized":"2024-09-19T09:27:11.340454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:04.583023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336926+0000","last_change":"2024-09-19T09:25:29.299917+0000","last_active":"2024-09-19T09:27:11.336926+0000","last_peered":"2024-09-19T09:27:11.336926+0000","last_clean":"2024-09-19T09:27:11.336926+0000","last_became_active":"2024-09-19T09:25:29.299571+0000","last_became_peered":"2024-09-19T09:25:29.299571+0000","last_unstale":"2024-09-19T09:27:11.336926+0000","last_undegraded":"2024-09-19T09:27:11.336926+0000","last_fullsized":"2024-09-19T09:27:11.336926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:25.142252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335108+0000","last_change":"2024-09-19T09:25:29.299713+0000","last_active":"2024-09-19T09:27:11.335108+0000","last_peered":"2024-09-19T09:27:11.335108+0000","last_clean":"2024-09-19T09:27:11.335108+0000","last_became_active":"2024-09-19T09:25:29.299234+0000","last_became_peered":"2024-09-19T09:25:29.299234+0000","last_unstale":"2024-09-19T09:27:11.335108+0000","last_undegraded":"2024-09-19T09:27:11.335108+0000","last_fullsized":"2024-09-19T09:27:11.335108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:34.714714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346606+0000","last_change":"2024-09-19T09:25:29.293010+0000","last_active":"2024-09-19T09:27:11.346606+0000","last_peered":"2024-09-19T09:27:11.346606+0000","last_clean":"2024-09-19T09:27:11.346606+0000","last_became_active":"2024-09-19T09:25:29.292645+0000","last_became_peered":"2024-09-19T09:25:29.292645+0000","last_unstale":"2024-09-19T09:27:11.346606+0000","last_undegraded":"2024-09-19T09:27:11.346606+0000","last_fullsized":"2024-09-19T09:27:11.346606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:24.749075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347900+0000","last_change":"2024-09-19T09:25:29.294530+0000","last_active":"2024-09-19T09:27:11.347900+0000","last_peered":"2024-09-19T09:27:11.347900+0000","last_clean":"2024-09-19T09:27:11.347900+0000","last_became_active":"2024-09-19T09:25:29.293699+0000","last_became_peered":"2024-09-19T09:25:29.293699+0000","last_unstale":"2024-09-19T09:27:11.347900+0000","last_undegraded":"2024-09-19T09:27:11.347900+0000","last_fullsized":"2024-09-19T09:27:11.347900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:53.491258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336452+0000","last_change":"2024-09-19T09:25:29.290600+0000","last_active":"2024-09-19T09:27:11.336452+0000","last_peered":"2024-09-19T09:27:11.336452+0000","last_clean":"2024-09-19T09:27:11.336452+0000","last_became_active":"2024-09-19T09:25:29.290305+0000","last_became_peered":"2024-09-19T09:25:29.290305+0000","last_unstale":"2024-09-19T09:27:11.336452+0000","last_undegraded":"2024-09-19T09:27:11.336452+0000","last_fullsized":"2024-09-19T09:27:11.336452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:42.809659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338092+0000","last_change":"2024-09-19T09:25:29.300591+0000","last_active":"2024-09-19T09:27:11.338092+0000","last_peered":"2024-09-19T09:27:11.338092+0000","last_clean":"2024-09-19T09:27:11.338092+0000","last_became_active":"2024-09-19T09:25:29.300251+0000","last_became_peered":"2024-09-19T09:25:29.300251+0000","last_unstale":"2024-09-19T09:27:11.338092+0000","last_undegraded":"2024-09-19T09:27:11.338092+0000","last_fullsized":"2024-09-19T09:27:11.338092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:08.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342064+0000","last_change":"2024-09-19T09:25:29.298452+0000","last_active":"2024-09-19T09:27:11.342064+0000","last_peered":"2024-09-19T09:27:11.342064+0000","last_clean":"2024-09-19T09:27:11.342064+0000","last_became_active":"2024-09-19T09:25:29.298137+0000","last_became_peered":"2024-09-19T09:25:29.298137+0000","last_unstale":"2024-09-19T09:27:11.342064+0000","last_undegraded":"2024-09-19T09:27:11.342064+0000","last_fullsized":"2024-09-19T09:27:11.342064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:11.716228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337968+0000","last_change":"2024-09-19T09:25:29.294278+0000","last_active":"2024-09-19T09:27:11.337968+0000","last_peered":"2024-09-19T09:27:11.337968+0000","last_clean":"2024-09-19T09:27:11.337968+0000","last_became_active":"2024-09-19T09:25:29.293972+0000","last_became_peered":"2024-09-19T09:25:29.293972+0000","last_unstale":"2024-09-19T09:27:11.337968+0000","last_undegraded":"2024-09-19T09:27:11.337968+0000","last_fullsized":"2024-09-19T09:27:11.337968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:27.923839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346634+0000","last_change":"2024-09-19T09:25:29.294000+0000","last_active":"2024-09-19T09:27:11.346634+0000","last_peered":"2024-09-19T09:27:11.346634+0000","last_clean":"2024-09-19T09:27:11.346634+0000","last_became_active":"2024-09-19T09:25:29.293486+0000","last_became_peered":"2024-09-19T09:25:29.293486+0000","last_unstale":"2024-09-19T09:27:11.346634+0000","last_undegraded":"2024-09-19T09:27:11.346634+0000","last_fullsized":"2024-09-19T09:27:11.346634+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:44.053620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334040+0000","last_change":"2024-09-19T09:25:42.994951+0000","last_active":"2024-09-19T09:27:11.334040+0000","last_peered":"2024-09-19T09:27:11.334040+0000","last_clean":"2024-09-19T09:27:11.334040+0000","last_became_active":"2024-09-19T09:25:42.994588+0000","last_became_peered":"2024-09-19T09:25:42.994588+0000","last_unstale":"2024-09-19T09:27:11.334040+0000","last_undegraded":"2024-09-19T09:27:11.334040+0000","last_fullsized":"2024-09-19T09:27:11.334040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:10.141169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334658+0000","last_change":"2024-09-19T09:25:29.286395+0000","last_active":"2024-09-19T09:27:11.334658+0000","last_peered":"2024-09-19T09:27:11.334658+0000","last_clean":"2024-09-19T09:27:11.334658+0000","last_became_active":"2024-09-19T09:25:29.286048+0000","last_became_peered":"2024-09-19T09:25:29.286048+0000","last_unstale":"2024-09-19T09:27:11.334658+0000","last_undegraded":"2024-09-19T09:27:11.334658+0000","last_fullsized":"2024-09-19T09:27:11.334658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:43.121166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.327390+0000","last_change":"2024-09-19T09:25:29.298458+0000","last_active":"2024-09-19T09:27:11.327390+0000","last_peered":"2024-09-19T09:27:11.327390+0000","last_clean":"2024-09-19T09:27:11.327390+0000","last_became_active":"2024-09-19T09:25:29.297993+0000","last_became_peered":"2024-09-19T09:25:29.297993+0000","last_unstale":"2024-09-19T09:27:11.327390+0000","last_undegraded":"2024-09-19T09:27:11.327390+0000","last_fullsized":"2024-09-19T09:27:11.327390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:44.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344574+0000","last_change":"2024-09-19T09:25:29.297167+0000","last_active":"2024-09-19T09:27:11.344574+0000","last_peered":"2024-09-19T09:27:11.344574+0000","last_clean":"2024-09-19T09:27:11.344574+0000","last_became_active":"2024-09-19T09:25:29.296777+0000","last_became_peered":"2024-09-19T09:25:29.296777+0000","last_unstale":"2024-09-19T09:27:11.344574+0000","last_undegraded":"2024-09-19T09:27:11.344574+0000","last_fullsized":"2024-09-19T09:27:11.344574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:40.581643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.339599+0000","last_change":"2024-09-19T09:25:29.296169+0000","last_active":"2024-09-19T09:27:11.339599+0000","last_peered":"2024-09-19T09:27:11.339599+0000","last_clean":"2024-09-19T09:27:11.339599+0000","last_became_active":"2024-09-19T09:25:29.295806+0000","last_became_peered":"2024-09-19T09:25:29.295806+0000","last_unstale":"2024-09-19T09:27:11.339599+0000","last_undegraded":"2024-09-19T09:27:11.339599+0000","last_fullsized":"2024-09-19T09:27:11.339599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:39.639884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334590+0000","last_change":"2024-09-19T09:25:29.297960+0000","last_active":"2024-09-19T09:27:11.334590+0000","last_peered":"2024-09-19T09:27:11.334590+0000","last_clean":"2024-09-19T09:27:11.334590+0000","last_became_active":"2024-09-19T09:25:29.297577+0000","last_became_peered":"2024-09-19T09:25:29.297577+0000","last_unstale":"2024-09-19T09:27:11.334590+0000","last_undegraded":"2024-09-19T09:27:11.334590+0000","last_fullsized":"2024-09-19T09:27:11.334590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:27.035504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335188+0000","last_change":"2024-09-19T09:25:29.297896+0000","last_active":"2024-09-19T09:27:11.335188+0000","last_peered":"2024-09-19T09:27:11.335188+0000","last_clean":"2024-09-19T09:27:11.335188+0000","last_became_active":"2024-09-19T09:25:29.297594+0000","last_became_peered":"2024-09-19T09:25:29.297594+0000","last_unstale":"2024-09-19T09:27:11.335188+0000","last_undegraded":"2024-09-19T09:27:11.335188+0000","last_fullsized":"2024-09-19T09:27:11.335188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.366325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334560+0000","last_change":"2024-09-19T09:25:29.301851+0000","last_active":"2024-09-19T09:27:11.334560+0000","last_peered":"2024-09-19T09:27:11.334560+0000","last_clean":"2024-09-19T09:27:11.334560+0000","last_became_active":"2024-09-19T09:25:29.299282+0000","last_became_peered":"2024-09-19T09:25:29.299282+0000","last_unstale":"2024-09-19T09:27:11.334560+0000","last_undegraded":"2024-09-19T09:27:11.334560+0000","last_fullsized":"2024-09-19T09:27:11.334560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:00.382114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342990+0000","last_change":"2024-09-19T09:25:29.303316+0000","last_active":"2024-09-19T09:27:11.342990+0000","last_peered":"2024-09-19T09:27:11.342990+0000","last_clean":"2024-09-19T09:27:11.342990+0000","last_became_active":"2024-09-19T09:25:29.302994+0000","last_became_peered":"2024-09-19T09:25:29.302994+0000","last_unstale":"2024-09-19T09:27:11.342990+0000","last_undegraded":"2024-09-19T09:27:11.342990+0000","last_fullsized":"2024-09-19T09:27:11.342990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:46.233991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341767+0000","last_change":"2024-09-19T09:25:29.293004+0000","last_active":"2024-09-19T09:27:11.341767+0000","last_peered":"2024-09-19T09:27:11.341767+0000","last_clean":"2024-09-19T09:27:11.341767+0000","last_became_active":"2024-09-19T09:25:29.292657+0000","last_became_peered":"2024-09-19T09:25:29.292657+0000","last_unstale":"2024-09-19T09:27:11.341767+0000","last_undegraded":"2024-09-19T09:27:11.341767+0000","last_fullsized":"2024-09-19T09:27:11.341767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:50.966477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.332473+0000","last_change":"2024-09-19T09:25:29.300434+0000","last_active":"2024-09-19T09:27:11.332473+0000","last_peered":"2024-09-19T09:27:11.332473+0000","last_clean":"2024-09-19T09:27:11.332473+0000","last_became_active":"2024-09-19T09:25:29.300053+0000","last_became_peered":"2024-09-19T09:25:29.300053+0000","last_unstale":"2024-09-19T09:27:11.332473+0000","last_undegraded":"2024-09-19T09:27:11.332473+0000","last_fullsized":"2024-09-19T09:27:11.332473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:13.283298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":132,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.106724+0000","last_change":"2024-09-19T09:25:27.280792+0000","last_active":"2024-09-19T09:27:16.106724+0000","last_peered":"2024-09-19T09:27:16.106724+0000","last_clean":"2024-09-19T09:27:16.106724+0000","last_became_active":"2024-09-19T09:25:27.280443+0000","last_became_peered":"2024-09-19T09:25:27.280443+0000","last_unstale":"2024-09-19T09:27:16.106724+0000","last_undegraded":"2024-09-19T09:27:16.106724+0000","last_fullsized":"2024-09-19T09:27:16.106724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.599512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334165+0000","last_change":"2024-09-19T09:25:29.298975+0000","last_active":"2024-09-19T09:27:11.334165+0000","last_peered":"2024-09-19T09:27:11.334165+0000","last_clean":"2024-09-19T09:27:11.334165+0000","last_became_active":"2024-09-19T09:25:29.298476+0000","last_became_peered":"2024-09-19T09:25:29.298476+0000","last_unstale":"2024-09-19T09:27:11.334165+0000","last_undegraded":"2024-09-19T09:27:11.334165+0000","last_fullsized":"2024-09-19T09:27:11.334165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:09.658894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986623+0000","last_change":"2024-09-19T09:25:27.278257+0000","last_active":"2024-09-19T09:25:42.986623+0000","last_peered":"2024-09-19T09:25:42.986623+0000","last_clean":"2024-09-19T09:25:42.986623+0000","last_became_active":"2024-09-19T09:25:27.277897+0000","last_became_peered":"2024-09-19T09:25:27.277897+0000","last_unstale":"2024-09-19T09:25:42.986623+0000","last_undegraded":"2024-09-19T09:25:42.986623+0000","last_fullsized":"2024-09-19T09:25:42.986623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:47.550314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985617+0000","last_change":"2024-09-19T09:25:27.272215+0000","last_active":"2024-09-19T09:25:42.985617+0000","last_peered":"2024-09-19T09:25:42.985617+0000","last_clean":"2024-09-19T09:25:42.985617+0000","last_became_active":"2024-09-19T09:25:27.271821+0000","last_became_peered":"2024-09-19T09:25:27.271821+0000","last_unstale":"2024-09-19T09:25:42.985617+0000","last_undegraded":"2024-09-19T09:25:42.985617+0000","last_fullsized":"2024-09-19T09:25:42.985617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:30.615730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.329744+0000","last_change":"2024-09-19T09:25:29.289347+0000","last_active":"2024-09-19T09:27:11.329744+0000","last_peered":"2024-09-19T09:27:11.329744+0000","last_clean":"2024-09-19T09:27:11.329744+0000","last_became_active":"2024-09-19T09:25:29.289015+0000","last_became_peered":"2024-09-19T09:25:29.289015+0000","last_unstale":"2024-09-19T09:27:11.329744+0000","last_undegraded":"2024-09-19T09:27:11.329744+0000","last_fullsized":"2024-09-19T09:27:11.329744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:18.419334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995432+0000","last_change":"2024-09-19T09:25:27.277342+0000","last_active":"2024-09-19T09:25:42.995432+0000","last_peered":"2024-09-19T09:25:42.995432+0000","last_clean":"2024-09-19T09:25:42.995432+0000","last_became_active":"2024-09-19T09:25:27.277016+0000","last_became_peered":"2024-09-19T09:25:27.277016+0000","last_unstale":"2024-09-19T09:25:42.995432+0000","last_undegraded":"2024-09-19T09:25:42.995432+0000","last_fullsized":"2024-09-19T09:25:42.995432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:56.736156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336629+0000","last_change":"2024-09-19T09:25:29.290924+0000","last_active":"2024-09-19T09:27:11.336629+0000","last_peered":"2024-09-19T09:27:11.336629+0000","last_clean":"2024-09-19T09:27:11.336629+0000","last_became_active":"2024-09-19T09:25:29.290482+0000","last_became_peered":"2024-09-19T09:25:29.290482+0000","last_unstale":"2024-09-19T09:27:11.336629+0000","last_undegraded":"2024-09-19T09:27:11.336629+0000","last_fullsized":"2024-09-19T09:27:11.336629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:16.265040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'6","reported_seq":222,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.105891+0000","last_change":"2024-09-19T09:25:27.277604+0000","last_active":"2024-09-19T09:27:16.105891+0000","last_peered":"2024-09-19T09:27:16.105891+0000","last_clean":"2024-09-19T09:27:16.105891+0000","last_became_active":"2024-09-19T09:25:27.277271+0000","last_became_peered":"2024-09-19T09:25:27.277271+0000","last_unstale":"2024-09-19T09:27:16.105891+0000","last_undegraded":"2024-09-19T09:27:16.105891+0000","last_fullsized":"2024-09-19T09:27:16.105891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:32.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347371+0000","last_change":"2024-09-19T09:25:29.290537+0000","last_active":"2024-09-19T09:27:11.347371+0000","last_peered":"2024-09-19T09:27:11.347371+0000","last_clean":"2024-09-19T09:27:11.347371+0000","last_became_active":"2024-09-19T09:25:29.287984+0000","last_became_peered":"2024-09-19T09:25:29.287984+0000","last_unstale":"2024-09-19T09:27:11.347371+0000","last_undegraded":"2024-09-19T09:27:11.347371+0000","last_fullsized":"2024-09-19T09:27:11.347371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:26.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990018+0000","last_change":"2024-09-19T09:25:27.278560+0000","last_active":"2024-09-19T09:25:42.990018+0000","last_peered":"2024-09-19T09:25:42.990018+0000","last_clean":"2024-09-19T09:25:42.990018+0000","last_became_active":"2024-09-19T09:25:27.277950+0000","last_became_peered":"2024-09-19T09:25:27.277950+0000","last_unstale":"2024-09-19T09:25:42.990018+0000","last_undegraded":"2024-09-19T09:25:42.990018+0000","last_fullsized":"2024-09-19T09:25:42.990018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:28.387725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.339317+0000","last_change":"2024-09-19T09:25:29.298897+0000","last_active":"2024-09-19T09:27:11.339317+0000","last_peered":"2024-09-19T09:27:11.339317+0000","last_clean":"2024-09-19T09:27:11.339317+0000","last_became_active":"2024-09-19T09:25:29.298617+0000","last_became_peered":"2024-09-19T09:25:29.298617+0000","last_unstale":"2024-09-19T09:27:11.339317+0000","last_undegraded":"2024-09-19T09:27:11.339317+0000","last_fullsized":"2024-09-19T09:27:11.339317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:53.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994870+0000","last_change":"2024-09-19T09:25:27.324871+0000","last_active":"2024-09-19T09:25:42.994870+0000","last_peered":"2024-09-19T09:25:42.994870+0000","last_clean":"2024-09-19T09:25:42.994870+0000","last_became_active":"2024-09-19T09:25:27.324471+0000","last_became_peered":"2024-09-19T09:25:27.324471+0000","last_unstale":"2024-09-19T09:25:42.994870+0000","last_undegraded":"2024-09-19T09:25:42.994870+0000","last_fullsized":"2024-09-19T09:25:42.994870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:35.936413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347260+0000","last_change":"2024-09-19T09:25:29.298332+0000","last_active":"2024-09-19T09:27:11.347260+0000","last_peered":"2024-09-19T09:27:11.347260+0000","last_clean":"2024-09-19T09:27:11.347260+0000","last_became_active":"2024-09-19T09:25:29.298006+0000","last_became_peered":"2024-09-19T09:25:29.298006+0000","last_unstale":"2024-09-19T09:27:11.347260+0000","last_undegraded":"2024-09-19T09:27:11.347260+0000","last_fullsized":"2024-09-19T09:27:11.347260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:02.621821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'33","reported_seq":261,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.107843+0000","last_change":"2024-09-19T09:25:27.279528+0000","last_active":"2024-09-19T09:27:16.107843+0000","last_peered":"2024-09-19T09:27:16.107843+0000","last_clean":"2024-09-19T09:27:16.107843+0000","last_became_active":"2024-09-19T09:25:27.279149+0000","last_became_peered":"2024-09-19T09:25:27.279149+0000","last_unstale":"2024-09-19T09:27:16.107843+0000","last_undegraded":"2024-09-19T09:27:16.107843+0000","last_fullsized":"2024-09-19T09:27:16.107843+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:47.030736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60013,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":206,"num_read_kb":161,"num_write":35,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.324455+0000","last_change":"2024-09-19T09:25:29.297538+0000","last_active":"2024-09-19T09:27:11.324455+0000","last_peered":"2024-09-19T09:27:11.324455+0000","last_clean":"2024-09-19T09:27:11.324455+0000","last_became_active":"2024-09-19T09:25:29.297181+0000","last_became_peered":"2024-09-19T09:25:29.297181+0000","last_unstale":"2024-09-19T09:27:11.324455+0000","last_undegraded":"2024-09-19T09:27:11.324455+0000","last_fullsized":"2024-09-19T09:27:11.324455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:53.135438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992860+0000","last_change":"2024-09-19T09:25:27.277090+0000","last_active":"2024-09-19T09:25:42.992860+0000","last_peered":"2024-09-19T09:25:42.992860+0000","last_clean":"2024-09-19T09:25:42.992860+0000","last_became_active":"2024-09-19T09:25:27.276718+0000","last_became_peered":"2024-09-19T09:25:27.276718+0000","last_unstale":"2024-09-19T09:25:42.992860+0000","last_undegraded":"2024-09-19T09:25:42.992860+0000","last_fullsized":"2024-09-19T09:25:42.992860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:25.664261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.325840+0000","last_change":"2024-09-19T09:25:29.294393+0000","last_active":"2024-09-19T09:27:11.325840+0000","last_peered":"2024-09-19T09:27:11.325840+0000","last_clean":"2024-09-19T09:27:11.325840+0000","last_became_active":"2024-09-19T09:25:29.294099+0000","last_became_peered":"2024-09-19T09:25:29.294099+0000","last_unstale":"2024-09-19T09:27:11.325840+0000","last_undegraded":"2024-09-19T09:27:11.325840+0000","last_fullsized":"2024-09-19T09:27:11.325840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:17.333736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995398+0000","last_change":"2024-09-19T09:25:27.278593+0000","last_active":"2024-09-19T09:25:42.995398+0000","last_peered":"2024-09-19T09:25:42.995398+0000","last_clean":"2024-09-19T09:25:42.995398+0000","last_became_active":"2024-09-19T09:25:27.278232+0000","last_became_peered":"2024-09-19T09:25:27.278232+0000","last_unstale":"2024-09-19T09:25:42.995398+0000","last_undegraded":"2024-09-19T09:25:42.995398+0000","last_fullsized":"2024-09-19T09:25:42.995398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:05.852671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343581+0000","last_change":"2024-09-19T09:25:29.300024+0000","last_active":"2024-09-19T09:27:11.343581+0000","last_peered":"2024-09-19T09:27:11.343581+0000","last_clean":"2024-09-19T09:27:11.343581+0000","last_became_active":"2024-09-19T09:25:29.299683+0000","last_became_peered":"2024-09-19T09:25:29.299683+0000","last_unstale":"2024-09-19T09:27:11.343581+0000","last_undegraded":"2024-09-19T09:27:11.343581+0000","last_fullsized":"2024-09-19T09:27:11.343581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:42.249399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'7","reported_seq":131,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:15.107233+0000","last_change":"2024-09-19T09:25:27.272739+0000","last_active":"2024-09-19T09:27:15.107233+0000","last_peered":"2024-09-19T09:27:15.107233+0000","last_clean":"2024-09-19T09:27:15.107233+0000","last_became_active":"2024-09-19T09:25:27.272354+0000","last_became_peered":"2024-09-19T09:25:27.272354+0000","last_unstale":"2024-09-19T09:27:15.107233+0000","last_undegraded":"2024-09-19T09:27:15.107233+0000","last_fullsized":"2024-09-19T09:27:15.107233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:27.863840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343011+0000","last_change":"2024-09-19T09:25:29.303460+0000","last_active":"2024-09-19T09:27:11.343011+0000","last_peered":"2024-09-19T09:27:11.343011+0000","last_clean":"2024-09-19T09:27:11.343011+0000","last_became_active":"2024-09-19T09:25:29.303127+0000","last_became_peered":"2024-09-19T09:25:29.303127+0000","last_unstale":"2024-09-19T09:27:11.343011+0000","last_undegraded":"2024-09-19T09:27:11.343011+0000","last_fullsized":"2024-09-19T09:27:11.343011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:35.313924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990876+0000","last_change":"2024-09-19T09:25:27.323238+0000","last_active":"2024-09-19T09:25:42.990876+0000","last_peered":"2024-09-19T09:25:42.990876+0000","last_clean":"2024-09-19T09:25:42.990876+0000","last_became_active":"2024-09-19T09:25:27.322868+0000","last_became_peered":"2024-09-19T09:25:27.322868+0000","last_unstale":"2024-09-19T09:25:42.990876+0000","last_undegraded":"2024-09-19T09:25:42.990876+0000","last_fullsized":"2024-09-19T09:25:42.990876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:22.890280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338463+0000","last_change":"2024-09-19T09:25:43.005569+0000","last_active":"2024-09-19T09:27:11.338463+0000","last_peered":"2024-09-19T09:27:11.338463+0000","last_clean":"2024-09-19T09:27:11.338463+0000","last_became_active":"2024-09-19T09:25:43.005143+0000","last_became_peered":"2024-09-19T09:25:43.005143+0000","last_unstale":"2024-09-19T09:27:11.338463+0000","last_undegraded":"2024-09-19T09:27:11.338463+0000","last_fullsized":"2024-09-19T09:27:11.338463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:26.297013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991438+0000","last_change":"2024-09-19T09:25:27.282737+0000","last_active":"2024-09-19T09:25:42.991438+0000","last_peered":"2024-09-19T09:25:42.991438+0000","last_clean":"2024-09-19T09:25:42.991438+0000","last_became_active":"2024-09-19T09:25:27.281785+0000","last_became_peered":"2024-09-19T09:25:27.281785+0000","last_unstale":"2024-09-19T09:25:42.991438+0000","last_undegraded":"2024-09-19T09:25:42.991438+0000","last_fullsized":"2024-09-19T09:25:42.991438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:41.552206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.340095+0000","last_change":"2024-09-19T09:25:29.300168+0000","last_active":"2024-09-19T09:27:11.340095+0000","last_peered":"2024-09-19T09:27:11.340095+0000","last_clean":"2024-09-19T09:27:11.340095+0000","last_became_active":"2024-09-19T09:25:29.299811+0000","last_became_peered":"2024-09-19T09:25:29.299811+0000","last_unstale":"2024-09-19T09:27:11.340095+0000","last_undegraded":"2024-09-19T09:27:11.340095+0000","last_fullsized":"2024-09-19T09:27:11.340095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:06.691259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985783+0000","last_change":"2024-09-19T09:25:27.278433+0000","last_active":"2024-09-19T09:25:42.985783+0000","last_peered":"2024-09-19T09:25:42.985783+0000","last_clean":"2024-09-19T09:25:42.985783+0000","last_became_active":"2024-09-19T09:25:27.278076+0000","last_became_peered":"2024-09-19T09:25:27.278076+0000","last_unstale":"2024-09-19T09:25:42.985783+0000","last_undegraded":"2024-09-19T09:25:42.985783+0000","last_fullsized":"2024-09-19T09:25:42.985783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:53.799265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338515+0000","last_change":"2024-09-19T09:25:29.295650+0000","last_active":"2024-09-19T09:27:11.338515+0000","last_peered":"2024-09-19T09:27:11.338515+0000","last_clean":"2024-09-19T09:27:11.338515+0000","last_became_active":"2024-09-19T09:25:29.295277+0000","last_became_peered":"2024-09-19T09:25:29.295277+0000","last_unstale":"2024-09-19T09:27:11.338515+0000","last_undegraded":"2024-09-19T09:27:11.338515+0000","last_fullsized":"2024-09-19T09:27:11.338515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:57.690699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996123+0000","last_change":"2024-09-19T09:25:27.279239+0000","last_active":"2024-09-19T09:25:42.996123+0000","last_peered":"2024-09-19T09:25:42.996123+0000","last_clean":"2024-09-19T09:25:42.996123+0000","last_became_active":"2024-09-19T09:25:27.278868+0000","last_became_peered":"2024-09-19T09:25:27.278868+0000","last_unstale":"2024-09-19T09:25:42.996123+0000","last_undegraded":"2024-09-19T09:25:42.996123+0000","last_fullsized":"2024-09-19T09:25:42.996123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:41.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341911+0000","last_change":"2024-09-19T09:25:29.300833+0000","last_active":"2024-09-19T09:27:11.341911+0000","last_peered":"2024-09-19T09:27:11.341911+0000","last_clean":"2024-09-19T09:27:11.341911+0000","last_became_active":"2024-09-19T09:25:29.300472+0000","last_became_peered":"2024-09-19T09:25:29.300472+0000","last_unstale":"2024-09-19T09:27:11.341911+0000","last_undegraded":"2024-09-19T09:27:11.341911+0000","last_fullsized":"2024-09-19T09:27:11.341911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:50.591276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'7","reported_seq":117,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:15.140661+0000","last_change":"2024-09-19T09:25:43.001722+0000","last_active":"2024-09-19T09:27:15.140661+0000","last_peered":"2024-09-19T09:27:15.140661+0000","last_clean":"2024-09-19T09:27:15.140661+0000","last_became_active":"2024-09-19T09:25:42.992796+0000","last_became_peered":"2024-09-19T09:25:42.992796+0000","last_unstale":"2024-09-19T09:27:15.140661+0000","last_undegraded":"2024-09-19T09:27:15.140661+0000","last_fullsized":"2024-09-19T09:27:15.140661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:59.639010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346495+0000","last_change":"2024-09-19T09:25:29.295906+0000","last_active":"2024-09-19T09:27:11.346495+0000","last_peered":"2024-09-19T09:27:11.346495+0000","last_clean":"2024-09-19T09:27:11.346495+0000","last_became_active":"2024-09-19T09:25:29.295565+0000","last_became_peered":"2024-09-19T09:25:29.295565+0000","last_unstale":"2024-09-19T09:27:11.346495+0000","last_undegraded":"2024-09-19T09:27:11.346495+0000","last_fullsized":"2024-09-19T09:27:11.346495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:24.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989907+0000","last_change":"2024-09-19T09:24:19.256300+0000","last_active":"2024-09-19T09:25:42.989907+0000","last_peered":"2024-09-19T09:25:42.989907+0000","last_clean":"2024-09-19T09:25:42.989907+0000","last_became_active":"2024-09-19T09:24:19.255833+0000","last_became_peered":"2024-09-19T09:24:19.255833+0000","last_unstale":"2024-09-19T09:25:42.989907+0000","last_undegraded":"2024-09-19T09:25:42.989907+0000","last_fullsized":"2024-09-19T09:25:42.989907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_clean_scrub_stamp":"2024-09-19T09:20:57.798838+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:14:33.343748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989637+0000","last_change":"2024-09-19T09:25:27.272090+0000","last_active":"2024-09-19T09:25:42.989637+0000","last_peered":"2024-09-19T09:25:42.989637+0000","last_clean":"2024-09-19T09:25:42.989637+0000","last_became_active":"2024-09-19T09:25:27.271651+0000","last_became_peered":"2024-09-19T09:25:27.271651+0000","last_unstale":"2024-09-19T09:25:42.989637+0000","last_undegraded":"2024-09-19T09:25:42.989637+0000","last_fullsized":"2024-09-19T09:25:42.989637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:45.847082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344375+0000","last_change":"2024-09-19T09:25:29.300718+0000","last_active":"2024-09-19T09:27:11.344375+0000","last_peered":"2024-09-19T09:27:11.344375+0000","last_clean":"2024-09-19T09:27:11.344375+0000","last_became_active":"2024-09-19T09:25:29.299460+0000","last_became_peered":"2024-09-19T09:25:29.299460+0000","last_unstale":"2024-09-19T09:27:11.344375+0000","last_undegraded":"2024-09-19T09:27:11.344375+0000","last_fullsized":"2024-09-19T09:27:11.344375+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:43.063909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003299+0000","last_change":"2024-09-19T09:25:43.003299+0000","last_active":"2024-09-19T09:25:43.003299+0000","last_peered":"2024-09-19T09:25:43.003299+0000","last_clean":"2024-09-19T09:25:43.003299+0000","last_became_active":"2024-09-19T09:25:42.999686+0000","last_became_peered":"2024-09-19T09:25:42.999686+0000","last_unstale":"2024-09-19T09:25:43.003299+0000","last_undegraded":"2024-09-19T09:25:43.003299+0000","last_fullsized":"2024-09-19T09:25:43.003299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:05.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337734+0000","last_change":"2024-09-19T09:25:29.297833+0000","last_active":"2024-09-19T09:27:11.337734+0000","last_peered":"2024-09-19T09:27:11.337734+0000","last_clean":"2024-09-19T09:27:11.337734+0000","last_became_active":"2024-09-19T09:25:29.297534+0000","last_became_peered":"2024-09-19T09:25:29.297534+0000","last_unstale":"2024-09-19T09:27:11.337734+0000","last_undegraded":"2024-09-19T09:27:11.337734+0000","last_fullsized":"2024-09-19T09:27:11.337734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:47.156253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":129,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:17.109301+0000","last_change":"2024-09-19T09:25:27.322759+0000","last_active":"2024-09-19T09:27:17.109301+0000","last_peered":"2024-09-19T09:27:17.109301+0000","last_clean":"2024-09-19T09:27:17.109301+0000","last_became_active":"2024-09-19T09:25:27.322402+0000","last_became_peered":"2024-09-19T09:25:27.322402+0000","last_unstale":"2024-09-19T09:27:17.109301+0000","last_undegraded":"2024-09-19T09:27:17.109301+0000","last_fullsized":"2024-09-19T09:27:17.109301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:06.276957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335476+0000","last_change":"2024-09-19T09:25:42.999834+0000","last_active":"2024-09-19T09:27:11.335476+0000","last_peered":"2024-09-19T09:27:11.335476+0000","last_clean":"2024-09-19T09:27:11.335476+0000","last_became_active":"2024-09-19T09:25:42.999450+0000","last_became_peered":"2024-09-19T09:25:42.999450+0000","last_unstale":"2024-09-19T09:27:11.335476+0000","last_undegraded":"2024-09-19T09:27:11.335476+0000","last_fullsized":"2024-09-19T09:27:11.335476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:00.509844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'7","reported_seq":129,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.117110+0000","last_change":"2024-09-19T09:25:27.275212+0000","last_active":"2024-09-19T09:27:16.117110+0000","last_peered":"2024-09-19T09:27:16.117110+0000","last_clean":"2024-09-19T09:27:16.117110+0000","last_became_active":"2024-09-19T09:25:27.274802+0000","last_became_peered":"2024-09-19T09:25:27.274802+0000","last_unstale":"2024-09-19T09:27:16.117110+0000","last_undegraded":"2024-09-19T09:27:16.117110+0000","last_fullsized":"2024-09-19T09:27:16.117110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:48.392026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.333364+0000","last_change":"2024-09-19T09:25:29.304747+0000","last_active":"2024-09-19T09:27:11.333364+0000","last_peered":"2024-09-19T09:27:11.333364+0000","last_clean":"2024-09-19T09:27:11.333364+0000","last_became_active":"2024-09-19T09:25:29.304386+0000","last_became_peered":"2024-09-19T09:25:29.304386+0000","last_unstale":"2024-09-19T09:27:11.333364+0000","last_undegraded":"2024-09-19T09:27:11.333364+0000","last_fullsized":"2024-09-19T09:27:11.333364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.788697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988854+0000","last_change":"2024-09-19T09:25:27.323083+0000","last_active":"2024-09-19T09:25:42.988854+0000","last_peered":"2024-09-19T09:25:42.988854+0000","last_clean":"2024-09-19T09:25:42.988854+0000","last_became_active":"2024-09-19T09:25:27.322707+0000","last_became_peered":"2024-09-19T09:25:27.322707+0000","last_unstale":"2024-09-19T09:25:42.988854+0000","last_undegraded":"2024-09-19T09:25:42.988854+0000","last_fullsized":"2024-09-19T09:25:42.988854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:24.415116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334132+0000","last_change":"2024-09-19T09:25:29.292472+0000","last_active":"2024-09-19T09:27:11.334132+0000","last_peered":"2024-09-19T09:27:11.334132+0000","last_clean":"2024-09-19T09:27:11.334132+0000","last_became_active":"2024-09-19T09:25:29.292079+0000","last_became_peered":"2024-09-19T09:25:29.292079+0000","last_unstale":"2024-09-19T09:27:11.334132+0000","last_undegraded":"2024-09-19T09:27:11.334132+0000","last_fullsized":"2024-09-19T09:27:11.334132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:35.801594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991224+0000","last_change":"2024-09-19T09:25:27.324839+0000","last_active":"2024-09-19T09:25:42.991224+0000","last_peered":"2024-09-19T09:25:42.991224+0000","last_clean":"2024-09-19T09:25:42.991224+0000","last_became_active":"2024-09-19T09:25:27.324475+0000","last_became_peered":"2024-09-19T09:25:27.324475+0000","last_unstale":"2024-09-19T09:25:42.991224+0000","last_undegraded":"2024-09-19T09:25:42.991224+0000","last_fullsized":"2024-09-19T09:25:42.991224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:32.251753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338007+0000","last_change":"2024-09-19T09:25:29.293025+0000","last_active":"2024-09-19T09:27:11.338007+0000","last_peered":"2024-09-19T09:27:11.338007+0000","last_clean":"2024-09-19T09:27:11.338007+0000","last_became_active":"2024-09-19T09:25:29.292570+0000","last_became_peered":"2024-09-19T09:25:29.292570+0000","last_unstale":"2024-09-19T09:27:11.338007+0000","last_undegraded":"2024-09-19T09:27:11.338007+0000","last_fullsized":"2024-09-19T09:27:11.338007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:16.394524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:55.295924+0000","last_change":"2024-09-19T09:25:27.278100+0000","last_active":"2024-09-19T09:25:55.295924+0000","last_peered":"2024-09-19T09:25:55.295924+0000","last_clean":"2024-09-19T09:25:55.295924+0000","last_became_active":"2024-09-19T09:25:27.277769+0000","last_became_peered":"2024-09-19T09:25:27.277769+0000","last_unstale":"2024-09-19T09:25:55.295924+0000","last_undegraded":"2024-09-19T09:25:55.295924+0000","last_fullsized":"2024-09-19T09:25:55.295924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:31.885142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338230+0000","last_change":"2024-09-19T09:25:29.293651+0000","last_active":"2024-09-19T09:27:11.338230+0000","last_peered":"2024-09-19T09:27:11.338230+0000","last_clean":"2024-09-19T09:27:11.338230+0000","last_became_active":"2024-09-19T09:25:29.293317+0000","last_became_peered":"2024-09-19T09:25:29.293317+0000","last_unstale":"2024-09-19T09:27:11.338230+0000","last_undegraded":"2024-09-19T09:27:11.338230+0000","last_fullsized":"2024-09-19T09:27:11.338230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:40.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990030+0000","last_change":"2024-09-19T09:25:27.269367+0000","last_active":"2024-09-19T09:25:42.990030+0000","last_peered":"2024-09-19T09:25:42.990030+0000","last_clean":"2024-09-19T09:25:42.990030+0000","last_became_active":"2024-09-19T09:25:27.269011+0000","last_became_peered":"2024-09-19T09:25:27.269011+0000","last_unstale":"2024-09-19T09:25:42.990030+0000","last_undegraded":"2024-09-19T09:25:42.990030+0000","last_fullsized":"2024-09-19T09:25:42.990030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:00.930752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334968+0000","last_change":"2024-09-19T09:25:29.299134+0000","last_active":"2024-09-19T09:27:11.334968+0000","last_peered":"2024-09-19T09:27:11.334968+0000","last_clean":"2024-09-19T09:27:11.334968+0000","last_became_active":"2024-09-19T09:25:29.298648+0000","last_became_peered":"2024-09-19T09:25:29.298648+0000","last_unstale":"2024-09-19T09:27:11.334968+0000","last_undegraded":"2024-09-19T09:27:11.334968+0000","last_fullsized":"2024-09-19T09:27:11.334968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:39.627852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003686+0000","last_change":"2024-09-19T09:25:27.322861+0000","last_active":"2024-09-19T09:25:43.003686+0000","last_peered":"2024-09-19T09:25:43.003686+0000","last_clean":"2024-09-19T09:25:43.003686+0000","last_became_active":"2024-09-19T09:25:27.322560+0000","last_became_peered":"2024-09-19T09:25:27.322560+0000","last_unstale":"2024-09-19T09:25:43.003686+0000","last_undegraded":"2024-09-19T09:25:43.003686+0000","last_fullsized":"2024-09-19T09:25:43.003686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:35.993093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.301522+0000","last_change":"2024-09-19T09:25:29.301680+0000","last_active":"2024-09-19T09:27:11.301522+0000","last_peered":"2024-09-19T09:27:11.301522+0000","last_clean":"2024-09-19T09:27:11.301522+0000","last_became_active":"2024-09-19T09:25:29.301331+0000","last_became_peered":"2024-09-19T09:25:29.301331+0000","last_unstale":"2024-09-19T09:27:11.301522+0000","last_undegraded":"2024-09-19T09:27:11.301522+0000","last_fullsized":"2024-09-19T09:27:11.301522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:57.364245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:27.272609+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:27.272169+0000","last_became_peered":"2024-09-19T09:25:27.272169+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.879474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341200+0000","last_change":"2024-09-19T09:25:29.296847+0000","last_active":"2024-09-19T09:27:11.341200+0000","last_peered":"2024-09-19T09:27:11.341200+0000","last_clean":"2024-09-19T09:27:11.341200+0000","last_became_active":"2024-09-19T09:25:29.296465+0000","last_became_peered":"2024-09-19T09:25:29.296465+0000","last_unstale":"2024-09-19T09:27:11.341200+0000","last_undegraded":"2024-09-19T09:27:11.341200+0000","last_fullsized":"2024-09-19T09:27:11.341200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:05:48.553252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.012954+0000","last_change":"2024-09-19T09:25:43.012532+0000","last_active":"2024-09-19T09:25:43.012954+0000","last_peered":"2024-09-19T09:25:43.012954+0000","last_clean":"2024-09-19T09:25:43.012954+0000","last_became_active":"2024-09-19T09:25:42.994996+0000","last_became_peered":"2024-09-19T09:25:42.994996+0000","last_unstale":"2024-09-19T09:25:43.012954+0000","last_undegraded":"2024-09-19T09:25:43.012954+0000","last_fullsized":"2024-09-19T09:25:43.012954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:25.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337187+0000","last_change":"2024-09-19T09:25:29.297633+0000","last_active":"2024-09-19T09:27:11.337187+0000","last_peered":"2024-09-19T09:27:11.337187+0000","last_clean":"2024-09-19T09:27:11.337187+0000","last_became_active":"2024-09-19T09:25:29.297284+0000","last_became_peered":"2024-09-19T09:25:29.297284+0000","last_unstale":"2024-09-19T09:27:11.337187+0000","last_undegraded":"2024-09-19T09:27:11.337187+0000","last_fullsized":"2024-09-19T09:27:11.337187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:50.582546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991172+0000","last_change":"2024-09-19T09:25:27.281778+0000","last_active":"2024-09-19T09:25:42.991172+0000","last_peered":"2024-09-19T09:25:42.991172+0000","last_clean":"2024-09-19T09:25:42.991172+0000","last_became_active":"2024-09-19T09:25:27.281280+0000","last_became_peered":"2024-09-19T09:25:27.281280+0000","last_unstale":"2024-09-19T09:25:42.991172+0000","last_undegraded":"2024-09-19T09:25:42.991172+0000","last_fullsized":"2024-09-19T09:25:42.991172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:46.811968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343342+0000","last_change":"2024-09-19T09:25:29.295663+0000","last_active":"2024-09-19T09:27:11.343342+0000","last_peered":"2024-09-19T09:27:11.343342+0000","last_clean":"2024-09-19T09:27:11.343342+0000","last_became_active":"2024-09-19T09:25:29.295146+0000","last_became_peered":"2024-09-19T09:25:29.295146+0000","last_unstale":"2024-09-19T09:27:11.343342+0000","last_undegraded":"2024-09-19T09:27:11.343342+0000","last_fullsized":"2024-09-19T09:27:11.343342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:36.749339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.345435+0000","last_change":"2024-09-19T09:25:29.298004+0000","last_active":"2024-09-19T09:27:11.345435+0000","last_peered":"2024-09-19T09:27:11.345435+0000","last_clean":"2024-09-19T09:27:11.345435+0000","last_became_active":"2024-09-19T09:25:29.297600+0000","last_became_peered":"2024-09-19T09:25:29.297600+0000","last_unstale":"2024-09-19T09:27:11.345435+0000","last_undegraded":"2024-09-19T09:27:11.345435+0000","last_fullsized":"2024-09-19T09:27:11.345435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:09.738615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992323+0000","last_change":"2024-09-19T09:25:27.282876+0000","last_active":"2024-09-19T09:25:42.992323+0000","last_peered":"2024-09-19T09:25:42.992323+0000","last_clean":"2024-09-19T09:25:42.992323+0000","last_became_active":"2024-09-19T09:25:27.282201+0000","last_became_peered":"2024-09-19T09:25:27.282201+0000","last_unstale":"2024-09-19T09:25:42.992323+0000","last_undegraded":"2024-09-19T09:25:42.992323+0000","last_fullsized":"2024-09-19T09:25:42.992323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:08.508662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347692+0000","last_change":"2024-09-19T09:25:29.288379+0000","last_active":"2024-09-19T09:27:11.347692+0000","last_peered":"2024-09-19T09:27:11.347692+0000","last_clean":"2024-09-19T09:27:11.347692+0000","last_became_active":"2024-09-19T09:25:29.287917+0000","last_became_peered":"2024-09-19T09:25:29.287917+0000","last_unstale":"2024-09-19T09:27:11.347692+0000","last_undegraded":"2024-09-19T09:27:11.347692+0000","last_fullsized":"2024-09-19T09:27:11.347692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:03.628233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991076+0000","last_change":"2024-09-19T09:25:27.279551+0000","last_active":"2024-09-19T09:25:42.991076+0000","last_peered":"2024-09-19T09:25:42.991076+0000","last_clean":"2024-09-19T09:25:42.991076+0000","last_became_active":"2024-09-19T09:25:27.278956+0000","last_became_peered":"2024-09-19T09:25:27.278956+0000","last_unstale":"2024-09-19T09:25:42.991076+0000","last_undegraded":"2024-09-19T09:25:42.991076+0000","last_fullsized":"2024-09-19T09:25:42.991076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:17.186453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347269+0000","last_change":"2024-09-19T09:25:29.288657+0000","last_active":"2024-09-19T09:27:11.347269+0000","last_peered":"2024-09-19T09:27:11.347269+0000","last_clean":"2024-09-19T09:27:11.347269+0000","last_became_active":"2024-09-19T09:25:29.288355+0000","last_became_peered":"2024-09-19T09:25:29.288355+0000","last_unstale":"2024-09-19T09:27:11.347269+0000","last_undegraded":"2024-09-19T09:27:11.347269+0000","last_fullsized":"2024-09-19T09:27:11.347269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:46.237558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991315+0000","last_change":"2024-09-19T09:25:27.323312+0000","last_active":"2024-09-19T09:25:42.991315+0000","last_peered":"2024-09-19T09:25:42.991315+0000","last_clean":"2024-09-19T09:25:42.991315+0000","last_became_active":"2024-09-19T09:25:27.322984+0000","last_became_peered":"2024-09-19T09:25:27.322984+0000","last_unstale":"2024-09-19T09:25:42.991315+0000","last_undegraded":"2024-09-19T09:25:42.991315+0000","last_fullsized":"2024-09-19T09:25:42.991315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:48.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.327987+0000","last_change":"2024-09-19T09:25:29.301832+0000","last_active":"2024-09-19T09:27:11.327987+0000","last_peered":"2024-09-19T09:27:11.327987+0000","last_clean":"2024-09-19T09:27:11.327987+0000","last_became_active":"2024-09-19T09:25:29.301539+0000","last_became_peered":"2024-09-19T09:25:29.301539+0000","last_unstale":"2024-09-19T09:27:11.327987+0000","last_undegraded":"2024-09-19T09:27:11.327987+0000","last_fullsized":"2024-09-19T09:27:11.327987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:00.244948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'14","reported_seq":154,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.116536+0000","last_change":"2024-09-19T09:25:43.009668+0000","last_active":"2024-09-19T09:27:16.116536+0000","last_peered":"2024-09-19T09:27:16.116536+0000","last_clean":"2024-09-19T09:27:16.116536+0000","last_became_active":"2024-09-19T09:25:42.994966+0000","last_became_peered":"2024-09-19T09:25:42.994966+0000","last_unstale":"2024-09-19T09:27:16.116536+0000","last_undegraded":"2024-09-19T09:27:16.116536+0000","last_fullsized":"2024-09-19T09:27:16.116536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:04:05.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68571,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1020,"num_read_kb":587,"num_write":258,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1842,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612644,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":431506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841576,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1593344,"data_stored":1081568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005100,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":917504,"data_stored":429922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234033,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":430300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1023357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593788,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":901120,"data_stored":430173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1022639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":431738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215178,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":490149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":491099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":431512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:27:22.502 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T09:27:22.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T09:27:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.638+0000 7fa3454a5640 1 -- 172.21.15.16:0/2577839225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3401036e0 msgr2=0x7fa340109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.638+0000 7fa3454a5640 1 --2- 172.21.15.16:0/2577839225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3401036e0 0x7fa340109f70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa338009900 tx=0x7fa33802a410 comp rx=0 tx=0).stop 2024-09-19T09:27:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.638+0000 7fa3454a5640 1 -- 172.21.15.16:0/2577839225 shutdown_connections 2024-09-19T09:27:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.638+0000 7fa3454a5640 1 --2- 172.21.15.16:0/2577839225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3401036e0 0x7fa340109f70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.638+0000 7fa3454a5640 1 --2- 172.21.15.16:0/2577839225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa340102d20 0x7fa3401031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.638+0000 7fa3454a5640 1 --2- 172.21.15.16:0/2577839225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa340101b30 0x7fa340101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.638+0000 7fa3454a5640 1 -- 172.21.15.16:0/2577839225 >> 172.21.15.16:0/2577839225 conn(0x7fa3400fd270 msgr2=0x7fa3400ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.638+0000 7fa3454a5640 1 -- 172.21.15.16:0/2577839225 shutdown_connections 2024-09-19T09:27:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.638+0000 7fa3454a5640 1 -- 172.21.15.16:0/2577839225 wait complete. 2024-09-19T09:27:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.639+0000 7fa3454a5640 1 Processor -- start 2024-09-19T09:27:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.639+0000 7fa3454a5640 1 -- start start 2024-09-19T09:27:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.639+0000 7fa3454a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa340101b30 0x7fa340197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.639+0000 7fa3454a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa340102d20 0x7fa340198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.639+0000 7fa3454a5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3401036e0 0x7fa34019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.639+0000 7fa3454a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa34010d7e0 con 0x7fa340101b30 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.639+0000 7fa3454a5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa34010d660 con 0x7fa340102d20 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.639+0000 7fa3454a5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa34010d960 con 0x7fa3401036e0 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.639+0000 7fa33effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa340101b30 0x7fa340197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.639+0000 7fa33effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa340101b30 0x7fa340197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39700/0 (socket says 172.21.15.16:39700) 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.639+0000 7fa33effd640 1 -- 172.21.15.16:0/3949308170 learned_addr learned my addr 172.21.15.16:0/3949308170 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.639+0000 7fa33f7fe640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3401036e0 0x7fa34019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.640+0000 7fa33e7fc640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa340102d20 0x7fa340198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.640+0000 7fa33effd640 1 -- 172.21.15.16:0/3949308170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3401036e0 msgr2=0x7fa34019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.640+0000 7fa33effd640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3401036e0 0x7fa34019f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.640+0000 7fa33effd640 1 -- 172.21.15.16:0/3949308170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa340102d20 msgr2=0x7fa340198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:22.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.640+0000 7fa33effd640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa340102d20 0x7fa340198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.640+0000 7fa33effd640 1 -- 172.21.15.16:0/3949308170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34019fa20 con 0x7fa340101b30 2024-09-19T09:27:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.640+0000 7fa33e7fc640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa340102d20 0x7fa340198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.640+0000 7fa33effd640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa340101b30 0x7fa340197e00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa330009fd0 tx=0x7fa33000c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.640+0000 7fa337fff640 1 -- 172.21.15.16:0/3949308170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330011470 con 0x7fa340101b30 2024-09-19T09:27:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.640+0000 7fa337fff640 1 -- 172.21.15.16:0/3949308170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa330011af0 con 0x7fa340101b30 2024-09-19T09:27:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.640+0000 7fa3454a5640 1 -- 172.21.15.16:0/3949308170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34019fd10 con 0x7fa340101b30 2024-09-19T09:27:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.640+0000 7fa337fff640 1 -- 172.21.15.16:0/3949308170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33001ab90 con 0x7fa340101b30 2024-09-19T09:27:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.640+0000 7fa3454a5640 1 -- 172.21.15.16:0/3949308170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3401a01f0 con 0x7fa340101b30 2024-09-19T09:27:22.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.641+0000 7fa3454a5640 1 -- 172.21.15.16:0/3949308170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa340108f20 con 0x7fa340101b30 2024-09-19T09:27:22.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.642+0000 7fa337fff640 1 -- 172.21.15.16:0/3949308170 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa330011610 con 0x7fa340101b30 2024-09-19T09:27:22.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.643+0000 7fa337fff640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa32c0a32d0 0x7fa32c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:22.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.643+0000 7fa33e7fc640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa32c0a32d0 0x7fa32c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:22.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.643+0000 7fa337fff640 1 -- 172.21.15.16:0/3949308170 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa3300d0100 con 0x7fa340101b30 2024-09-19T09:27:22.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.644+0000 7fa33e7fc640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa32c0a32d0 0x7fa32c0a5790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa328005fd0 tx=0x7fa3280094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:22.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.646+0000 7fa337fff640 1 -- 172.21.15.16:0/3949308170 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa330016030 con 0x7fa340101b30 2024-09-19T09:27:22.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.821+0000 7fa3454a5640 1 -- 172.21.15.16:0/3949308170 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa340062630 con 0x7fa32c0a32d0 2024-09-19T09:27:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.828+0000 7fa337fff640 1 -- 172.21.15.16:0/3949308170 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258890 (secure 0 0 0) 0x7fa340062630 con 0x7fa32c0a32d0 2024-09-19T09:27:22.832 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:22.834 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T09:27:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.831+0000 7fa3454a5640 1 -- 172.21.15.16:0/3949308170 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa32c0a32d0 msgr2=0x7fa32c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.831+0000 7fa3454a5640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa32c0a32d0 0x7fa32c0a5790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa328005fd0 tx=0x7fa3280094d0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.831+0000 7fa3454a5640 1 -- 172.21.15.16:0/3949308170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa340101b30 msgr2=0x7fa340197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.831+0000 7fa3454a5640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa340101b30 0x7fa340197e00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa330009fd0 tx=0x7fa33000c270 comp rx=0 tx=0).stop 2024-09-19T09:27:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.831+0000 7fa3454a5640 1 -- 172.21.15.16:0/3949308170 shutdown_connections 2024-09-19T09:27:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.831+0000 7fa3454a5640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa32c0a32d0 0x7fa32c0a5790 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.831+0000 7fa3454a5640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3401036e0 0x7fa34019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.831+0000 7fa3454a5640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa340102d20 0x7fa340198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.831+0000 7fa3454a5640 1 --2- 172.21.15.16:0/3949308170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa340101b30 0x7fa340197e00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.831+0000 7fa3454a5640 1 -- 172.21.15.16:0/3949308170 >> 172.21.15.16:0/3949308170 conn(0x7fa3400fd270 msgr2=0x7fa3401060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.831+0000 7fa3454a5640 1 -- 172.21.15.16:0/3949308170 shutdown_connections 2024-09-19T09:27:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:22.831+0000 7fa3454a5640 1 -- 172.21.15.16:0/3949308170 wait complete. 2024-09-19T09:27:22.843 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-19T09:27:21.365506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659173,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1146,"num_read_kb":696,"num_write":441,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1182578,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391004,"kb_used_data":12636,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550820,"statfs":{"total":1151940427776,"available":1151540039680,"internally_reserved":0,"allocated":12939264,"data_stored":7123427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":813,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":13,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003207"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347860+0000","last_change":"2024-09-19T09:25:29.294965+0000","last_active":"2024-09-19T09:27:11.347860+0000","last_peered":"2024-09-19T09:27:11.347860+0000","last_clean":"2024-09-19T09:27:11.347860+0000","last_became_active":"2024-09-19T09:25:29.294157+0000","last_became_peered":"2024-09-19T09:25:29.294157+0000","last_unstale":"2024-09-19T09:27:11.347860+0000","last_undegraded":"2024-09-19T09:27:11.347860+0000","last_fullsized":"2024-09-19T09:27:11.347860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:24.643835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335438+0000","last_change":"2024-09-19T09:25:29.298888+0000","last_active":"2024-09-19T09:27:11.335438+0000","last_peered":"2024-09-19T09:27:11.335438+0000","last_clean":"2024-09-19T09:27:11.335438+0000","last_became_active":"2024-09-19T09:25:29.298245+0000","last_became_peered":"2024-09-19T09:25:29.298245+0000","last_unstale":"2024-09-19T09:27:11.335438+0000","last_undegraded":"2024-09-19T09:27:11.335438+0000","last_fullsized":"2024-09-19T09:27:11.335438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:03.162342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344990+0000","last_change":"2024-09-19T09:25:29.292300+0000","last_active":"2024-09-19T09:27:11.344990+0000","last_peered":"2024-09-19T09:27:11.344990+0000","last_clean":"2024-09-19T09:27:11.344990+0000","last_became_active":"2024-09-19T09:25:29.291973+0000","last_became_peered":"2024-09-19T09:25:29.291973+0000","last_unstale":"2024-09-19T09:27:11.344990+0000","last_undegraded":"2024-09-19T09:27:11.344990+0000","last_fullsized":"2024-09-19T09:27:11.344990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:19.672778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342109+0000","last_change":"2024-09-19T09:25:29.305695+0000","last_active":"2024-09-19T09:27:11.342109+0000","last_peered":"2024-09-19T09:27:11.342109+0000","last_clean":"2024-09-19T09:27:11.342109+0000","last_became_active":"2024-09-19T09:25:29.305244+0000","last_became_peered":"2024-09-19T09:25:29.305244+0000","last_unstale":"2024-09-19T09:27:11.342109+0000","last_undegraded":"2024-09-19T09:27:11.342109+0000","last_fullsized":"2024-09-19T09:27:11.342109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:38.054624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336846+0000","last_change":"2024-09-19T09:25:29.295259+0000","last_active":"2024-09-19T09:27:11.336846+0000","last_peered":"2024-09-19T09:27:11.336846+0000","last_clean":"2024-09-19T09:27:11.336846+0000","last_became_active":"2024-09-19T09:25:29.294936+0000","last_became_peered":"2024-09-19T09:25:29.294936+0000","last_unstale":"2024-09-19T09:27:11.336846+0000","last_undegraded":"2024-09-19T09:27:11.336846+0000","last_fullsized":"2024-09-19T09:27:11.336846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:59.193504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336175+0000","last_change":"2024-09-19T09:25:29.299650+0000","last_active":"2024-09-19T09:27:11.336175+0000","last_peered":"2024-09-19T09:27:11.336175+0000","last_clean":"2024-09-19T09:27:11.336175+0000","last_became_active":"2024-09-19T09:25:29.299182+0000","last_became_peered":"2024-09-19T09:25:29.299182+0000","last_unstale":"2024-09-19T09:27:11.336175+0000","last_undegraded":"2024-09-19T09:27:11.336175+0000","last_fullsized":"2024-09-19T09:27:11.336175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:44.637771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342329+0000","last_change":"2024-09-19T09:25:29.302501+0000","last_active":"2024-09-19T09:27:11.342329+0000","last_peered":"2024-09-19T09:27:11.342329+0000","last_clean":"2024-09-19T09:27:11.342329+0000","last_became_active":"2024-09-19T09:25:29.301419+0000","last_became_peered":"2024-09-19T09:25:29.301419+0000","last_unstale":"2024-09-19T09:27:11.342329+0000","last_undegraded":"2024-09-19T09:27:11.342329+0000","last_fullsized":"2024-09-19T09:27:11.342329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:50.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337306+0000","last_change":"2024-09-19T09:25:29.300212+0000","last_active":"2024-09-19T09:27:11.337306+0000","last_peered":"2024-09-19T09:27:11.337306+0000","last_clean":"2024-09-19T09:27:11.337306+0000","last_became_active":"2024-09-19T09:25:29.299873+0000","last_became_peered":"2024-09-19T09:25:29.299873+0000","last_unstale":"2024-09-19T09:27:11.337306+0000","last_undegraded":"2024-09-19T09:27:11.337306+0000","last_fullsized":"2024-09-19T09:27:11.337306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:34.620314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346891+0000","last_change":"2024-09-19T09:25:29.290214+0000","last_active":"2024-09-19T09:27:11.346891+0000","last_peered":"2024-09-19T09:27:11.346891+0000","last_clean":"2024-09-19T09:27:11.346891+0000","last_became_active":"2024-09-19T09:25:29.289707+0000","last_became_peered":"2024-09-19T09:25:29.289707+0000","last_unstale":"2024-09-19T09:27:11.346891+0000","last_undegraded":"2024-09-19T09:27:11.346891+0000","last_fullsized":"2024-09-19T09:27:11.346891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:16.341340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338183+0000","last_change":"2024-09-19T09:25:29.301774+0000","last_active":"2024-09-19T09:27:11.338183+0000","last_peered":"2024-09-19T09:27:11.338183+0000","last_clean":"2024-09-19T09:27:11.338183+0000","last_became_active":"2024-09-19T09:25:29.301398+0000","last_became_peered":"2024-09-19T09:25:29.301398+0000","last_unstale":"2024-09-19T09:27:11.338183+0000","last_undegraded":"2024-09-19T09:27:11.338183+0000","last_fullsized":"2024-09-19T09:27:11.338183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:36.058038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346172+0000","last_change":"2024-09-19T09:25:29.301475+0000","last_active":"2024-09-19T09:27:11.346172+0000","last_peered":"2024-09-19T09:27:11.346172+0000","last_clean":"2024-09-19T09:27:11.346172+0000","last_became_active":"2024-09-19T09:25:29.300908+0000","last_became_peered":"2024-09-19T09:25:29.300908+0000","last_unstale":"2024-09-19T09:27:11.346172+0000","last_undegraded":"2024-09-19T09:27:11.346172+0000","last_fullsized":"2024-09-19T09:27:11.346172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:44.774127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336219+0000","last_change":"2024-09-19T09:25:42.994395+0000","last_active":"2024-09-19T09:27:11.336219+0000","last_peered":"2024-09-19T09:27:11.336219+0000","last_clean":"2024-09-19T09:27:11.336219+0000","last_became_active":"2024-09-19T09:25:42.993992+0000","last_became_peered":"2024-09-19T09:25:42.993992+0000","last_unstale":"2024-09-19T09:27:11.336219+0000","last_undegraded":"2024-09-19T09:27:11.336219+0000","last_fullsized":"2024-09-19T09:27:11.336219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.486069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.340454+0000","last_change":"2024-09-19T09:25:42.989979+0000","last_active":"2024-09-19T09:27:11.340454+0000","last_peered":"2024-09-19T09:27:11.340454+0000","last_clean":"2024-09-19T09:27:11.340454+0000","last_became_active":"2024-09-19T09:25:42.989578+0000","last_became_peered":"2024-09-19T09:25:42.989578+0000","last_unstale":"2024-09-19T09:27:11.340454+0000","last_undegraded":"2024-09-19T09:27:11.340454+0000","last_fullsized":"2024-09-19T09:27:11.340454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:04.583023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336926+0000","last_change":"2024-09-19T09:25:29.299917+0000","last_active":"2024-09-19T09:27:11.336926+0000","last_peered":"2024-09-19T09:27:11.336926+0000","last_clean":"2024-09-19T09:27:11.336926+0000","last_became_active":"2024-09-19T09:25:29.299571+0000","last_became_peered":"2024-09-19T09:25:29.299571+0000","last_unstale":"2024-09-19T09:27:11.336926+0000","last_undegraded":"2024-09-19T09:27:11.336926+0000","last_fullsized":"2024-09-19T09:27:11.336926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:25.142252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335108+0000","last_change":"2024-09-19T09:25:29.299713+0000","last_active":"2024-09-19T09:27:11.335108+0000","last_peered":"2024-09-19T09:27:11.335108+0000","last_clean":"2024-09-19T09:27:11.335108+0000","last_became_active":"2024-09-19T09:25:29.299234+0000","last_became_peered":"2024-09-19T09:25:29.299234+0000","last_unstale":"2024-09-19T09:27:11.335108+0000","last_undegraded":"2024-09-19T09:27:11.335108+0000","last_fullsized":"2024-09-19T09:27:11.335108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:34.714714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346606+0000","last_change":"2024-09-19T09:25:29.293010+0000","last_active":"2024-09-19T09:27:11.346606+0000","last_peered":"2024-09-19T09:27:11.346606+0000","last_clean":"2024-09-19T09:27:11.346606+0000","last_became_active":"2024-09-19T09:25:29.292645+0000","last_became_peered":"2024-09-19T09:25:29.292645+0000","last_unstale":"2024-09-19T09:27:11.346606+0000","last_undegraded":"2024-09-19T09:27:11.346606+0000","last_fullsized":"2024-09-19T09:27:11.346606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:24.749075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347900+0000","last_change":"2024-09-19T09:25:29.294530+0000","last_active":"2024-09-19T09:27:11.347900+0000","last_peered":"2024-09-19T09:27:11.347900+0000","last_clean":"2024-09-19T09:27:11.347900+0000","last_became_active":"2024-09-19T09:25:29.293699+0000","last_became_peered":"2024-09-19T09:25:29.293699+0000","last_unstale":"2024-09-19T09:27:11.347900+0000","last_undegraded":"2024-09-19T09:27:11.347900+0000","last_fullsized":"2024-09-19T09:27:11.347900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:53.491258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336452+0000","last_change":"2024-09-19T09:25:29.290600+0000","last_active":"2024-09-19T09:27:11.336452+0000","last_peered":"2024-09-19T09:27:11.336452+0000","last_clean":"2024-09-19T09:27:11.336452+0000","last_became_active":"2024-09-19T09:25:29.290305+0000","last_became_peered":"2024-09-19T09:25:29.290305+0000","last_unstale":"2024-09-19T09:27:11.336452+0000","last_undegraded":"2024-09-19T09:27:11.336452+0000","last_fullsized":"2024-09-19T09:27:11.336452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:42.809659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338092+0000","last_change":"2024-09-19T09:25:29.300591+0000","last_active":"2024-09-19T09:27:11.338092+0000","last_peered":"2024-09-19T09:27:11.338092+0000","last_clean":"2024-09-19T09:27:11.338092+0000","last_became_active":"2024-09-19T09:25:29.300251+0000","last_became_peered":"2024-09-19T09:25:29.300251+0000","last_unstale":"2024-09-19T09:27:11.338092+0000","last_undegraded":"2024-09-19T09:27:11.338092+0000","last_fullsized":"2024-09-19T09:27:11.338092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:08.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342064+0000","last_change":"2024-09-19T09:25:29.298452+0000","last_active":"2024-09-19T09:27:11.342064+0000","last_peered":"2024-09-19T09:27:11.342064+0000","last_clean":"2024-09-19T09:27:11.342064+0000","last_became_active":"2024-09-19T09:25:29.298137+0000","last_became_peered":"2024-09-19T09:25:29.298137+0000","last_unstale":"2024-09-19T09:27:11.342064+0000","last_undegraded":"2024-09-19T09:27:11.342064+0000","last_fullsized":"2024-09-19T09:27:11.342064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:11.716228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337968+0000","last_change":"2024-09-19T09:25:29.294278+0000","last_active":"2024-09-19T09:27:11.337968+0000","last_peered":"2024-09-19T09:27:11.337968+0000","last_clean":"2024-09-19T09:27:11.337968+0000","last_became_active":"2024-09-19T09:25:29.293972+0000","last_became_peered":"2024-09-19T09:25:29.293972+0000","last_unstale":"2024-09-19T09:27:11.337968+0000","last_undegraded":"2024-09-19T09:27:11.337968+0000","last_fullsized":"2024-09-19T09:27:11.337968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:27.923839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346634+0000","last_change":"2024-09-19T09:25:29.294000+0000","last_active":"2024-09-19T09:27:11.346634+0000","last_peered":"2024-09-19T09:27:11.346634+0000","last_clean":"2024-09-19T09:27:11.346634+0000","last_became_active":"2024-09-19T09:25:29.293486+0000","last_became_peered":"2024-09-19T09:25:29.293486+0000","last_unstale":"2024-09-19T09:27:11.346634+0000","last_undegraded":"2024-09-19T09:27:11.346634+0000","last_fullsized":"2024-09-19T09:27:11.346634+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:44.053620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334040+0000","last_change":"2024-09-19T09:25:42.994951+0000","last_active":"2024-09-19T09:27:11.334040+0000","last_peered":"2024-09-19T09:27:11.334040+0000","last_clean":"2024-09-19T09:27:11.334040+0000","last_became_active":"2024-09-19T09:25:42.994588+0000","last_became_peered":"2024-09-19T09:25:42.994588+0000","last_unstale":"2024-09-19T09:27:11.334040+0000","last_undegraded":"2024-09-19T09:27:11.334040+0000","last_fullsized":"2024-09-19T09:27:11.334040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:10.141169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334658+0000","last_change":"2024-09-19T09:25:29.286395+0000","last_active":"2024-09-19T09:27:11.334658+0000","last_peered":"2024-09-19T09:27:11.334658+0000","last_clean":"2024-09-19T09:27:11.334658+0000","last_became_active":"2024-09-19T09:25:29.286048+0000","last_became_peered":"2024-09-19T09:25:29.286048+0000","last_unstale":"2024-09-19T09:27:11.334658+0000","last_undegraded":"2024-09-19T09:27:11.334658+0000","last_fullsized":"2024-09-19T09:27:11.334658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:43.121166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.327390+0000","last_change":"2024-09-19T09:25:29.298458+0000","last_active":"2024-09-19T09:27:11.327390+0000","last_peered":"2024-09-19T09:27:11.327390+0000","last_clean":"2024-09-19T09:27:11.327390+0000","last_became_active":"2024-09-19T09:25:29.297993+0000","last_became_peered":"2024-09-19T09:25:29.297993+0000","last_unstale":"2024-09-19T09:27:11.327390+0000","last_undegraded":"2024-09-19T09:27:11.327390+0000","last_fullsized":"2024-09-19T09:27:11.327390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:44.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344574+0000","last_change":"2024-09-19T09:25:29.297167+0000","last_active":"2024-09-19T09:27:11.344574+0000","last_peered":"2024-09-19T09:27:11.344574+0000","last_clean":"2024-09-19T09:27:11.344574+0000","last_became_active":"2024-09-19T09:25:29.296777+0000","last_became_peered":"2024-09-19T09:25:29.296777+0000","last_unstale":"2024-09-19T09:27:11.344574+0000","last_undegraded":"2024-09-19T09:27:11.344574+0000","last_fullsized":"2024-09-19T09:27:11.344574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:40.581643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.339599+0000","last_change":"2024-09-19T09:25:29.296169+0000","last_active":"2024-09-19T09:27:11.339599+0000","last_peered":"2024-09-19T09:27:11.339599+0000","last_clean":"2024-09-19T09:27:11.339599+0000","last_became_active":"2024-09-19T09:25:29.295806+0000","last_became_peered":"2024-09-19T09:25:29.295806+0000","last_unstale":"2024-09-19T09:27:11.339599+0000","last_undegraded":"2024-09-19T09:27:11.339599+0000","last_fullsized":"2024-09-19T09:27:11.339599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:39.639884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334590+0000","last_change":"2024-09-19T09:25:29.297960+0000","last_active":"2024-09-19T09:27:11.334590+0000","last_peered":"2024-09-19T09:27:11.334590+0000","last_clean":"2024-09-19T09:27:11.334590+0000","last_became_active":"2024-09-19T09:25:29.297577+0000","last_became_peered":"2024-09-19T09:25:29.297577+0000","last_unstale":"2024-09-19T09:27:11.334590+0000","last_undegraded":"2024-09-19T09:27:11.334590+0000","last_fullsized":"2024-09-19T09:27:11.334590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:27.035504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335188+0000","last_change":"2024-09-19T09:25:29.297896+0000","last_active":"2024-09-19T09:27:11.335188+0000","last_peered":"2024-09-19T09:27:11.335188+0000","last_clean":"2024-09-19T09:27:11.335188+0000","last_became_active":"2024-09-19T09:25:29.297594+0000","last_became_peered":"2024-09-19T09:25:29.297594+0000","last_unstale":"2024-09-19T09:27:11.335188+0000","last_undegraded":"2024-09-19T09:27:11.335188+0000","last_fullsized":"2024-09-19T09:27:11.335188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.366325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334560+0000","last_change":"2024-09-19T09:25:29.301851+0000","last_active":"2024-09-19T09:27:11.334560+0000","last_peered":"2024-09-19T09:27:11.334560+0000","last_clean":"2024-09-19T09:27:11.334560+0000","last_became_active":"2024-09-19T09:25:29.299282+0000","last_became_peered":"2024-09-19T09:25:29.299282+0000","last_unstale":"2024-09-19T09:27:11.334560+0000","last_undegraded":"2024-09-19T09:27:11.334560+0000","last_fullsized":"2024-09-19T09:27:11.334560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:00.382114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342990+0000","last_change":"2024-09-19T09:25:29.303316+0000","last_active":"2024-09-19T09:27:11.342990+0000","last_peered":"2024-09-19T09:27:11.342990+0000","last_clean":"2024-09-19T09:27:11.342990+0000","last_became_active":"2024-09-19T09:25:29.302994+0000","last_became_peered":"2024-09-19T09:25:29.302994+0000","last_unstale":"2024-09-19T09:27:11.342990+0000","last_undegraded":"2024-09-19T09:27:11.342990+0000","last_fullsized":"2024-09-19T09:27:11.342990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:46.233991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341767+0000","last_change":"2024-09-19T09:25:29.293004+0000","last_active":"2024-09-19T09:27:11.341767+0000","last_peered":"2024-09-19T09:27:11.341767+0000","last_clean":"2024-09-19T09:27:11.341767+0000","last_became_active":"2024-09-19T09:25:29.292657+0000","last_became_peered":"2024-09-19T09:25:29.292657+0000","last_unstale":"2024-09-19T09:27:11.341767+0000","last_undegraded":"2024-09-19T09:27:11.341767+0000","last_fullsized":"2024-09-19T09:27:11.341767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:50.966477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.332473+0000","last_change":"2024-09-19T09:25:29.300434+0000","last_active":"2024-09-19T09:27:11.332473+0000","last_peered":"2024-09-19T09:27:11.332473+0000","last_clean":"2024-09-19T09:27:11.332473+0000","last_became_active":"2024-09-19T09:25:29.300053+0000","last_became_peered":"2024-09-19T09:25:29.300053+0000","last_unstale":"2024-09-19T09:27:11.332473+0000","last_undegraded":"2024-09-19T09:27:11.332473+0000","last_fullsized":"2024-09-19T09:27:11.332473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:13.283298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":132,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.106724+0000","last_change":"2024-09-19T09:25:27.280792+0000","last_active":"2024-09-19T09:27:16.106724+0000","last_peered":"2024-09-19T09:27:16.106724+0000","last_clean":"2024-09-19T09:27:16.106724+0000","last_became_active":"2024-09-19T09:25:27.280443+0000","last_became_peered":"2024-09-19T09:25:27.280443+0000","last_unstale":"2024-09-19T09:27:16.106724+0000","last_undegraded":"2024-09-19T09:27:16.106724+0000","last_fullsized":"2024-09-19T09:27:16.106724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.599512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334165+0000","last_change":"2024-09-19T09:25:29.298975+0000","last_active":"2024-09-19T09:27:11.334165+0000","last_peered":"2024-09-19T09:27:11.334165+0000","last_clean":"2024-09-19T09:27:11.334165+0000","last_became_active":"2024-09-19T09:25:29.298476+0000","last_became_peered":"2024-09-19T09:25:29.298476+0000","last_unstale":"2024-09-19T09:27:11.334165+0000","last_undegraded":"2024-09-19T09:27:11.334165+0000","last_fullsized":"2024-09-19T09:27:11.334165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:09.658894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986623+0000","last_change":"2024-09-19T09:25:27.278257+0000","last_active":"2024-09-19T09:25:42.986623+0000","last_peered":"2024-09-19T09:25:42.986623+0000","last_clean":"2024-09-19T09:25:42.986623+0000","last_became_active":"2024-09-19T09:25:27.277897+0000","last_became_peered":"2024-09-19T09:25:27.277897+0000","last_unstale":"2024-09-19T09:25:42.986623+0000","last_undegraded":"2024-09-19T09:25:42.986623+0000","last_fullsized":"2024-09-19T09:25:42.986623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:47.550314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985617+0000","last_change":"2024-09-19T09:25:27.272215+0000","last_active":"2024-09-19T09:25:42.985617+0000","last_peered":"2024-09-19T09:25:42.985617+0000","last_clean":"2024-09-19T09:25:42.985617+0000","last_became_active":"2024-09-19T09:25:27.271821+0000","last_became_peered":"2024-09-19T09:25:27.271821+0000","last_unstale":"2024-09-19T09:25:42.985617+0000","last_undegraded":"2024-09-19T09:25:42.985617+0000","last_fullsized":"2024-09-19T09:25:42.985617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:30.615730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.329744+0000","last_change":"2024-09-19T09:25:29.289347+0000","last_active":"2024-09-19T09:27:11.329744+0000","last_peered":"2024-09-19T09:27:11.329744+0000","last_clean":"2024-09-19T09:27:11.329744+0000","last_became_active":"2024-09-19T09:25:29.289015+0000","last_became_peered":"2024-09-19T09:25:29.289015+0000","last_unstale":"2024-09-19T09:27:11.329744+0000","last_undegraded":"2024-09-19T09:27:11.329744+0000","last_fullsized":"2024-09-19T09:27:11.329744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:18.419334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995432+0000","last_change":"2024-09-19T09:25:27.277342+0000","last_active":"2024-09-19T09:25:42.995432+0000","last_peered":"2024-09-19T09:25:42.995432+0000","last_clean":"2024-09-19T09:25:42.995432+0000","last_became_active":"2024-09-19T09:25:27.277016+0000","last_became_peered":"2024-09-19T09:25:27.277016+0000","last_unstale":"2024-09-19T09:25:42.995432+0000","last_undegraded":"2024-09-19T09:25:42.995432+0000","last_fullsized":"2024-09-19T09:25:42.995432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:56.736156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336629+0000","last_change":"2024-09-19T09:25:29.290924+0000","last_active":"2024-09-19T09:27:11.336629+0000","last_peered":"2024-09-19T09:27:11.336629+0000","last_clean":"2024-09-19T09:27:11.336629+0000","last_became_active":"2024-09-19T09:25:29.290482+0000","last_became_peered":"2024-09-19T09:25:29.290482+0000","last_unstale":"2024-09-19T09:27:11.336629+0000","last_undegraded":"2024-09-19T09:27:11.336629+0000","last_fullsized":"2024-09-19T09:27:11.336629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:16.265040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'6","reported_seq":222,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.105891+0000","last_change":"2024-09-19T09:25:27.277604+0000","last_active":"2024-09-19T09:27:16.105891+0000","last_peered":"2024-09-19T09:27:16.105891+0000","last_clean":"2024-09-19T09:27:16.105891+0000","last_became_active":"2024-09-19T09:25:27.277271+0000","last_became_peered":"2024-09-19T09:25:27.277271+0000","last_unstale":"2024-09-19T09:27:16.105891+0000","last_undegraded":"2024-09-19T09:27:16.105891+0000","last_fullsized":"2024-09-19T09:27:16.105891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:32.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347371+0000","last_change":"2024-09-19T09:25:29.290537+0000","last_active":"2024-09-19T09:27:11.347371+0000","last_peered":"2024-09-19T09:27:11.347371+0000","last_clean":"2024-09-19T09:27:11.347371+0000","last_became_active":"2024-09-19T09:25:29.287984+0000","last_became_peered":"2024-09-19T09:25:29.287984+0000","last_unstale":"2024-09-19T09:27:11.347371+0000","last_undegraded":"2024-09-19T09:27:11.347371+0000","last_fullsized":"2024-09-19T09:27:11.347371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:26.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990018+0000","last_change":"2024-09-19T09:25:27.278560+0000","last_active":"2024-09-19T09:25:42.990018+0000","last_peered":"2024-09-19T09:25:42.990018+0000","last_clean":"2024-09-19T09:25:42.990018+0000","last_became_active":"2024-09-19T09:25:27.277950+0000","last_became_peered":"2024-09-19T09:25:27.277950+0000","last_unstale":"2024-09-19T09:25:42.990018+0000","last_undegraded":"2024-09-19T09:25:42.990018+0000","last_fullsized":"2024-09-19T09:25:42.990018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:28.387725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.339317+0000","last_change":"2024-09-19T09:25:29.298897+0000","last_active":"2024-09-19T09:27:11.339317+0000","last_peered":"2024-09-19T09:27:11.339317+0000","last_clean":"2024-09-19T09:27:11.339317+0000","last_became_active":"2024-09-19T09:25:29.298617+0000","last_became_peered":"2024-09-19T09:25:29.298617+0000","last_unstale":"2024-09-19T09:27:11.339317+0000","last_undegraded":"2024-09-19T09:27:11.339317+0000","last_fullsized":"2024-09-19T09:27:11.339317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:53.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994870+0000","last_change":"2024-09-19T09:25:27.324871+0000","last_active":"2024-09-19T09:25:42.994870+0000","last_peered":"2024-09-19T09:25:42.994870+0000","last_clean":"2024-09-19T09:25:42.994870+0000","last_became_active":"2024-09-19T09:25:27.324471+0000","last_became_peered":"2024-09-19T09:25:27.324471+0000","last_unstale":"2024-09-19T09:25:42.994870+0000","last_undegraded":"2024-09-19T09:25:42.994870+0000","last_fullsized":"2024-09-19T09:25:42.994870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:35.936413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347260+0000","last_change":"2024-09-19T09:25:29.298332+0000","last_active":"2024-09-19T09:27:11.347260+0000","last_peered":"2024-09-19T09:27:11.347260+0000","last_clean":"2024-09-19T09:27:11.347260+0000","last_became_active":"2024-09-19T09:25:29.298006+0000","last_became_peered":"2024-09-19T09:25:29.298006+0000","last_unstale":"2024-09-19T09:27:11.347260+0000","last_undegraded":"2024-09-19T09:27:11.347260+0000","last_fullsized":"2024-09-19T09:27:11.347260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:02.621821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'33","reported_seq":261,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.107843+0000","last_change":"2024-09-19T09:25:27.279528+0000","last_active":"2024-09-19T09:27:16.107843+0000","last_peered":"2024-09-19T09:27:16.107843+0000","last_clean":"2024-09-19T09:27:16.107843+0000","last_became_active":"2024-09-19T09:25:27.279149+0000","last_became_peered":"2024-09-19T09:25:27.279149+0000","last_unstale":"2024-09-19T09:27:16.107843+0000","last_undegraded":"2024-09-19T09:27:16.107843+0000","last_fullsized":"2024-09-19T09:27:16.107843+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:47.030736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60013,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":206,"num_read_kb":161,"num_write":35,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.324455+0000","last_change":"2024-09-19T09:25:29.297538+0000","last_active":"2024-09-19T09:27:11.324455+0000","last_peered":"2024-09-19T09:27:11.324455+0000","last_clean":"2024-09-19T09:27:11.324455+0000","last_became_active":"2024-09-19T09:25:29.297181+0000","last_became_peered":"2024-09-19T09:25:29.297181+0000","last_unstale":"2024-09-19T09:27:11.324455+0000","last_undegraded":"2024-09-19T09:27:11.324455+0000","last_fullsized":"2024-09-19T09:27:11.324455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:53.135438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992860+0000","last_change":"2024-09-19T09:25:27.277090+0000","last_active":"2024-09-19T09:25:42.992860+0000","last_peered":"2024-09-19T09:25:42.992860+0000","last_clean":"2024-09-19T09:25:42.992860+0000","last_became_active":"2024-09-19T09:25:27.276718+0000","last_became_peered":"2024-09-19T09:25:27.276718+0000","last_unstale":"2024-09-19T09:25:42.992860+0000","last_undegraded":"2024-09-19T09:25:42.992860+0000","last_fullsized":"2024-09-19T09:25:42.992860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:25.664261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.325840+0000","last_change":"2024-09-19T09:25:29.294393+0000","last_active":"2024-09-19T09:27:11.325840+0000","last_peered":"2024-09-19T09:27:11.325840+0000","last_clean":"2024-09-19T09:27:11.325840+0000","last_became_active":"2024-09-19T09:25:29.294099+0000","last_became_peered":"2024-09-19T09:25:29.294099+0000","last_unstale":"2024-09-19T09:27:11.325840+0000","last_undegraded":"2024-09-19T09:27:11.325840+0000","last_fullsized":"2024-09-19T09:27:11.325840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:17.333736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995398+0000","last_change":"2024-09-19T09:25:27.278593+0000","last_active":"2024-09-19T09:25:42.995398+0000","last_peered":"2024-09-19T09:25:42.995398+0000","last_clean":"2024-09-19T09:25:42.995398+0000","last_became_active":"2024-09-19T09:25:27.278232+0000","last_became_peered":"2024-09-19T09:25:27.278232+0000","last_unstale":"2024-09-19T09:25:42.995398+0000","last_undegraded":"2024-09-19T09:25:42.995398+0000","last_fullsized":"2024-09-19T09:25:42.995398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:05.852671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343581+0000","last_change":"2024-09-19T09:25:29.300024+0000","last_active":"2024-09-19T09:27:11.343581+0000","last_peered":"2024-09-19T09:27:11.343581+0000","last_clean":"2024-09-19T09:27:11.343581+0000","last_became_active":"2024-09-19T09:25:29.299683+0000","last_became_peered":"2024-09-19T09:25:29.299683+0000","last_unstale":"2024-09-19T09:27:11.343581+0000","last_undegraded":"2024-09-19T09:27:11.343581+0000","last_fullsized":"2024-09-19T09:27:11.343581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:42.249399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'7","reported_seq":131,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:15.107233+0000","last_change":"2024-09-19T09:25:27.272739+0000","last_active":"2024-09-19T09:27:15.107233+0000","last_peered":"2024-09-19T09:27:15.107233+0000","last_clean":"2024-09-19T09:27:15.107233+0000","last_became_active":"2024-09-19T09:25:27.272354+0000","last_became_peered":"2024-09-19T09:25:27.272354+0000","last_unstale":"2024-09-19T09:27:15.107233+0000","last_undegraded":"2024-09-19T09:27:15.107233+0000","last_fullsized":"2024-09-19T09:27:15.107233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:27.863840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343011+0000","last_change":"2024-09-19T09:25:29.303460+0000","last_active":"2024-09-19T09:27:11.343011+0000","last_peered":"2024-09-19T09:27:11.343011+0000","last_clean":"2024-09-19T09:27:11.343011+0000","last_became_active":"2024-09-19T09:25:29.303127+0000","last_became_peered":"2024-09-19T09:25:29.303127+0000","last_unstale":"2024-09-19T09:27:11.343011+0000","last_undegraded":"2024-09-19T09:27:11.343011+0000","last_fullsized":"2024-09-19T09:27:11.343011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:35.313924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990876+0000","last_change":"2024-09-19T09:25:27.323238+0000","last_active":"2024-09-19T09:25:42.990876+0000","last_peered":"2024-09-19T09:25:42.990876+0000","last_clean":"2024-09-19T09:25:42.990876+0000","last_became_active":"2024-09-19T09:25:27.322868+0000","last_became_peered":"2024-09-19T09:25:27.322868+0000","last_unstale":"2024-09-19T09:25:42.990876+0000","last_undegraded":"2024-09-19T09:25:42.990876+0000","last_fullsized":"2024-09-19T09:25:42.990876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:22.890280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338463+0000","last_change":"2024-09-19T09:25:43.005569+0000","last_active":"2024-09-19T09:27:11.338463+0000","last_peered":"2024-09-19T09:27:11.338463+0000","last_clean":"2024-09-19T09:27:11.338463+0000","last_became_active":"2024-09-19T09:25:43.005143+0000","last_became_peered":"2024-09-19T09:25:43.005143+0000","last_unstale":"2024-09-19T09:27:11.338463+0000","last_undegraded":"2024-09-19T09:27:11.338463+0000","last_fullsized":"2024-09-19T09:27:11.338463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:26.297013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991438+0000","last_change":"2024-09-19T09:25:27.282737+0000","last_active":"2024-09-19T09:25:42.991438+0000","last_peered":"2024-09-19T09:25:42.991438+0000","last_clean":"2024-09-19T09:25:42.991438+0000","last_became_active":"2024-09-19T09:25:27.281785+0000","last_became_peered":"2024-09-19T09:25:27.281785+0000","last_unstale":"2024-09-19T09:25:42.991438+0000","last_undegraded":"2024-09-19T09:25:42.991438+0000","last_fullsized":"2024-09-19T09:25:42.991438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:41.552206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.340095+0000","last_change":"2024-09-19T09:25:29.300168+0000","last_active":"2024-09-19T09:27:11.340095+0000","last_peered":"2024-09-19T09:27:11.340095+0000","last_clean":"2024-09-19T09:27:11.340095+0000","last_became_active":"2024-09-19T09:25:29.299811+0000","last_became_peered":"2024-09-19T09:25:29.299811+0000","last_unstale":"2024-09-19T09:27:11.340095+0000","last_undegraded":"2024-09-19T09:27:11.340095+0000","last_fullsized":"2024-09-19T09:27:11.340095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:06.691259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985783+0000","last_change":"2024-09-19T09:25:27.278433+0000","last_active":"2024-09-19T09:25:42.985783+0000","last_peered":"2024-09-19T09:25:42.985783+0000","last_clean":"2024-09-19T09:25:42.985783+0000","last_became_active":"2024-09-19T09:25:27.278076+0000","last_became_peered":"2024-09-19T09:25:27.278076+0000","last_unstale":"2024-09-19T09:25:42.985783+0000","last_undegraded":"2024-09-19T09:25:42.985783+0000","last_fullsized":"2024-09-19T09:25:42.985783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:53.799265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338515+0000","last_change":"2024-09-19T09:25:29.295650+0000","last_active":"2024-09-19T09:27:11.338515+0000","last_peered":"2024-09-19T09:27:11.338515+0000","last_clean":"2024-09-19T09:27:11.338515+0000","last_became_active":"2024-09-19T09:25:29.295277+0000","last_became_peered":"2024-09-19T09:25:29.295277+0000","last_unstale":"2024-09-19T09:27:11.338515+0000","last_undegraded":"2024-09-19T09:27:11.338515+0000","last_fullsized":"2024-09-19T09:27:11.338515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:57.690699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996123+0000","last_change":"2024-09-19T09:25:27.279239+0000","last_active":"2024-09-19T09:25:42.996123+0000","last_peered":"2024-09-19T09:25:42.996123+0000","last_clean":"2024-09-19T09:25:42.996123+0000","last_became_active":"2024-09-19T09:25:27.278868+0000","last_became_peered":"2024-09-19T09:25:27.278868+0000","last_unstale":"2024-09-19T09:25:42.996123+0000","last_undegraded":"2024-09-19T09:25:42.996123+0000","last_fullsized":"2024-09-19T09:25:42.996123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:41.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341911+0000","last_change":"2024-09-19T09:25:29.300833+0000","last_active":"2024-09-19T09:27:11.341911+0000","last_peered":"2024-09-19T09:27:11.341911+0000","last_clean":"2024-09-19T09:27:11.341911+0000","last_became_active":"2024-09-19T09:25:29.300472+0000","last_became_peered":"2024-09-19T09:25:29.300472+0000","last_unstale":"2024-09-19T09:27:11.341911+0000","last_undegraded":"2024-09-19T09:27:11.341911+0000","last_fullsized":"2024-09-19T09:27:11.341911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:50.591276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'7","reported_seq":117,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:15.140661+0000","last_change":"2024-09-19T09:25:43.001722+0000","last_active":"2024-09-19T09:27:15.140661+0000","last_peered":"2024-09-19T09:27:15.140661+0000","last_clean":"2024-09-19T09:27:15.140661+0000","last_became_active":"2024-09-19T09:25:42.992796+0000","last_became_peered":"2024-09-19T09:25:42.992796+0000","last_unstale":"2024-09-19T09:27:15.140661+0000","last_undegraded":"2024-09-19T09:27:15.140661+0000","last_fullsized":"2024-09-19T09:27:15.140661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:59.639010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346495+0000","last_change":"2024-09-19T09:25:29.295906+0000","last_active":"2024-09-19T09:27:11.346495+0000","last_peered":"2024-09-19T09:27:11.346495+0000","last_clean":"2024-09-19T09:27:11.346495+0000","last_became_active":"2024-09-19T09:25:29.295565+0000","last_became_peered":"2024-09-19T09:25:29.295565+0000","last_unstale":"2024-09-19T09:27:11.346495+0000","last_undegraded":"2024-09-19T09:27:11.346495+0000","last_fullsized":"2024-09-19T09:27:11.346495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:24.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989907+0000","last_change":"2024-09-19T09:24:19.256300+0000","last_active":"2024-09-19T09:25:42.989907+0000","last_peered":"2024-09-19T09:25:42.989907+0000","last_clean":"2024-09-19T09:25:42.989907+0000","last_became_active":"2024-09-19T09:24:19.255833+0000","last_became_peered":"2024-09-19T09:24:19.255833+0000","last_unstale":"2024-09-19T09:25:42.989907+0000","last_undegraded":"2024-09-19T09:25:42.989907+0000","last_fullsized":"2024-09-19T09:25:42.989907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_clean_scrub_stamp":"2024-09-19T09:20:57.798838+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:14:33.343748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989637+0000","last_change":"2024-09-19T09:25:27.272090+0000","last_active":"2024-09-19T09:25:42.989637+0000","last_peered":"2024-09-19T09:25:42.989637+0000","last_clean":"2024-09-19T09:25:42.989637+0000","last_became_active":"2024-09-19T09:25:27.271651+0000","last_became_peered":"2024-09-19T09:25:27.271651+0000","last_unstale":"2024-09-19T09:25:42.989637+0000","last_undegraded":"2024-09-19T09:25:42.989637+0000","last_fullsized":"2024-09-19T09:25:42.989637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:45.847082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344375+0000","last_change":"2024-09-19T09:25:29.300718+0000","last_active":"2024-09-19T09:27:11.344375+0000","last_peered":"2024-09-19T09:27:11.344375+0000","last_clean":"2024-09-19T09:27:11.344375+0000","last_became_active":"2024-09-19T09:25:29.299460+0000","last_became_peered":"2024-09-19T09:25:29.299460+0000","last_unstale":"2024-09-19T09:27:11.344375+0000","last_undegraded":"2024-09-19T09:27:11.344375+0000","last_fullsized":"2024-09-19T09:27:11.344375+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:43.063909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003299+0000","last_change":"2024-09-19T09:25:43.003299+0000","last_active":"2024-09-19T09:25:43.003299+0000","last_peered":"2024-09-19T09:25:43.003299+0000","last_clean":"2024-09-19T09:25:43.003299+0000","last_became_active":"2024-09-19T09:25:42.999686+0000","last_became_peered":"2024-09-19T09:25:42.999686+0000","last_unstale":"2024-09-19T09:25:43.003299+0000","last_undegraded":"2024-09-19T09:25:43.003299+0000","last_fullsized":"2024-09-19T09:25:43.003299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:05.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337734+0000","last_change":"2024-09-19T09:25:29.297833+0000","last_active":"2024-09-19T09:27:11.337734+0000","last_peered":"2024-09-19T09:27:11.337734+0000","last_clean":"2024-09-19T09:27:11.337734+0000","last_became_active":"2024-09-19T09:25:29.297534+0000","last_became_peered":"2024-09-19T09:25:29.297534+0000","last_unstale":"2024-09-19T09:27:11.337734+0000","last_undegraded":"2024-09-19T09:27:11.337734+0000","last_fullsized":"2024-09-19T09:27:11.337734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:47.156253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":129,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:17.109301+0000","last_change":"2024-09-19T09:25:27.322759+0000","last_active":"2024-09-19T09:27:17.109301+0000","last_peered":"2024-09-19T09:27:17.109301+0000","last_clean":"2024-09-19T09:27:17.109301+0000","last_became_active":"2024-09-19T09:25:27.322402+0000","last_became_peered":"2024-09-19T09:25:27.322402+0000","last_unstale":"2024-09-19T09:27:17.109301+0000","last_undegraded":"2024-09-19T09:27:17.109301+0000","last_fullsized":"2024-09-19T09:27:17.109301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:06.276957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335476+0000","last_change":"2024-09-19T09:25:42.999834+0000","last_active":"2024-09-19T09:27:11.335476+0000","last_peered":"2024-09-19T09:27:11.335476+0000","last_clean":"2024-09-19T09:27:11.335476+0000","last_became_active":"2024-09-19T09:25:42.999450+0000","last_became_peered":"2024-09-19T09:25:42.999450+0000","last_unstale":"2024-09-19T09:27:11.335476+0000","last_undegraded":"2024-09-19T09:27:11.335476+0000","last_fullsized":"2024-09-19T09:27:11.335476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:00.509844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'7","reported_seq":129,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.117110+0000","last_change":"2024-09-19T09:25:27.275212+0000","last_active":"2024-09-19T09:27:16.117110+0000","last_peered":"2024-09-19T09:27:16.117110+0000","last_clean":"2024-09-19T09:27:16.117110+0000","last_became_active":"2024-09-19T09:25:27.274802+0000","last_became_peered":"2024-09-19T09:25:27.274802+0000","last_unstale":"2024-09-19T09:27:16.117110+0000","last_undegraded":"2024-09-19T09:27:16.117110+0000","last_fullsized":"2024-09-19T09:27:16.117110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:48.392026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.333364+0000","last_change":"2024-09-19T09:25:29.304747+0000","last_active":"2024-09-19T09:27:11.333364+0000","last_peered":"2024-09-19T09:27:11.333364+0000","last_clean":"2024-09-19T09:27:11.333364+0000","last_became_active":"2024-09-19T09:25:29.304386+0000","last_became_peered":"2024-09-19T09:25:29.304386+0000","last_unstale":"2024-09-19T09:27:11.333364+0000","last_undegraded":"2024-09-19T09:27:11.333364+0000","last_fullsized":"2024-09-19T09:27:11.333364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.788697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988854+0000","last_change":"2024-09-19T09:25:27.323083+0000","last_active":"2024-09-19T09:25:42.988854+0000","last_peered":"2024-09-19T09:25:42.988854+0000","last_clean":"2024-09-19T09:25:42.988854+0000","last_became_active":"2024-09-19T09:25:27.322707+0000","last_became_peered":"2024-09-19T09:25:27.322707+0000","last_unstale":"2024-09-19T09:25:42.988854+0000","last_undegraded":"2024-09-19T09:25:42.988854+0000","last_fullsized":"2024-09-19T09:25:42.988854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:24.415116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334132+0000","last_change":"2024-09-19T09:25:29.292472+0000","last_active":"2024-09-19T09:27:11.334132+0000","last_peered":"2024-09-19T09:27:11.334132+0000","last_clean":"2024-09-19T09:27:11.334132+0000","last_became_active":"2024-09-19T09:25:29.292079+0000","last_became_peered":"2024-09-19T09:25:29.292079+0000","last_unstale":"2024-09-19T09:27:11.334132+0000","last_undegraded":"2024-09-19T09:27:11.334132+0000","last_fullsized":"2024-09-19T09:27:11.334132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:35.801594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991224+0000","last_change":"2024-09-19T09:25:27.324839+0000","last_active":"2024-09-19T09:25:42.991224+0000","last_peered":"2024-09-19T09:25:42.991224+0000","last_clean":"2024-09-19T09:25:42.991224+0000","last_became_active":"2024-09-19T09:25:27.324475+0000","last_became_peered":"2024-09-19T09:25:27.324475+0000","last_unstale":"2024-09-19T09:25:42.991224+0000","last_undegraded":"2024-09-19T09:25:42.991224+0000","last_fullsized":"2024-09-19T09:25:42.991224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:32.251753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338007+0000","last_change":"2024-09-19T09:25:29.293025+0000","last_active":"2024-09-19T09:27:11.338007+0000","last_peered":"2024-09-19T09:27:11.338007+0000","last_clean":"2024-09-19T09:27:11.338007+0000","last_became_active":"2024-09-19T09:25:29.292570+0000","last_became_peered":"2024-09-19T09:25:29.292570+0000","last_unstale":"2024-09-19T09:27:11.338007+0000","last_undegraded":"2024-09-19T09:27:11.338007+0000","last_fullsized":"2024-09-19T09:27:11.338007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:16.394524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:55.295924+0000","last_change":"2024-09-19T09:25:27.278100+0000","last_active":"2024-09-19T09:25:55.295924+0000","last_peered":"2024-09-19T09:25:55.295924+0000","last_clean":"2024-09-19T09:25:55.295924+0000","last_became_active":"2024-09-19T09:25:27.277769+0000","last_became_peered":"2024-09-19T09:25:27.277769+0000","last_unstale":"2024-09-19T09:25:55.295924+0000","last_undegraded":"2024-09-19T09:25:55.295924+0000","last_fullsized":"2024-09-19T09:25:55.295924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:31.885142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338230+0000","last_change":"2024-09-19T09:25:29.293651+0000","last_active":"2024-09-19T09:27:11.338230+0000","last_peered":"2024-09-19T09:27:11.338230+0000","last_clean":"2024-09-19T09:27:11.338230+0000","last_became_active":"2024-09-19T09:25:29.293317+0000","last_became_peered":"2024-09-19T09:25:29.293317+0000","last_unstale":"2024-09-19T09:27:11.338230+0000","last_undegraded":"2024-09-19T09:27:11.338230+0000","last_fullsized":"2024-09-19T09:27:11.338230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:40.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990030+0000","last_change":"2024-09-19T09:25:27.269367+0000","last_active":"2024-09-19T09:25:42.990030+0000","last_peered":"2024-09-19T09:25:42.990030+0000","last_clean":"2024-09-19T09:25:42.990030+0000","last_became_active":"2024-09-19T09:25:27.269011+0000","last_became_peered":"2024-09-19T09:25:27.269011+0000","last_unstale":"2024-09-19T09:25:42.990030+0000","last_undegraded":"2024-09-19T09:25:42.990030+0000","last_fullsized":"2024-09-19T09:25:42.990030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:00.930752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334968+0000","last_change":"2024-09-19T09:25:29.299134+0000","last_active":"2024-09-19T09:27:11.334968+0000","last_peered":"2024-09-19T09:27:11.334968+0000","last_clean":"2024-09-19T09:27:11.334968+0000","last_became_active":"2024-09-19T09:25:29.298648+0000","last_became_peered":"2024-09-19T09:25:29.298648+0000","last_unstale":"2024-09-19T09:27:11.334968+0000","last_undegraded":"2024-09-19T09:27:11.334968+0000","last_fullsized":"2024-09-19T09:27:11.334968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:39.627852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003686+0000","last_change":"2024-09-19T09:25:27.322861+0000","last_active":"2024-09-19T09:25:43.003686+0000","last_peered":"2024-09-19T09:25:43.003686+0000","last_clean":"2024-09-19T09:25:43.003686+0000","last_became_active":"2024-09-19T09:25:27.322560+0000","last_became_peered":"2024-09-19T09:25:27.322560+0000","last_unstale":"2024-09-19T09:25:43.003686+0000","last_undegraded":"2024-09-19T09:25:43.003686+0000","last_fullsized":"2024-09-19T09:25:43.003686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:35.993093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.301522+0000","last_change":"2024-09-19T09:25:29.301680+0000","last_active":"2024-09-19T09:27:11.301522+0000","last_peered":"2024-09-19T09:27:11.301522+0000","last_clean":"2024-09-19T09:27:11.301522+0000","last_became_active":"2024-09-19T09:25:29.301331+0000","last_became_peered":"2024-09-19T09:25:29.301331+0000","last_unstale":"2024-09-19T09:27:11.301522+0000","last_undegraded":"2024-09-19T09:27:11.301522+0000","last_fullsized":"2024-09-19T09:27:11.301522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:57.364245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:27.272609+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:27.272169+0000","last_became_peered":"2024-09-19T09:25:27.272169+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.879474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341200+0000","last_change":"2024-09-19T09:25:29.296847+0000","last_active":"2024-09-19T09:27:11.341200+0000","last_peered":"2024-09-19T09:27:11.341200+0000","last_clean":"2024-09-19T09:27:11.341200+0000","last_became_active":"2024-09-19T09:25:29.296465+0000","last_became_peered":"2024-09-19T09:25:29.296465+0000","last_unstale":"2024-09-19T09:27:11.341200+0000","last_undegraded":"2024-09-19T09:27:11.341200+0000","last_fullsized":"2024-09-19T09:27:11.341200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:05:48.553252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.012954+0000","last_change":"2024-09-19T09:25:43.012532+0000","last_active":"2024-09-19T09:25:43.012954+0000","last_peered":"2024-09-19T09:25:43.012954+0000","last_clean":"2024-09-19T09:25:43.012954+0000","last_became_active":"2024-09-19T09:25:42.994996+0000","last_became_peered":"2024-09-19T09:25:42.994996+0000","last_unstale":"2024-09-19T09:25:43.012954+0000","last_undegraded":"2024-09-19T09:25:43.012954+0000","last_fullsized":"2024-09-19T09:25:43.012954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:25.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337187+0000","last_change":"2024-09-19T09:25:29.297633+0000","last_active":"2024-09-19T09:27:11.337187+0000","last_peered":"2024-09-19T09:27:11.337187+0000","last_clean":"2024-09-19T09:27:11.337187+0000","last_became_active":"2024-09-19T09:25:29.297284+0000","last_became_peered":"2024-09-19T09:25:29.297284+0000","last_unstale":"2024-09-19T09:27:11.337187+0000","last_undegraded":"2024-09-19T09:27:11.337187+0000","last_fullsized":"2024-09-19T09:27:11.337187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:50.582546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991172+0000","last_change":"2024-09-19T09:25:27.281778+0000","last_active":"2024-09-19T09:25:42.991172+0000","last_peered":"2024-09-19T09:25:42.991172+0000","last_clean":"2024-09-19T09:25:42.991172+0000","last_became_active":"2024-09-19T09:25:27.281280+0000","last_became_peered":"2024-09-19T09:25:27.281280+0000","last_unstale":"2024-09-19T09:25:42.991172+0000","last_undegraded":"2024-09-19T09:25:42.991172+0000","last_fullsized":"2024-09-19T09:25:42.991172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:46.811968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343342+0000","last_change":"2024-09-19T09:25:29.295663+0000","last_active":"2024-09-19T09:27:11.343342+0000","last_peered":"2024-09-19T09:27:11.343342+0000","last_clean":"2024-09-19T09:27:11.343342+0000","last_became_active":"2024-09-19T09:25:29.295146+0000","last_became_peered":"2024-09-19T09:25:29.295146+0000","last_unstale":"2024-09-19T09:27:11.343342+0000","last_undegraded":"2024-09-19T09:27:11.343342+0000","last_fullsized":"2024-09-19T09:27:11.343342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:36.749339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.345435+0000","last_change":"2024-09-19T09:25:29.298004+0000","last_active":"2024-09-19T09:27:11.345435+0000","last_peered":"2024-09-19T09:27:11.345435+0000","last_clean":"2024-09-19T09:27:11.345435+0000","last_became_active":"2024-09-19T09:25:29.297600+0000","last_became_peered":"2024-09-19T09:25:29.297600+0000","last_unstale":"2024-09-19T09:27:11.345435+0000","last_undegraded":"2024-09-19T09:27:11.345435+0000","last_fullsized":"2024-09-19T09:27:11.345435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:09.738615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992323+0000","last_change":"2024-09-19T09:25:27.282876+0000","last_active":"2024-09-19T09:25:42.992323+0000","last_peered":"2024-09-19T09:25:42.992323+0000","last_clean":"2024-09-19T09:25:42.992323+0000","last_became_active":"2024-09-19T09:25:27.282201+0000","last_became_peered":"2024-09-19T09:25:27.282201+0000","last_unstale":"2024-09-19T09:25:42.992323+0000","last_undegraded":"2024-09-19T09:25:42.992323+0000","last_fullsized":"2024-09-19T09:25:42.992323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:08.508662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347692+0000","last_change":"2024-09-19T09:25:29.288379+0000","last_active":"2024-09-19T09:27:11.347692+0000","last_peered":"2024-09-19T09:27:11.347692+0000","last_clean":"2024-09-19T09:27:11.347692+0000","last_became_active":"2024-09-19T09:25:29.287917+0000","last_became_peered":"2024-09-19T09:25:29.287917+0000","last_unstale":"2024-09-19T09:27:11.347692+0000","last_undegraded":"2024-09-19T09:27:11.347692+0000","last_fullsized":"2024-09-19T09:27:11.347692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:03.628233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991076+0000","last_change":"2024-09-19T09:25:27.279551+0000","last_active":"2024-09-19T09:25:42.991076+0000","last_peered":"2024-09-19T09:25:42.991076+0000","last_clean":"2024-09-19T09:25:42.991076+0000","last_became_active":"2024-09-19T09:25:27.278956+0000","last_became_peered":"2024-09-19T09:25:27.278956+0000","last_unstale":"2024-09-19T09:25:42.991076+0000","last_undegraded":"2024-09-19T09:25:42.991076+0000","last_fullsized":"2024-09-19T09:25:42.991076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:17.186453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347269+0000","last_change":"2024-09-19T09:25:29.288657+0000","last_active":"2024-09-19T09:27:11.347269+0000","last_peered":"2024-09-19T09:27:11.347269+0000","last_clean":"2024-09-19T09:27:11.347269+0000","last_became_active":"2024-09-19T09:25:29.288355+0000","last_became_peered":"2024-09-19T09:25:29.288355+0000","last_unstale":"2024-09-19T09:27:11.347269+0000","last_undegraded":"2024-09-19T09:27:11.347269+0000","last_fullsized":"2024-09-19T09:27:11.347269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:46.237558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991315+0000","last_change":"2024-09-19T09:25:27.323312+0000","last_active":"2024-09-19T09:25:42.991315+0000","last_peered":"2024-09-19T09:25:42.991315+0000","last_clean":"2024-09-19T09:25:42.991315+0000","last_became_active":"2024-09-19T09:25:27.322984+0000","last_became_peered":"2024-09-19T09:25:27.322984+0000","last_unstale":"2024-09-19T09:25:42.991315+0000","last_undegraded":"2024-09-19T09:25:42.991315+0000","last_fullsized":"2024-09-19T09:25:42.991315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:48.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.327987+0000","last_change":"2024-09-19T09:25:29.301832+0000","last_active":"2024-09-19T09:27:11.327987+0000","last_peered":"2024-09-19T09:27:11.327987+0000","last_clean":"2024-09-19T09:27:11.327987+0000","last_became_active":"2024-09-19T09:25:29.301539+0000","last_became_peered":"2024-09-19T09:25:29.301539+0000","last_unstale":"2024-09-19T09:27:11.327987+0000","last_undegraded":"2024-09-19T09:27:11.327987+0000","last_fullsized":"2024-09-19T09:27:11.327987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:00.244948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'14","reported_seq":154,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:16.116536+0000","last_change":"2024-09-19T09:25:43.009668+0000","last_active":"2024-09-19T09:27:16.116536+0000","last_peered":"2024-09-19T09:27:16.116536+0000","last_clean":"2024-09-19T09:27:16.116536+0000","last_became_active":"2024-09-19T09:25:42.994966+0000","last_became_peered":"2024-09-19T09:25:42.994966+0000","last_unstale":"2024-09-19T09:27:16.116536+0000","last_undegraded":"2024-09-19T09:27:16.116536+0000","last_fullsized":"2024-09-19T09:27:16.116536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:04:05.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68571,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1020,"num_read_kb":587,"num_write":258,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1842,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612644,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":431506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841576,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1593344,"data_stored":1081568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005100,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":917504,"data_stored":429922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234033,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":430300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1023357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593788,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":901120,"data_stored":430173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1022639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":431738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215178,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":490149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":491099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":431512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:27:22.848 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:27:23.123 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:22 smithi016 ceph-mon[30979]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T09:27:23.124 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:22 smithi016 ceph-mon[30979]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:23.146 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:22 smithi142 ceph-mon[35032]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T09:27:23.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:22 smithi142 ceph-mon[35032]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:23.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:22 smithi097 ceph-mon[34864]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-19T09:27:23.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:22 smithi097 ceph-mon[34864]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.642+0000 7f74c24b4640 1 -- 172.21.15.16:0/2946597934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc101de0 msgr2=0x7f74bc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.642+0000 7f74c24b4640 1 --2- 172.21.15.16:0/2946597934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc101de0 0x7f74bc1021c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009900 tx=0x7f74ac02a410 comp rx=0 tx=0).stop 2024-09-19T09:27:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.642+0000 7f74c24b4640 1 -- 172.21.15.16:0/2946597934 shutdown_connections 2024-09-19T09:27:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.642+0000 7f74c24b4640 1 --2- 172.21.15.16:0/2946597934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74bc106f10 0x7f74bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.642+0000 7f74c24b4640 1 --2- 172.21.15.16:0/2946597934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74bc102790 0x7f74bc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.642+0000 7f74c24b4640 1 --2- 172.21.15.16:0/2946597934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc101de0 0x7f74bc1021c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.642+0000 7f74c24b4640 1 -- 172.21.15.16:0/2946597934 >> 172.21.15.16:0/2946597934 conn(0x7f74bc0fd8c0 msgr2=0x7f74bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.642+0000 7f74c24b4640 1 -- 172.21.15.16:0/2946597934 shutdown_connections 2024-09-19T09:27:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.642+0000 7f74c24b4640 1 -- 172.21.15.16:0/2946597934 wait complete. 2024-09-19T09:27:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.643+0000 7f74c24b4640 1 Processor -- start 2024-09-19T09:27:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.643+0000 7f74c24b4640 1 -- start start 2024-09-19T09:27:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.643+0000 7f74c24b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc101de0 0x7f74bc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.643+0000 7f74c24b4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74bc102790 0x7f74bc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.643+0000 7f74c24b4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74bc106f10 0x7f74bc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.643+0000 7f74c24b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc111f70 con 0x7f74bc101de0 2024-09-19T09:27:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.643+0000 7f74c24b4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc111df0 con 0x7f74bc102790 2024-09-19T09:27:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.643+0000 7f74c24b4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc1120f0 con 0x7f74bc106f10 2024-09-19T09:27:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.643+0000 7f74bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc101de0 0x7f74bc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.643+0000 7f74bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc101de0 0x7f74bc198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39722/0 (socket says 172.21.15.16:39722) 2024-09-19T09:27:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.643+0000 7f74bbfff640 1 -- 172.21.15.16:0/3695421524 learned_addr learned my addr 172.21.15.16:0/3695421524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.643+0000 7f74c0a51640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74bc106f10 0x7f74bc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.643+0000 7f74bb7fe640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74bc102790 0x7f74bc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.644+0000 7f74bbfff640 1 -- 172.21.15.16:0/3695421524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74bc106f10 msgr2=0x7f74bc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.644+0000 7f74bbfff640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74bc106f10 0x7f74bc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.644+0000 7f74bbfff640 1 -- 172.21.15.16:0/3695421524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74bc102790 msgr2=0x7f74bc198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.644+0000 7f74bbfff640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74bc102790 0x7f74bc198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.644+0000 7f74bbfff640 1 -- 172.21.15.16:0/3695421524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc19d3e0 con 0x7f74bc101de0 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.644+0000 7f74bbfff640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc101de0 0x7f74bc198430 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f74ac006010 tx=0x7f74ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.644+0000 7f74b97fa640 1 -- 172.21.15.16:0/3695421524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac03f030 con 0x7f74bc101de0 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.644+0000 7f74b97fa640 1 -- 172.21.15.16:0/3695421524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ac003ce0 con 0x7f74bc101de0 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.644+0000 7f74c24b4640 1 -- 172.21.15.16:0/3695421524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc19d670 con 0x7f74bc101de0 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.644+0000 7f74b97fa640 1 -- 172.21.15.16:0/3695421524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac035070 con 0x7f74bc101de0 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.644+0000 7f74c24b4640 1 -- 172.21.15.16:0/3695421524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc1a4fb0 con 0x7f74bc101de0 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.645+0000 7f74c24b4640 1 -- 172.21.15.16:0/3695421524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74bc1032c0 con 0x7f74bc101de0 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.646+0000 7f74b97fa640 1 -- 172.21.15.16:0/3695421524 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f74ac042030 con 0x7f74bc101de0 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.647+0000 7f74b97fa640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f74a40a3240 0x7f74a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.647+0000 7f74bb7fe640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f74a40a3240 0x7f74a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.647+0000 7f74b97fa640 1 -- 172.21.15.16:0/3695421524 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f74ac0eff20 con 0x7f74bc101de0 2024-09-19T09:27:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.647+0000 7f74bb7fe640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f74a40a3240 0x7f74a40a5700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0fdce0 tx=0x7f74b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.651+0000 7f74b97fa640 1 -- 172.21.15.16:0/3695421524 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f74ac0bc6b0 con 0x7f74bc101de0 2024-09-19T09:27:23.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.865+0000 7f74c24b4640 1 -- 172.21.15.16:0/3695421524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74bc10a0f0 con 0x7f74bc101de0 2024-09-19T09:27:23.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.866+0000 7f74b97fa640 1 -- 172.21.15.16:0/3695421524 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f74ac0c09b0 con 0x7f74bc101de0 2024-09-19T09:27:23.867 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:23.868 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:27:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:27:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.868+0000 7f74c24b4640 1 -- 172.21.15.16:0/3695421524 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f74a40a3240 msgr2=0x7f74a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.868+0000 7f74c24b4640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f74a40a3240 0x7f74a40a5700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0fdce0 tx=0x7f74b000b390 comp rx=0 tx=0).stop 2024-09-19T09:27:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.868+0000 7f74c24b4640 1 -- 172.21.15.16:0/3695421524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc101de0 msgr2=0x7f74bc198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.868+0000 7f74c24b4640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc101de0 0x7f74bc198430 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f74ac006010 tx=0x7f74ac0042f0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.868+0000 7f74c24b4640 1 -- 172.21.15.16:0/3695421524 shutdown_connections 2024-09-19T09:27:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.868+0000 7f74c24b4640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f74a40a3240 0x7f74a40a5700 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.868+0000 7f74c24b4640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74bc106f10 0x7f74bc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.868+0000 7f74c24b4640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74bc102790 0x7f74bc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.868+0000 7f74c24b4640 1 --2- 172.21.15.16:0/3695421524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74bc101de0 0x7f74bc198430 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.869+0000 7f74c24b4640 1 -- 172.21.15.16:0/3695421524 >> 172.21.15.16:0/3695421524 conn(0x7f74bc0fd8c0 msgr2=0x7f74bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.869+0000 7f74c24b4640 1 -- 172.21.15.16:0/3695421524 shutdown_connections 2024-09-19T09:27:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:23.869+0000 7f74c24b4640 1 -- 172.21.15.16:0/3695421524 wait complete. 2024-09-19T09:27:24.231 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:23 smithi016 ceph-mon[30979]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:24.232 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:23 smithi016 ceph-mon[30979]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:24.232 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:23 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3695421524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:24.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:23 smithi142 ceph-mon[35032]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:24.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:23 smithi142 ceph-mon[35032]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:24.280 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:23 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3695421524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:23 smithi097 ceph-mon[34864]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:23 smithi097 ceph-mon[34864]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:23 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3695421524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:24.608 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:27:24.871 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:25.150 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:24 smithi016 ceph-mon[30979]: pgmap v332: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-09-19T09:27:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.342+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2653729894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c10c170 msgr2=0x7f8b9c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.342+0000 7f8ba2e34640 1 --2- 172.21.15.16:0/2653729894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c10c170 0x7f8b9c10e610 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009ca0 tx=0x7f8b9402a950 comp rx=0 tx=0).stop 2024-09-19T09:27:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.342+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2653729894 shutdown_connections 2024-09-19T09:27:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.342+0000 7f8ba2e34640 1 --2- 172.21.15.16:0/2653729894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b9c103660 0x7f8b9c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.342+0000 7f8ba2e34640 1 --2- 172.21.15.16:0/2653729894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c10c170 0x7f8b9c10e610 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.342+0000 7f8ba2e34640 1 --2- 172.21.15.16:0/2653729894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b9c102d40 0x7f8b9c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.342+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2653729894 >> 172.21.15.16:0/2653729894 conn(0x7f8b9c0fc7a0 msgr2=0x7f8b9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.342+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2653729894 shutdown_connections 2024-09-19T09:27:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.342+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2653729894 wait complete. 2024-09-19T09:27:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.342+0000 7f8ba2e34640 1 Processor -- start 2024-09-19T09:27:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.342+0000 7f8ba2e34640 1 -- start start 2024-09-19T09:27:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba2e34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c102d40 0x7f8b9c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba2e34640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b9c103660 0x7f8b9c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba2e34640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b9c10c170 0x7f8b9c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba2e34640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c111f70 con 0x7f8b9c102d40 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba2e34640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c111df0 con 0x7f8b9c103660 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba2e34640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c1120f0 con 0x7f8b9c10c170 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba0bd0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c102d40 0x7f8b9c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba0bd0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c102d40 0x7f8b9c198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39740/0 (socket says 172.21.15.16:39740) 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8b9bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b9c103660 0x7f8b9c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba0bd0640 1 -- 172.21.15.16:0/2808147134 learned_addr learned my addr 172.21.15.16:0/2808147134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba13d1640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b9c10c170 0x7f8b9c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba0bd0640 1 -- 172.21.15.16:0/2808147134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b9c10c170 msgr2=0x7f8b9c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba0bd0640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b9c10c170 0x7f8b9c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba0bd0640 1 -- 172.21.15.16:0/2808147134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b9c103660 msgr2=0x7f8b9c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba0bd0640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b9c103660 0x7f8b9c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba0bd0640 1 -- 172.21.15.16:0/2808147134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9c19d400 con 0x7f8b9c102d40 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba13d1640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b9c10c170 0x7f8b9c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8ba0bd0640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c102d40 0x7f8b9c198450 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00e9f0 tx=0x7f8b8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:25.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.343+0000 7f8b99ffb640 1 -- 172.21.15.16:0/2808147134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c00cd80 con 0x7f8b9c102d40 2024-09-19T09:27:25.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.344+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2808147134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9c19d6f0 con 0x7f8b9c102d40 2024-09-19T09:27:25.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.344+0000 7f8b99ffb640 1 -- 172.21.15.16:0/2808147134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b8c010910 con 0x7f8b9c102d40 2024-09-19T09:27:25.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.344+0000 7f8b99ffb640 1 -- 172.21.15.16:0/2808147134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0188f0 con 0x7f8b9c102d40 2024-09-19T09:27:25.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.344+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2808147134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b9c1008d0 con 0x7f8b9c102d40 2024-09-19T09:27:25.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.345+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2808147134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b9c103120 con 0x7f8b9c102d40 2024-09-19T09:27:25.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.346+0000 7f8b99ffb640 1 -- 172.21.15.16:0/2808147134 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8b8c010ac0 con 0x7f8b9c102d40 2024-09-19T09:27:25.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.347+0000 7f8b99ffb640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8b840a3240 0x7f8b840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:25.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.347+0000 7f8b9bfff640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8b840a3240 0x7f8b840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:25.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.347+0000 7f8b99ffb640 1 -- 172.21.15.16:0/2808147134 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8b8c0cfd50 con 0x7f8b9c102d40 2024-09-19T09:27:25.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.348+0000 7f8b9bfff640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8b840a3240 0x7f8b840a5700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0fed50 tx=0x7f8b9000b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:25.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.352+0000 7f8b99ffb640 1 -- 172.21.15.16:0/2808147134 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8b8c09ca70 con 0x7f8b9c102d40 2024-09-19T09:27:25.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:24 smithi142 ceph-mon[35032]: pgmap v332: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-09-19T09:27:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:24 smithi097 ceph-mon[34864]: pgmap v332: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-09-19T09:27:25.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.529+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2808147134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b9c1071e0 con 0x7f8b9c102d40 2024-09-19T09:27:25.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.531+0000 7f8b99ffb640 1 -- 172.21.15.16:0/2808147134 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22665 (secure 0 0 0) 0x7f8b8c09c370 con 0x7f8b9c102d40 2024-09-19T09:27:25.532 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:25.532 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","created":"2024-09-19T09:17:48.242238+0000","modified":"2024-09-19T09:25:42.982671+0000","last_up_change":"2024-09-19T09:24:40.190289+0000","last_in_change":"2024-09-19T09:24:17.190380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:20:57.748686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:25:25.694451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:25:28.262997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbf3b28b-2282-4057-9e65-19ceb78ff19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6803","nonce":1578957623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6805","nonce":1578957623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6809","nonce":1578957623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6807","nonce":1578957623}]},"public_addr":"172.21.15.16:6803/1578957623","cluster_addr":"172.21.15.16:6805/1578957623","heartbeat_back_addr":"172.21.15.16:6809/1578957623","heartbeat_front_addr":"172.21.15.16:6807/1578957623","state":["exists","up"]},{"osd":1,"uuid":"c2517aeb-1d79-41ad-a180-369eecb9732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6801","nonce":3804152349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6803","nonce":3804152349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6807","nonce":3804152349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6805","nonce":3804152349}]},"public_addr":"172.21.15.97:6801/3804152349","cluster_addr":"172.21.15.97:6803/3804152349","heartbeat_back_addr":"172.21.15.97:6807/3804152349","heartbeat_front_addr":"172.21.15.97:6805/3804152349","state":["exists","up"]},{"osd":2,"uuid":"3c367518-5085-471a-98cb-ec5bf5facdf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6801","nonce":2692613951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6803","nonce":2692613951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6807","nonce":2692613951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6805","nonce":2692613951}]},"public_addr":"172.21.15.142:6801/2692613951","cluster_addr":"172.21.15.142:6803/2692613951","heartbeat_back_addr":"172.21.15.142:6807/2692613951","heartbeat_front_addr":"172.21.15.142:6805/2692613951","state":["exists","up"]},{"osd":3,"uuid":"e7118925-f0fa-46b3-8552-3e3bb76a6489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6811","nonce":3169290004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6813","nonce":3169290004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6817","nonce":3169290004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6815","nonce":3169290004}]},"public_addr":"172.21.15.16:6811/3169290004","cluster_addr":"172.21.15.16:6813/3169290004","heartbeat_back_addr":"172.21.15.16:6817/3169290004","heartbeat_front_addr":"172.21.15.16:6815/3169290004","state":["exists","up"]},{"osd":4,"uuid":"dc1964f7-af9c-4853-843b-9818f87c7768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6809","nonce":3491353825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6811","nonce":3491353825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6815","nonce":3491353825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6813","nonce":3491353825}]},"public_addr":"172.21.15.97:6809/3491353825","cluster_addr":"172.21.15.97:6811/3491353825","heartbeat_back_addr":"172.21.15.97:6815/3491353825","heartbeat_front_addr":"172.21.15.97:6813/3491353825","state":["exists","up"]},{"osd":5,"uuid":"4d7fbafc-d9ba-4036-a89d-aa295b8a23a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6809","nonce":577099986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6811","nonce":577099986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6815","nonce":577099986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6813","nonce":577099986}]},"public_addr":"172.21.15.142:6809/577099986","cluster_addr":"172.21.15.142:6811/577099986","heartbeat_back_addr":"172.21.15.142:6815/577099986","heartbeat_front_addr":"172.21.15.142:6813/577099986","state":["exists","up"]},{"osd":6,"uuid":"e3f346fc-c027-44f8-bd37-bccf6ec09e3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6819","nonce":4053539536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6821","nonce":4053539536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6825","nonce":4053539536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6823","nonce":4053539536}]},"public_addr":"172.21.15.16:6819/4053539536","cluster_addr":"172.21.15.16:6821/4053539536","heartbeat_back_addr":"172.21.15.16:6825/4053539536","heartbeat_front_addr":"172.21.15.16:6823/4053539536","state":["exists","up"]},{"osd":7,"uuid":"d390b547-8949-4d4b-8481-c49d6ae7e424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6817","nonce":1157122670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6819","nonce":1157122670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6823","nonce":1157122670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6821","nonce":1157122670}]},"public_addr":"172.21.15.97:6817/1157122670","cluster_addr":"172.21.15.97:6819/1157122670","heartbeat_back_addr":"172.21.15.97:6823/1157122670","heartbeat_front_addr":"172.21.15.97:6821/1157122670","state":["exists","up"]},{"osd":8,"uuid":"78a476c4-5deb-4f9e-a266-db4493850732","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6817","nonce":643795004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6819","nonce":643795004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6823","nonce":643795004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6821","nonce":643795004}]},"public_addr":"172.21.15.142:6817/643795004","cluster_addr":"172.21.15.142:6819/643795004","heartbeat_back_addr":"172.21.15.142:6823/643795004","heartbeat_front_addr":"172.21.15.142:6821/643795004","state":["exists","up"]},{"osd":9,"uuid":"0415d187-4a77-42f7-b8bb-2d0b52e9cb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6827","nonce":2651132360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6829","nonce":2651132360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2651132360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6831","nonce":2651132360}]},"public_addr":"172.21.15.16:6827/2651132360","cluster_addr":"172.21.15.16:6829/2651132360","heartbeat_back_addr":"172.21.15.16:6833/2651132360","heartbeat_front_addr":"172.21.15.16:6831/2651132360","state":["exists","up"]},{"osd":10,"uuid":"6fa78f96-cdfe-42a0-8d27-0705a0c7df8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6825","nonce":1848371486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6827","nonce":1848371486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6831","nonce":1848371486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6829","nonce":1848371486}]},"public_addr":"172.21.15.97:6825/1848371486","cluster_addr":"172.21.15.97:6827/1848371486","heartbeat_back_addr":"172.21.15.97:6831/1848371486","heartbeat_front_addr":"172.21.15.97:6829/1848371486","state":["exists","up"]},{"osd":11,"uuid":"d27f20e3-84ce-470c-9889-cd86906b158d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6825","nonce":155083878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6827","nonce":155083878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6831","nonce":155083878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6829","nonce":155083878}]},"public_addr":"172.21.15.142:6825/155083878","cluster_addr":"172.21.15.142:6827/155083878","heartbeat_back_addr":"172.21.15.142:6831/155083878","heartbeat_front_addr":"172.21.15.142:6829/155083878","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:05.088764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:29.928529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:55.236930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:20.379471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:44.984151+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:10.410074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:35.233553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:59.928700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:24.503069+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:49.853073+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:14.006841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:38.546467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3710267992":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3740275561":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/2755421444":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/357390846":"2024-09-20T09:18:17.070034+0000","172.21.15.16:6800/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3104087402":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3546947161":"2024-09-20T09:18:17.070034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:27:25.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.533+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2808147134 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8b840a3240 msgr2=0x7f8b840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.533+0000 7f8ba2e34640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8b840a3240 0x7f8b840a5700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0fed50 tx=0x7f8b9000b2e0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.533+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2808147134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c102d40 msgr2=0x7f8b9c198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:25.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.533+0000 7f8ba2e34640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c102d40 0x7f8b9c198450 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00e9f0 tx=0x7f8b8c00eeb0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.533+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2808147134 shutdown_connections 2024-09-19T09:27:25.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.533+0000 7f8ba2e34640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8b840a3240 0x7f8b840a5700 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.533+0000 7f8ba2e34640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b9c10c170 0x7f8b9c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.533+0000 7f8ba2e34640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b9c103660 0x7f8b9c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.533+0000 7f8ba2e34640 1 --2- 172.21.15.16:0/2808147134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c102d40 0x7f8b9c198450 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:25.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.533+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2808147134 >> 172.21.15.16:0/2808147134 conn(0x7f8b9c0fc7a0 msgr2=0x7f8b9c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:25.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.533+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2808147134 shutdown_connections 2024-09-19T09:27:25.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:25.533+0000 7f8ba2e34640 1 -- 172.21.15.16:0/2808147134 wait complete. 2024-09-19T09:27:26.147 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T09:27:26.147 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-19T09:27:26.158 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T09:27:26.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-19T09:27:26.158 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T09:27:26.158 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:27:26.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:27:26.161 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-19T09:27:26.190 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T09:27:26.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:27:26.192 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:25 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2808147134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:27:26.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:25 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2808147134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:27:26.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:25 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2808147134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:27:26.436 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:26.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.946+0000 7f329f2bd640 1 -- 172.21.15.16:0/3198443279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f329810e480 msgr2=0x7f3298110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.946+0000 7f329f2bd640 1 --2- 172.21.15.16:0/3198443279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f329810e480 0x7f3298110870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3290009900 tx=0x7f329002a410 comp rx=0 tx=0).stop 2024-09-19T09:27:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.946+0000 7f329f2bd640 1 -- 172.21.15.16:0/3198443279 shutdown_connections 2024-09-19T09:27:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.946+0000 7f329f2bd640 1 --2- 172.21.15.16:0/3198443279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f329810e480 0x7f3298110870 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.946+0000 7f329f2bd640 1 --2- 172.21.15.16:0/3198443279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3298101280 0x7f329810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.946+0000 7f329f2bd640 1 --2- 172.21.15.16:0/3198443279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32981008d0 0x7f3298100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.946+0000 7f329f2bd640 1 -- 172.21.15.16:0/3198443279 >> 172.21.15.16:0/3198443279 conn(0x7f32980fc7a0 msgr2=0x7f32980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.946+0000 7f329f2bd640 1 -- 172.21.15.16:0/3198443279 shutdown_connections 2024-09-19T09:27:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.946+0000 7f329f2bd640 1 -- 172.21.15.16:0/3198443279 wait complete. 2024-09-19T09:27:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.946+0000 7f329f2bd640 1 Processor -- start 2024-09-19T09:27:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.946+0000 7f329f2bd640 1 -- start start 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329f2bd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32981008d0 0x7f329819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329f2bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3298101280 0x7f329819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329f2bd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f329810e480 0x7f329819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329f2bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32981141b0 con 0x7f3298101280 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329f2bd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3298114030 con 0x7f329810e480 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329f2bd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3298114330 con 0x7f32981008d0 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329c858640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3298101280 0x7f329819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329c858640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3298101280 0x7f329819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42584/0 (socket says 172.21.15.16:42584) 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329c858640 1 -- 172.21.15.16:0/561885662 learned_addr learned my addr 172.21.15.16:0/561885662 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329d059640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32981008d0 0x7f329819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329d85a640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f329810e480 0x7f329819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329c858640 1 -- 172.21.15.16:0/561885662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32981008d0 msgr2=0x7f329819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329c858640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32981008d0 0x7f329819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329c858640 1 -- 172.21.15.16:0/561885662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f329810e480 msgr2=0x7f329819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329c858640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f329810e480 0x7f329819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329c858640 1 -- 172.21.15.16:0/561885662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329819f600 con 0x7f3298101280 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.947+0000 7f329c858640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3298101280 0x7f329819ab90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f328c00d950 tx=0x7f328c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.948+0000 7f32967fc640 1 -- 172.21.15.16:0/561885662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c017070 con 0x7f3298101280 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.948+0000 7f329f2bd640 1 -- 172.21.15.16:0/561885662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329819f8f0 con 0x7f3298101280 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.948+0000 7f32967fc640 1 -- 172.21.15.16:0/561885662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f328c009c30 con 0x7f3298101280 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.948+0000 7f32967fc640 1 -- 172.21.15.16:0/561885662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c00f6b0 con 0x7f3298101280 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.948+0000 7f329f2bd640 1 -- 172.21.15.16:0/561885662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32981a71d0 con 0x7f3298101280 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.950+0000 7f32967fc640 1 -- 172.21.15.16:0/561885662 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f328c0109d0 con 0x7f3298101280 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.950+0000 7f32a1696740 1 Processor -- start 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.950+0000 7f32a1696740 1 -- start start 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f32a1696740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105db60 0x562f0105df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f32a1696740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105e480 0x562f01068800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f32a1696740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f0106b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f32a1696740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f00fc3280 con 0x562f01068d40 2024-09-19T09:27:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f32a1696740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562f00f6ab10 con 0x562f0105db60 2024-09-19T09:27:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f32a1696740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562f00fbf240 con 0x562f0105e480 2024-09-19T09:27:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f32967fc640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f32800a3240 0x7f32800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f329c858640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f0106b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f329d85a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105db60 0x562f0105df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f329c858640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f0106b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42586/0 (socket says 172.21.15.16:42586) 2024-09-19T09:27:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f329c858640 1 -- 172.21.15.16:0/3854148392 learned_addr learned my addr 172.21.15.16:0/3854148392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f329d85a640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f32800a3240 0x7f32800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f32967fc640 1 -- 172.21.15.16:0/561885662 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f328c0cf5c0 con 0x7f3298101280 2024-09-19T09:27:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f329d059640 1 --2- 172.21.15.16:0/3854148392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105e480 0x562f01068800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f329c858640 1 -- 172.21.15.16:0/3854148392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105e480 msgr2=0x562f01068800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:26.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f329c858640 1 --2- 172.21.15.16:0/3854148392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105e480 0x562f01068800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f329c858640 1 -- 172.21.15.16:0/3854148392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105db60 msgr2=0x562f0105df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f329c858640 1 --2- 172.21.15.16:0/3854148392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105db60 0x562f0105df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f329c858640 1 -- 172.21.15.16:0/3854148392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f00f771f0 con 0x562f01068d40 2024-09-19T09:27:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f329c858640 1 --2- 172.21.15.16:0/3854148392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f0106b130 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f328c0cd0c0 tx=0x7f328c06ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38674bada996a774 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.951+0000 7f329d85a640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f32800a3240 0x7f32800a5700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f329002a930 tx=0x7f3290005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.952+0000 7f327f7fe640 1 -- 172.21.15.16:0/3854148392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c020070 con 0x562f01068d40 2024-09-19T09:27:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.952+0000 7f327f7fe640 1 -- 172.21.15.16:0/3854148392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f328c09c2e0 con 0x562f01068d40 2024-09-19T09:27:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.952+0000 7f327f7fe640 1 -- 172.21.15.16:0/3854148392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c0a14a0 con 0x562f01068d40 2024-09-19T09:27:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.952+0000 7f32a1696740 1 -- 172.21.15.16:0/3854148392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 msgr2=0x562f0106b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.952+0000 7f32a1696740 1 --2- 172.21.15.16:0/3854148392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f0106b130 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f328c0cd0c0 tx=0x7f328c06ea60 comp rx=0 tx=0).stop 2024-09-19T09:27:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.952+0000 7f32a1696740 1 -- 172.21.15.16:0/3854148392 shutdown_connections 2024-09-19T09:27:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.952+0000 7f32a1696740 1 --2- 172.21.15.16:0/3854148392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105e480 0x562f01068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.952+0000 7f32a1696740 1 --2- 172.21.15.16:0/3854148392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f0106b130 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.952+0000 7f32a1696740 1 --2- 172.21.15.16:0/3854148392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105db60 0x562f0105df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.952+0000 7f32a1696740 1 -- 172.21.15.16:0/3854148392 >> 172.21.15.16:0/3854148392 conn(0x562f0105a0f0 msgr2=0x562f0105a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.952+0000 7f32a1696740 1 -- 172.21.15.16:0/3854148392 shutdown_connections 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.952+0000 7f32a1696740 1 -- 172.21.15.16:0/3854148392 wait complete. 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.953+0000 7f32a1696740 1 -- start start 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.953+0000 7f32a1696740 1 Processor -- start 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.953+0000 7f32a1696740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105db60 0x562f01065ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.953+0000 7f32a1696740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105e480 0x562f010661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.953+0000 7f32a1696740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f01066720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.953+0000 7f32a1696740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f0106b910 con 0x562f01068d40 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.953+0000 7f32a1696740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562f00f7df10 con 0x562f0105e480 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.953+0000 7f32a1696740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562f0106ba90 con 0x562f0105db60 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.953+0000 7f32a1696740 10 client.0 _ll_register_callbacks cb 0x562f00f46a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:26.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.953+0000 7f329d85a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f01066720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.953+0000 7f329d85a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f01066720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42602/0 (socket says 172.21.15.16:42602) 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.953+0000 7f329d85a640 1 -- 172.21.15.16:0/2640652389 learned_addr learned my addr 172.21.15.16:0/2640652389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.953+0000 7f329c858640 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105e480 0x562f010661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f329d059640 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105db60 0x562f01065ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f329d85a640 1 -- 172.21.15.16:0/2640652389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105db60 msgr2=0x562f01065ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f329d85a640 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105db60 0x562f01065ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f329d85a640 1 -- 172.21.15.16:0/2640652389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105e480 msgr2=0x562f010661e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f329d85a640 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105e480 0x562f010661e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f329d85a640 1 -- 172.21.15.16:0/2640652389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f00d346a0 con 0x562f01068d40 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f329d059640 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105db60 0x562f01065ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f329d85a640 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f01066720 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3290033000 tx=0x7f3290009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdffa6d6be52cde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f327cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f327cff9640 1 -- 172.21.15.16:0/2640652389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290035040 con 0x562f01068d40 2024-09-19T09:27:26.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f327cff9640 1 -- 172.21.15.16:0/2640652389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f32900317c0 con 0x562f01068d40 2024-09-19T09:27:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f32a1696740 1 -- 172.21.15.16:0/2640652389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f328c09c2e0 con 0x562f01068d40 2024-09-19T09:27:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f327cff9640 1 -- 172.21.15.16:0/2640652389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3290031a00 con 0x562f01068d40 2024-09-19T09:27:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f32a1696740 10 client.15261 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f32a1696740 1 -- 172.21.15.16:0/2640652389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f010775e0 con 0x562f01068d40 2024-09-19T09:27:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.954+0000 7f327cff9640 1 -- 172.21.15.16:0/2640652389 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f328c09c2e0 con 0x562f01068d40 2024-09-19T09:27:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.955+0000 7f327cff9640 1 -- 172.21.15.16:0/2640652389 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f329003a040 con 0x562f01068d40 2024-09-19T09:27:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.955+0000 7f32a1696740 10 client.15261 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.955+0000 7f32a1696740 10 client.15261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.955+0000 7f32a1696740 20 client.15261 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.955+0000 7f32a1696740 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f01078aa0 0x562f0107aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.955+0000 7f32a1696740 4 client.15261 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.955+0000 7f32a1696740 1 -- 172.21.15.16:0/2640652389 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f00fd07f0 con 0x562f01078aa0 2024-09-19T09:27:26.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.955+0000 7f329d059640 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f01078aa0 0x562f0107aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.955+0000 7f327cff9640 10 client.15261 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.955+0000 7f329d059640 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f01078aa0 0x562f0107aed0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50e72cd6aec61316 server_cookie=ffae8ef9bba31aab in_seq=0 out_seq=0 2024-09-19T09:27:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.956+0000 7f327cff9640 1 -- 172.21.15.16:0/2640652389 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f00fd07f0 con 0x562f01078aa0 2024-09-19T09:27:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.956+0000 7f327cff9640 10 client.15261 handle_command_reply: tid=0 2024-09-19T09:27:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.956+0000 7f32a1696740 1 client.15261 shutdown 2024-09-19T09:27:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.956+0000 7f32a1696740 1 -- 172.21.15.16:0/2640652389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 msgr2=0x562f01066720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.956+0000 7f32a1696740 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f01066720 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3290033000 tx=0x7f3290009260 comp rx=0 tx=0).stop 2024-09-19T09:27:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.956+0000 7f32a1696740 1 -- 172.21.15.16:0/2640652389 shutdown_connections 2024-09-19T09:27:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.956+0000 7f32a1696740 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f01078aa0 0x562f0107aed0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.956+0000 7f32a1696740 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f01066720 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.956+0000 7f32a1696740 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105e480 0x562f010661e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.956+0000 7f32a1696740 1 --2- 172.21.15.16:0/2640652389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105db60 0x562f01065ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.956+0000 7f327cff9640 0 client.15261 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:26.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.956+0000 7f32a1696740 1 -- 172.21.15.16:0/2640652389 >> 172.21.15.16:0/2640652389 conn(0x562f0105a0f0 msgr2=0x562f0105fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:26.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.957+0000 7f32a1696740 1 -- 172.21.15.16:0/2640652389 shutdown_connections 2024-09-19T09:27:26.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.957+0000 7f32a1696740 1 -- 172.21.15.16:0/2640652389 wait complete. 2024-09-19T09:27:26.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.957+0000 7f32a1696740 20 client.15261 trim_cache size 0 max 16384 2024-09-19T09:27:26.970 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:26.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f32a1696740 1 Processor -- start 2024-09-19T09:27:26.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f32a1696740 1 -- start start 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f32a1696740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105db60 0x562f01067b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f32a1696740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105e480 0x562f010807f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f32a1696740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f01080d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f32a1696740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32981af800 con 0x562f01068d40 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f32a1696740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f32981b1340 con 0x562f0105db60 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f32a1696740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32981afed0 con 0x562f0105e480 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f329d85a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f01080d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f329d85a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f01080d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42618/0 (socket says 172.21.15.16:42618) 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f329d85a640 1 -- 172.21.15.16:0/843821966 learned_addr learned my addr 172.21.15.16:0/843821966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f329d059640 1 --2- 172.21.15.16:0/843821966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105db60 0x562f01067b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f329c858640 1 --2- 172.21.15.16:0/843821966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105e480 0x562f010807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f329d85a640 1 -- 172.21.15.16:0/843821966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105e480 msgr2=0x562f010807f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f329d85a640 1 --2- 172.21.15.16:0/843821966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105e480 0x562f010807f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f329d85a640 1 -- 172.21.15.16:0/843821966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105db60 msgr2=0x562f01067b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f329d85a640 1 --2- 172.21.15.16:0/843821966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105db60 0x562f01067b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f329d85a640 1 -- 172.21.15.16:0/843821966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f01059f20 con 0x562f01068d40 2024-09-19T09:27:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.975+0000 7f329d059640 1 --2- 172.21.15.16:0/843821966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105db60 0x562f01067b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.976+0000 7f329d85a640 1 --2- 172.21.15.16:0/843821966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f01080d30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3290031550 tx=0x7f3290031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f0f2eceff89756 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.976+0000 7f327effd640 1 -- 172.21.15.16:0/843821966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290035070 con 0x562f01068d40 2024-09-19T09:27:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.976+0000 7f327effd640 1 -- 172.21.15.16:0/843821966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329004f070 con 0x562f01068d40 2024-09-19T09:27:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.976+0000 7f327effd640 1 -- 172.21.15.16:0/843821966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329004ac80 con 0x562f01068d40 2024-09-19T09:27:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.977+0000 7f32a1696740 1 -- 172.21.15.16:0/843821966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 msgr2=0x562f01080d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.977+0000 7f32a1696740 1 --2- 172.21.15.16:0/843821966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f01080d30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3290031550 tx=0x7f3290031460 comp rx=0 tx=0).stop 2024-09-19T09:27:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.977+0000 7f32a1696740 1 -- 172.21.15.16:0/843821966 shutdown_connections 2024-09-19T09:27:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.977+0000 7f32a1696740 1 --2- 172.21.15.16:0/843821966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105e480 0x562f010807f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.977+0000 7f32a1696740 1 --2- 172.21.15.16:0/843821966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f01080d30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.977+0000 7f32a1696740 1 --2- 172.21.15.16:0/843821966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105db60 0x562f01067b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.977+0000 7f32a1696740 1 -- 172.21.15.16:0/843821966 >> 172.21.15.16:0/843821966 conn(0x562f0105a0f0 msgr2=0x562f010673f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.977+0000 7f32a1696740 1 -- 172.21.15.16:0/843821966 shutdown_connections 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.977+0000 7f32a1696740 1 -- 172.21.15.16:0/843821966 wait complete. 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.977+0000 7f32a1696740 1 -- start start 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.977+0000 7f32a1696740 1 Processor -- start 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f32a1696740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105db60 0x562f0106cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f32a1696740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105e480 0x562f0106d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f32a1696740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f0106d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f32a1696740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32981b0c70 con 0x562f01068d40 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f32a1696740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f32981b05a0 con 0x562f0105e480 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f32a1696740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32981ad7c0 con 0x562f0105db60 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f32a1696740 10 client.0 _ll_register_callbacks cb 0x7f3290003a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:26.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f329d85a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f0106d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f329d85a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f0106d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42620/0 (socket says 172.21.15.16:42620) 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f329d85a640 1 -- 172.21.15.16:0/4163161530 learned_addr learned my addr 172.21.15.16:0/4163161530 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f329c858640 1 --2- 172.21.15.16:0/4163161530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105e480 0x562f0106d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f329d059640 1 --2- 172.21.15.16:0/4163161530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105db60 0x562f0106cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f329d85a640 1 -- 172.21.15.16:0/4163161530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105db60 msgr2=0x562f0106cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f329d85a640 1 --2- 172.21.15.16:0/4163161530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105db60 0x562f0106cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f329d85a640 1 -- 172.21.15.16:0/4163161530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105e480 msgr2=0x562f0106d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f329d85a640 1 --2- 172.21.15.16:0/4163161530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105e480 0x562f0106d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f329d85a640 1 -- 172.21.15.16:0/4163161530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f010e2930 con 0x562f01068d40 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f329d85a640 1 --2- 172.21.15.16:0/4163161530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f0106d620 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3290053420 tx=0x7f3290003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afa704af3de063 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.978+0000 7f327cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f327cff9640 1 -- 172.21.15.16:0/4163161530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290035040 con 0x562f01068d40 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f327cff9640 1 -- 172.21.15.16:0/4163161530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f329004a670 con 0x562f01068d40 2024-09-19T09:27:26.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f32a1696740 1 -- 172.21.15.16:0/4163161530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329004f070 con 0x562f01068d40 2024-09-19T09:27:26.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f327cff9640 1 -- 172.21.15.16:0/4163161530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f329004a870 con 0x562f01068d40 2024-09-19T09:27:26.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f32a1696740 10 client.15267 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:26.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f32a1696740 1 -- 172.21.15.16:0/4163161530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f010e3c30 con 0x562f01068d40 2024-09-19T09:27:26.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f327cff9640 1 -- 172.21.15.16:0/4163161530 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329004aa10 con 0x562f01068d40 2024-09-19T09:27:26.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f327cff9640 1 -- 172.21.15.16:0/4163161530 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32900534a0 con 0x562f01068d40 2024-09-19T09:27:26.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f32a1696740 10 client.15267 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:26.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f32a1696740 10 client.15267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:26.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f32a1696740 20 client.15267 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:26.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f32a1696740 1 --2- 172.21.15.16:0/4163161530 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f010e42e0 0x562f010e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:26.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f32a1696740 4 client.15267 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:27:26.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.979+0000 7f32a1696740 1 -- 172.21.15.16:0/4163161530 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f329004f070 con 0x562f010e42e0 2024-09-19T09:27:26.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.980+0000 7f329d059640 1 --2- 172.21.15.16:0/4163161530 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f010e42e0 0x562f010e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:26.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.980+0000 7f327cff9640 10 client.15267 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:26.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.980+0000 7f329d059640 1 --2- 172.21.15.16:0/4163161530 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f010e42e0 0x562f010e66d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d954d0ef39859ed server_cookie=8ce130a474fe991a in_seq=0 out_seq=0 2024-09-19T09:27:26.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.981+0000 7f327cff9640 1 -- 172.21.15.16:0/4163161530 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f329004f070 con 0x562f010e42e0 2024-09-19T09:27:26.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.981+0000 7f327cff9640 10 client.15267 handle_command_reply: tid=0 2024-09-19T09:27:26.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.981+0000 7f32a1696740 1 client.15267 shutdown 2024-09-19T09:27:26.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.981+0000 7f32a1696740 1 -- 172.21.15.16:0/4163161530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 msgr2=0x562f0106d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.981+0000 7f32a1696740 1 --2- 172.21.15.16:0/4163161530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f0106d620 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3290053420 tx=0x7f3290003eb0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.982+0000 7f32a1696740 1 -- 172.21.15.16:0/4163161530 shutdown_connections 2024-09-19T09:27:26.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.982+0000 7f32a1696740 1 --2- 172.21.15.16:0/4163161530 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f010e42e0 0x562f010e66d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.982+0000 7f32a1696740 1 --2- 172.21.15.16:0/4163161530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f01068d40 0x562f0106d620 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.982+0000 7f32a1696740 1 --2- 172.21.15.16:0/4163161530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f0105e480 0x562f0106d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.982+0000 7f327cff9640 0 client.15267 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:26.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.982+0000 7f32a1696740 1 --2- 172.21.15.16:0/4163161530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f0105db60 0x562f0106cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.982+0000 7f32a1696740 1 -- 172.21.15.16:0/4163161530 >> 172.21.15.16:0/4163161530 conn(0x562f0105a0f0 msgr2=0x562f01064140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:26.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.982+0000 7f32a1696740 1 -- 172.21.15.16:0/4163161530 shutdown_connections 2024-09-19T09:27:26.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.982+0000 7f32a1696740 1 -- 172.21.15.16:0/4163161530 wait complete. 2024-09-19T09:27:26.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.983+0000 7f32a1696740 20 client.15267 trim_cache size 0 max 16384 2024-09-19T09:27:26.987 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:27:26.987 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-19T09:27:26.987 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "a58cc1a9-d204-4dd7-9f0b-6d170fc938b4", 2024-09-19T09:27:26.987 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-19T09:27:26.987 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:27:26.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.984+0000 7f329f2bd640 1 -- 172.21.15.16:0/561885662 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f32800a3240 msgr2=0x7f32800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.984+0000 7f329f2bd640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f32800a3240 0x7f32800a5700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f329002a930 tx=0x7f3290005a20 comp rx=0 tx=0).stop 2024-09-19T09:27:26.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.984+0000 7f329f2bd640 1 -- 172.21.15.16:0/561885662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3298101280 msgr2=0x7f329819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.984+0000 7f329f2bd640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3298101280 0x7f329819ab90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f328c00d950 tx=0x7f328c00de10 comp rx=0 tx=0).stop 2024-09-19T09:27:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.984+0000 7f329f2bd640 1 -- 172.21.15.16:0/561885662 shutdown_connections 2024-09-19T09:27:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.984+0000 7f329f2bd640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f329810e480 0x7f329819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.984+0000 7f329f2bd640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3298101280 0x7f329819ab90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.984+0000 7f329f2bd640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f32800a3240 0x7f32800a5700 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.984+0000 7f329f2bd640 1 --2- 172.21.15.16:0/561885662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32981008d0 0x7f329819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.984+0000 7f329f2bd640 1 -- 172.21.15.16:0/561885662 >> 172.21.15.16:0/561885662 conn(0x7f32980fc7a0 msgr2=0x7f32980feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.984+0000 7f329f2bd640 1 -- 172.21.15.16:0/561885662 shutdown_connections 2024-09-19T09:27:26.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:26.984+0000 7f329f2bd640 1 -- 172.21.15.16:0/561885662 wait complete. 2024-09-19T09:27:27.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:26 smithi016 ceph-mon[30979]: pgmap v333: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T09:27:27.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:26 smithi142 ceph-mon[35032]: pgmap v333: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T09:27:27.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:26 smithi097 ceph-mon[34864]: pgmap v333: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T09:27:27.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.715+0000 7f998b49b640 1 -- 172.21.15.16:0/1343084914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984106f10 msgr2=0x7f998410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.715+0000 7f998b49b640 1 --2- 172.21.15.16:0/1343084914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984106f10 0x7f998410ea90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f997c009ca0 tx=0x7f997c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.715+0000 7f998b49b640 1 -- 172.21.15.16:0/1343084914 shutdown_connections 2024-09-19T09:27:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.715+0000 7f998b49b640 1 --2- 172.21.15.16:0/1343084914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984106f10 0x7f998410ea90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.715+0000 7f998b49b640 1 --2- 172.21.15.16:0/1343084914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9984102790 0x7f99841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.715+0000 7f998b49b640 1 --2- 172.21.15.16:0/1343084914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9984101de0 0x7f99841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.715+0000 7f998b49b640 1 -- 172.21.15.16:0/1343084914 >> 172.21.15.16:0/1343084914 conn(0x7f99840fd8c0 msgr2=0x7f99840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.715+0000 7f998b49b640 1 -- 172.21.15.16:0/1343084914 shutdown_connections 2024-09-19T09:27:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.715+0000 7f998b49b640 1 -- 172.21.15.16:0/1343084914 wait complete. 2024-09-19T09:27:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.716+0000 7f998b49b640 1 Processor -- start 2024-09-19T09:27:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.716+0000 7f998b49b640 1 -- start start 2024-09-19T09:27:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.716+0000 7f998b49b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9984101de0 0x7f99841983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.716+0000 7f998b49b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984102790 0x7f9984198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.716+0000 7f998b49b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9984106f10 0x7f998419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.716+0000 7f998b49b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9984111f70 con 0x7f9984102790 2024-09-19T09:27:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.716+0000 7f998b49b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9984111df0 con 0x7f9984106f10 2024-09-19T09:27:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.716+0000 7f998b49b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f99841120f0 con 0x7f9984101de0 2024-09-19T09:27:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.716+0000 7f9988a36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984102790 0x7f9984198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.716+0000 7f9988a36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984102790 0x7f9984198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42626/0 (socket says 172.21.15.16:42626) 2024-09-19T09:27:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.716+0000 7f9988a36640 1 -- 172.21.15.16:0/1288026308 learned_addr learned my addr 172.21.15.16:0/1288026308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.716+0000 7f9989237640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9984101de0 0x7f99841983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.716+0000 7f9989a38640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9984106f10 0x7f998419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.717+0000 7f9988a36640 1 -- 172.21.15.16:0/1288026308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9984101de0 msgr2=0x7f99841983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.717+0000 7f9988a36640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9984101de0 0x7f99841983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.717+0000 7f9988a36640 1 -- 172.21.15.16:0/1288026308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9984106f10 msgr2=0x7f998419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.717+0000 7f9988a36640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9984106f10 0x7f998419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.717+0000 7f9988a36640 1 -- 172.21.15.16:0/1288026308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998419d380 con 0x7f9984102790 2024-09-19T09:27:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.717+0000 7f9988a36640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984102790 0x7f9984198910 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f997400d950 tx=0x7f997400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.717+0000 7f997a7fc640 1 -- 172.21.15.16:0/1288026308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974017070 con 0x7f9984102790 2024-09-19T09:27:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.717+0000 7f997a7fc640 1 -- 172.21.15.16:0/1288026308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9974009c30 con 0x7f9984102790 2024-09-19T09:27:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.717+0000 7f998b49b640 1 -- 172.21.15.16:0/1288026308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998419d670 con 0x7f9984102790 2024-09-19T09:27:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.717+0000 7f997a7fc640 1 -- 172.21.15.16:0/1288026308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997400f6b0 con 0x7f9984102790 2024-09-19T09:27:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.717+0000 7f998b49b640 1 -- 172.21.15.16:0/1288026308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99841a4f50 con 0x7f9984102790 2024-09-19T09:27:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.718+0000 7f998b49b640 1 -- 172.21.15.16:0/1288026308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99841021c0 con 0x7f9984102790 2024-09-19T09:27:27.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.719+0000 7f997a7fc640 1 -- 172.21.15.16:0/1288026308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f99740109d0 con 0x7f9984102790 2024-09-19T09:27:27.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.720+0000 7f997a7fc640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f99700a3240 0x7f99700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:27.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.720+0000 7f9989237640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f99700a3240 0x7f99700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:27.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.720+0000 7f997a7fc640 1 -- 172.21.15.16:0/1288026308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f99740cf450 con 0x7f9984102790 2024-09-19T09:27:27.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.721+0000 7f9989237640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f99700a3240 0x7f99700a5700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9980005fd0 tx=0x7f998000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:27.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.723+0000 7f997a7fc640 1 -- 172.21.15.16:0/1288026308 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9974009de0 con 0x7f9984102790 2024-09-19T09:27:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.897+0000 7f998b49b640 1 -- 172.21.15.16:0/1288026308 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f998410a1b0 con 0x7f99700a3240 2024-09-19T09:27:27.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.904+0000 7f997a7fc640 1 -- 172.21.15.16:0/1288026308 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258892 (secure 0 0 0) 0x7f998410a1b0 con 0x7f99700a3240 2024-09-19T09:27:27.907 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:27.908 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T09:27:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.907+0000 7f998b49b640 1 -- 172.21.15.16:0/1288026308 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f99700a3240 msgr2=0x7f99700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.907+0000 7f998b49b640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f99700a3240 0x7f99700a5700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9980005fd0 tx=0x7f998000b390 comp rx=0 tx=0).stop 2024-09-19T09:27:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.907+0000 7f998b49b640 1 -- 172.21.15.16:0/1288026308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984102790 msgr2=0x7f9984198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.907+0000 7f998b49b640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984102790 0x7f9984198910 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f997400d950 tx=0x7f997400de10 comp rx=0 tx=0).stop 2024-09-19T09:27:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.907+0000 7f998b49b640 1 -- 172.21.15.16:0/1288026308 shutdown_connections 2024-09-19T09:27:27.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.907+0000 7f998b49b640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9984106f10 0x7f998419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.907+0000 7f998b49b640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984102790 0x7f9984198910 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.907+0000 7f998b49b640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f99700a3240 0x7f99700a5700 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.907+0000 7f998b49b640 1 --2- 172.21.15.16:0/1288026308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9984101de0 0x7f99841983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:27.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.907+0000 7f998b49b640 1 -- 172.21.15.16:0/1288026308 >> 172.21.15.16:0/1288026308 conn(0x7f99840fd8c0 msgr2=0x7f99840feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:27.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.907+0000 7f998b49b640 1 -- 172.21.15.16:0/1288026308 shutdown_connections 2024-09-19T09:27:27.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:27.907+0000 7f998b49b640 1 -- 172.21.15.16:0/1288026308 wait complete. 2024-09-19T09:27:28.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:27 smithi016 ceph-mon[30979]: scrub queued for path: / 2024-09-19T09:27:28.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:27 smithi016 ceph-mon[30979]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:28.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:27 smithi016 ceph-mon[30979]: scrub summary: active paths [/] 2024-09-19T09:27:28.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:27 smithi016 ceph-mon[30979]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:28.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:27:28.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:27 smithi142 ceph-mon[35032]: scrub queued for path: / 2024-09-19T09:27:28.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:27 smithi142 ceph-mon[35032]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:28.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:27 smithi142 ceph-mon[35032]: scrub summary: active paths [/] 2024-09-19T09:27:28.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:27 smithi142 ceph-mon[35032]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:28.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:27 smithi097 ceph-mon[34864]: scrub queued for path: / 2024-09-19T09:27:28.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:27 smithi097 ceph-mon[34864]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:28.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:27 smithi097 ceph-mon[34864]: scrub summary: active paths [/] 2024-09-19T09:27:28.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:27 smithi097 ceph-mon[34864]: scrub summary: idle+waiting paths [/] 2024-09-19T09:27:29.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:28 smithi016 ceph-mon[30979]: pgmap v334: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T09:27:29.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:28 smithi016 ceph-mon[30979]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:29.040 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:29.068 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-19T09:27:27.366970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665162,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1234,"num_read_kb":744,"num_write":446,"num_write_kb":3096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1182578,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391016,"kb_used_data":12648,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550808,"statfs":{"total":1151940427776,"available":1151540027392,"internally_reserved":0,"allocated":12951552,"data_stored":7141086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":76,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002934"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347860+0000","last_change":"2024-09-19T09:25:29.294965+0000","last_active":"2024-09-19T09:27:11.347860+0000","last_peered":"2024-09-19T09:27:11.347860+0000","last_clean":"2024-09-19T09:27:11.347860+0000","last_became_active":"2024-09-19T09:25:29.294157+0000","last_became_peered":"2024-09-19T09:25:29.294157+0000","last_unstale":"2024-09-19T09:27:11.347860+0000","last_undegraded":"2024-09-19T09:27:11.347860+0000","last_fullsized":"2024-09-19T09:27:11.347860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:24.643835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335438+0000","last_change":"2024-09-19T09:25:29.298888+0000","last_active":"2024-09-19T09:27:11.335438+0000","last_peered":"2024-09-19T09:27:11.335438+0000","last_clean":"2024-09-19T09:27:11.335438+0000","last_became_active":"2024-09-19T09:25:29.298245+0000","last_became_peered":"2024-09-19T09:25:29.298245+0000","last_unstale":"2024-09-19T09:27:11.335438+0000","last_undegraded":"2024-09-19T09:27:11.335438+0000","last_fullsized":"2024-09-19T09:27:11.335438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:03.162342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344990+0000","last_change":"2024-09-19T09:25:29.292300+0000","last_active":"2024-09-19T09:27:11.344990+0000","last_peered":"2024-09-19T09:27:11.344990+0000","last_clean":"2024-09-19T09:27:11.344990+0000","last_became_active":"2024-09-19T09:25:29.291973+0000","last_became_peered":"2024-09-19T09:25:29.291973+0000","last_unstale":"2024-09-19T09:27:11.344990+0000","last_undegraded":"2024-09-19T09:27:11.344990+0000","last_fullsized":"2024-09-19T09:27:11.344990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:19.672778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342109+0000","last_change":"2024-09-19T09:25:29.305695+0000","last_active":"2024-09-19T09:27:11.342109+0000","last_peered":"2024-09-19T09:27:11.342109+0000","last_clean":"2024-09-19T09:27:11.342109+0000","last_became_active":"2024-09-19T09:25:29.305244+0000","last_became_peered":"2024-09-19T09:25:29.305244+0000","last_unstale":"2024-09-19T09:27:11.342109+0000","last_undegraded":"2024-09-19T09:27:11.342109+0000","last_fullsized":"2024-09-19T09:27:11.342109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:38.054624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336846+0000","last_change":"2024-09-19T09:25:29.295259+0000","last_active":"2024-09-19T09:27:11.336846+0000","last_peered":"2024-09-19T09:27:11.336846+0000","last_clean":"2024-09-19T09:27:11.336846+0000","last_became_active":"2024-09-19T09:25:29.294936+0000","last_became_peered":"2024-09-19T09:25:29.294936+0000","last_unstale":"2024-09-19T09:27:11.336846+0000","last_undegraded":"2024-09-19T09:27:11.336846+0000","last_fullsized":"2024-09-19T09:27:11.336846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:59.193504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336175+0000","last_change":"2024-09-19T09:25:29.299650+0000","last_active":"2024-09-19T09:27:11.336175+0000","last_peered":"2024-09-19T09:27:11.336175+0000","last_clean":"2024-09-19T09:27:11.336175+0000","last_became_active":"2024-09-19T09:25:29.299182+0000","last_became_peered":"2024-09-19T09:25:29.299182+0000","last_unstale":"2024-09-19T09:27:11.336175+0000","last_undegraded":"2024-09-19T09:27:11.336175+0000","last_fullsized":"2024-09-19T09:27:11.336175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:44.637771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342329+0000","last_change":"2024-09-19T09:25:29.302501+0000","last_active":"2024-09-19T09:27:11.342329+0000","last_peered":"2024-09-19T09:27:11.342329+0000","last_clean":"2024-09-19T09:27:11.342329+0000","last_became_active":"2024-09-19T09:25:29.301419+0000","last_became_peered":"2024-09-19T09:25:29.301419+0000","last_unstale":"2024-09-19T09:27:11.342329+0000","last_undegraded":"2024-09-19T09:27:11.342329+0000","last_fullsized":"2024-09-19T09:27:11.342329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:50.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337306+0000","last_change":"2024-09-19T09:25:29.300212+0000","last_active":"2024-09-19T09:27:11.337306+0000","last_peered":"2024-09-19T09:27:11.337306+0000","last_clean":"2024-09-19T09:27:11.337306+0000","last_became_active":"2024-09-19T09:25:29.299873+0000","last_became_peered":"2024-09-19T09:25:29.299873+0000","last_unstale":"2024-09-19T09:27:11.337306+0000","last_undegraded":"2024-09-19T09:27:11.337306+0000","last_fullsized":"2024-09-19T09:27:11.337306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:34.620314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346891+0000","last_change":"2024-09-19T09:25:29.290214+0000","last_active":"2024-09-19T09:27:11.346891+0000","last_peered":"2024-09-19T09:27:11.346891+0000","last_clean":"2024-09-19T09:27:11.346891+0000","last_became_active":"2024-09-19T09:25:29.289707+0000","last_became_peered":"2024-09-19T09:25:29.289707+0000","last_unstale":"2024-09-19T09:27:11.346891+0000","last_undegraded":"2024-09-19T09:27:11.346891+0000","last_fullsized":"2024-09-19T09:27:11.346891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:16.341340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338183+0000","last_change":"2024-09-19T09:25:29.301774+0000","last_active":"2024-09-19T09:27:11.338183+0000","last_peered":"2024-09-19T09:27:11.338183+0000","last_clean":"2024-09-19T09:27:11.338183+0000","last_became_active":"2024-09-19T09:25:29.301398+0000","last_became_peered":"2024-09-19T09:25:29.301398+0000","last_unstale":"2024-09-19T09:27:11.338183+0000","last_undegraded":"2024-09-19T09:27:11.338183+0000","last_fullsized":"2024-09-19T09:27:11.338183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:36.058038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346172+0000","last_change":"2024-09-19T09:25:29.301475+0000","last_active":"2024-09-19T09:27:11.346172+0000","last_peered":"2024-09-19T09:27:11.346172+0000","last_clean":"2024-09-19T09:27:11.346172+0000","last_became_active":"2024-09-19T09:25:29.300908+0000","last_became_peered":"2024-09-19T09:25:29.300908+0000","last_unstale":"2024-09-19T09:27:11.346172+0000","last_undegraded":"2024-09-19T09:27:11.346172+0000","last_fullsized":"2024-09-19T09:27:11.346172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:44.774127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336219+0000","last_change":"2024-09-19T09:25:42.994395+0000","last_active":"2024-09-19T09:27:11.336219+0000","last_peered":"2024-09-19T09:27:11.336219+0000","last_clean":"2024-09-19T09:27:11.336219+0000","last_became_active":"2024-09-19T09:25:42.993992+0000","last_became_peered":"2024-09-19T09:25:42.993992+0000","last_unstale":"2024-09-19T09:27:11.336219+0000","last_undegraded":"2024-09-19T09:27:11.336219+0000","last_fullsized":"2024-09-19T09:27:11.336219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.486069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.340454+0000","last_change":"2024-09-19T09:25:42.989979+0000","last_active":"2024-09-19T09:27:11.340454+0000","last_peered":"2024-09-19T09:27:11.340454+0000","last_clean":"2024-09-19T09:27:11.340454+0000","last_became_active":"2024-09-19T09:25:42.989578+0000","last_became_peered":"2024-09-19T09:25:42.989578+0000","last_unstale":"2024-09-19T09:27:11.340454+0000","last_undegraded":"2024-09-19T09:27:11.340454+0000","last_fullsized":"2024-09-19T09:27:11.340454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:04.583023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336926+0000","last_change":"2024-09-19T09:25:29.299917+0000","last_active":"2024-09-19T09:27:11.336926+0000","last_peered":"2024-09-19T09:27:11.336926+0000","last_clean":"2024-09-19T09:27:11.336926+0000","last_became_active":"2024-09-19T09:25:29.299571+0000","last_became_peered":"2024-09-19T09:25:29.299571+0000","last_unstale":"2024-09-19T09:27:11.336926+0000","last_undegraded":"2024-09-19T09:27:11.336926+0000","last_fullsized":"2024-09-19T09:27:11.336926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:25.142252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335108+0000","last_change":"2024-09-19T09:25:29.299713+0000","last_active":"2024-09-19T09:27:11.335108+0000","last_peered":"2024-09-19T09:27:11.335108+0000","last_clean":"2024-09-19T09:27:11.335108+0000","last_became_active":"2024-09-19T09:25:29.299234+0000","last_became_peered":"2024-09-19T09:25:29.299234+0000","last_unstale":"2024-09-19T09:27:11.335108+0000","last_undegraded":"2024-09-19T09:27:11.335108+0000","last_fullsized":"2024-09-19T09:27:11.335108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:34.714714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346606+0000","last_change":"2024-09-19T09:25:29.293010+0000","last_active":"2024-09-19T09:27:11.346606+0000","last_peered":"2024-09-19T09:27:11.346606+0000","last_clean":"2024-09-19T09:27:11.346606+0000","last_became_active":"2024-09-19T09:25:29.292645+0000","last_became_peered":"2024-09-19T09:25:29.292645+0000","last_unstale":"2024-09-19T09:27:11.346606+0000","last_undegraded":"2024-09-19T09:27:11.346606+0000","last_fullsized":"2024-09-19T09:27:11.346606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:24.749075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347900+0000","last_change":"2024-09-19T09:25:29.294530+0000","last_active":"2024-09-19T09:27:11.347900+0000","last_peered":"2024-09-19T09:27:11.347900+0000","last_clean":"2024-09-19T09:27:11.347900+0000","last_became_active":"2024-09-19T09:25:29.293699+0000","last_became_peered":"2024-09-19T09:25:29.293699+0000","last_unstale":"2024-09-19T09:27:11.347900+0000","last_undegraded":"2024-09-19T09:27:11.347900+0000","last_fullsized":"2024-09-19T09:27:11.347900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:53.491258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336452+0000","last_change":"2024-09-19T09:25:29.290600+0000","last_active":"2024-09-19T09:27:11.336452+0000","last_peered":"2024-09-19T09:27:11.336452+0000","last_clean":"2024-09-19T09:27:11.336452+0000","last_became_active":"2024-09-19T09:25:29.290305+0000","last_became_peered":"2024-09-19T09:25:29.290305+0000","last_unstale":"2024-09-19T09:27:11.336452+0000","last_undegraded":"2024-09-19T09:27:11.336452+0000","last_fullsized":"2024-09-19T09:27:11.336452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:42.809659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338092+0000","last_change":"2024-09-19T09:25:29.300591+0000","last_active":"2024-09-19T09:27:11.338092+0000","last_peered":"2024-09-19T09:27:11.338092+0000","last_clean":"2024-09-19T09:27:11.338092+0000","last_became_active":"2024-09-19T09:25:29.300251+0000","last_became_peered":"2024-09-19T09:25:29.300251+0000","last_unstale":"2024-09-19T09:27:11.338092+0000","last_undegraded":"2024-09-19T09:27:11.338092+0000","last_fullsized":"2024-09-19T09:27:11.338092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:08.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342064+0000","last_change":"2024-09-19T09:25:29.298452+0000","last_active":"2024-09-19T09:27:11.342064+0000","last_peered":"2024-09-19T09:27:11.342064+0000","last_clean":"2024-09-19T09:27:11.342064+0000","last_became_active":"2024-09-19T09:25:29.298137+0000","last_became_peered":"2024-09-19T09:25:29.298137+0000","last_unstale":"2024-09-19T09:27:11.342064+0000","last_undegraded":"2024-09-19T09:27:11.342064+0000","last_fullsized":"2024-09-19T09:27:11.342064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:11.716228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337968+0000","last_change":"2024-09-19T09:25:29.294278+0000","last_active":"2024-09-19T09:27:11.337968+0000","last_peered":"2024-09-19T09:27:11.337968+0000","last_clean":"2024-09-19T09:27:11.337968+0000","last_became_active":"2024-09-19T09:25:29.293972+0000","last_became_peered":"2024-09-19T09:25:29.293972+0000","last_unstale":"2024-09-19T09:27:11.337968+0000","last_undegraded":"2024-09-19T09:27:11.337968+0000","last_fullsized":"2024-09-19T09:27:11.337968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:27.923839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346634+0000","last_change":"2024-09-19T09:25:29.294000+0000","last_active":"2024-09-19T09:27:11.346634+0000","last_peered":"2024-09-19T09:27:11.346634+0000","last_clean":"2024-09-19T09:27:11.346634+0000","last_became_active":"2024-09-19T09:25:29.293486+0000","last_became_peered":"2024-09-19T09:25:29.293486+0000","last_unstale":"2024-09-19T09:27:11.346634+0000","last_undegraded":"2024-09-19T09:27:11.346634+0000","last_fullsized":"2024-09-19T09:27:11.346634+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:44.053620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334040+0000","last_change":"2024-09-19T09:25:42.994951+0000","last_active":"2024-09-19T09:27:11.334040+0000","last_peered":"2024-09-19T09:27:11.334040+0000","last_clean":"2024-09-19T09:27:11.334040+0000","last_became_active":"2024-09-19T09:25:42.994588+0000","last_became_peered":"2024-09-19T09:25:42.994588+0000","last_unstale":"2024-09-19T09:27:11.334040+0000","last_undegraded":"2024-09-19T09:27:11.334040+0000","last_fullsized":"2024-09-19T09:27:11.334040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:10.141169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334658+0000","last_change":"2024-09-19T09:25:29.286395+0000","last_active":"2024-09-19T09:27:11.334658+0000","last_peered":"2024-09-19T09:27:11.334658+0000","last_clean":"2024-09-19T09:27:11.334658+0000","last_became_active":"2024-09-19T09:25:29.286048+0000","last_became_peered":"2024-09-19T09:25:29.286048+0000","last_unstale":"2024-09-19T09:27:11.334658+0000","last_undegraded":"2024-09-19T09:27:11.334658+0000","last_fullsized":"2024-09-19T09:27:11.334658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:43.121166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.327390+0000","last_change":"2024-09-19T09:25:29.298458+0000","last_active":"2024-09-19T09:27:11.327390+0000","last_peered":"2024-09-19T09:27:11.327390+0000","last_clean":"2024-09-19T09:27:11.327390+0000","last_became_active":"2024-09-19T09:25:29.297993+0000","last_became_peered":"2024-09-19T09:25:29.297993+0000","last_unstale":"2024-09-19T09:27:11.327390+0000","last_undegraded":"2024-09-19T09:27:11.327390+0000","last_fullsized":"2024-09-19T09:27:11.327390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:44.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344574+0000","last_change":"2024-09-19T09:25:29.297167+0000","last_active":"2024-09-19T09:27:11.344574+0000","last_peered":"2024-09-19T09:27:11.344574+0000","last_clean":"2024-09-19T09:27:11.344574+0000","last_became_active":"2024-09-19T09:25:29.296777+0000","last_became_peered":"2024-09-19T09:25:29.296777+0000","last_unstale":"2024-09-19T09:27:11.344574+0000","last_undegraded":"2024-09-19T09:27:11.344574+0000","last_fullsized":"2024-09-19T09:27:11.344574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:40.581643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.339599+0000","last_change":"2024-09-19T09:25:29.296169+0000","last_active":"2024-09-19T09:27:11.339599+0000","last_peered":"2024-09-19T09:27:11.339599+0000","last_clean":"2024-09-19T09:27:11.339599+0000","last_became_active":"2024-09-19T09:25:29.295806+0000","last_became_peered":"2024-09-19T09:25:29.295806+0000","last_unstale":"2024-09-19T09:27:11.339599+0000","last_undegraded":"2024-09-19T09:27:11.339599+0000","last_fullsized":"2024-09-19T09:27:11.339599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:39.639884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334590+0000","last_change":"2024-09-19T09:25:29.297960+0000","last_active":"2024-09-19T09:27:11.334590+0000","last_peered":"2024-09-19T09:27:11.334590+0000","last_clean":"2024-09-19T09:27:11.334590+0000","last_became_active":"2024-09-19T09:25:29.297577+0000","last_became_peered":"2024-09-19T09:25:29.297577+0000","last_unstale":"2024-09-19T09:27:11.334590+0000","last_undegraded":"2024-09-19T09:27:11.334590+0000","last_fullsized":"2024-09-19T09:27:11.334590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:27.035504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335188+0000","last_change":"2024-09-19T09:25:29.297896+0000","last_active":"2024-09-19T09:27:11.335188+0000","last_peered":"2024-09-19T09:27:11.335188+0000","last_clean":"2024-09-19T09:27:11.335188+0000","last_became_active":"2024-09-19T09:25:29.297594+0000","last_became_peered":"2024-09-19T09:25:29.297594+0000","last_unstale":"2024-09-19T09:27:11.335188+0000","last_undegraded":"2024-09-19T09:27:11.335188+0000","last_fullsized":"2024-09-19T09:27:11.335188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.366325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334560+0000","last_change":"2024-09-19T09:25:29.301851+0000","last_active":"2024-09-19T09:27:11.334560+0000","last_peered":"2024-09-19T09:27:11.334560+0000","last_clean":"2024-09-19T09:27:11.334560+0000","last_became_active":"2024-09-19T09:25:29.299282+0000","last_became_peered":"2024-09-19T09:25:29.299282+0000","last_unstale":"2024-09-19T09:27:11.334560+0000","last_undegraded":"2024-09-19T09:27:11.334560+0000","last_fullsized":"2024-09-19T09:27:11.334560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:00.382114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342990+0000","last_change":"2024-09-19T09:25:29.303316+0000","last_active":"2024-09-19T09:27:11.342990+0000","last_peered":"2024-09-19T09:27:11.342990+0000","last_clean":"2024-09-19T09:27:11.342990+0000","last_became_active":"2024-09-19T09:25:29.302994+0000","last_became_peered":"2024-09-19T09:25:29.302994+0000","last_unstale":"2024-09-19T09:27:11.342990+0000","last_undegraded":"2024-09-19T09:27:11.342990+0000","last_fullsized":"2024-09-19T09:27:11.342990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:46.233991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341767+0000","last_change":"2024-09-19T09:25:29.293004+0000","last_active":"2024-09-19T09:27:11.341767+0000","last_peered":"2024-09-19T09:27:11.341767+0000","last_clean":"2024-09-19T09:27:11.341767+0000","last_became_active":"2024-09-19T09:25:29.292657+0000","last_became_peered":"2024-09-19T09:25:29.292657+0000","last_unstale":"2024-09-19T09:27:11.341767+0000","last_undegraded":"2024-09-19T09:27:11.341767+0000","last_fullsized":"2024-09-19T09:27:11.341767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:50.966477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.332473+0000","last_change":"2024-09-19T09:25:29.300434+0000","last_active":"2024-09-19T09:27:11.332473+0000","last_peered":"2024-09-19T09:27:11.332473+0000","last_clean":"2024-09-19T09:27:11.332473+0000","last_became_active":"2024-09-19T09:25:29.300053+0000","last_became_peered":"2024-09-19T09:25:29.300053+0000","last_unstale":"2024-09-19T09:27:11.332473+0000","last_undegraded":"2024-09-19T09:27:11.332473+0000","last_fullsized":"2024-09-19T09:27:11.332473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:13.283298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":137,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:21.114392+0000","last_change":"2024-09-19T09:25:27.280792+0000","last_active":"2024-09-19T09:27:21.114392+0000","last_peered":"2024-09-19T09:27:21.114392+0000","last_clean":"2024-09-19T09:27:21.114392+0000","last_became_active":"2024-09-19T09:25:27.280443+0000","last_became_peered":"2024-09-19T09:25:27.280443+0000","last_unstale":"2024-09-19T09:27:21.114392+0000","last_undegraded":"2024-09-19T09:27:21.114392+0000","last_fullsized":"2024-09-19T09:27:21.114392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.599512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334165+0000","last_change":"2024-09-19T09:25:29.298975+0000","last_active":"2024-09-19T09:27:11.334165+0000","last_peered":"2024-09-19T09:27:11.334165+0000","last_clean":"2024-09-19T09:27:11.334165+0000","last_became_active":"2024-09-19T09:25:29.298476+0000","last_became_peered":"2024-09-19T09:25:29.298476+0000","last_unstale":"2024-09-19T09:27:11.334165+0000","last_undegraded":"2024-09-19T09:27:11.334165+0000","last_fullsized":"2024-09-19T09:27:11.334165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:09.658894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986623+0000","last_change":"2024-09-19T09:25:27.278257+0000","last_active":"2024-09-19T09:25:42.986623+0000","last_peered":"2024-09-19T09:25:42.986623+0000","last_clean":"2024-09-19T09:25:42.986623+0000","last_became_active":"2024-09-19T09:25:27.277897+0000","last_became_peered":"2024-09-19T09:25:27.277897+0000","last_unstale":"2024-09-19T09:25:42.986623+0000","last_undegraded":"2024-09-19T09:25:42.986623+0000","last_fullsized":"2024-09-19T09:25:42.986623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:47.550314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985617+0000","last_change":"2024-09-19T09:25:27.272215+0000","last_active":"2024-09-19T09:25:42.985617+0000","last_peered":"2024-09-19T09:25:42.985617+0000","last_clean":"2024-09-19T09:25:42.985617+0000","last_became_active":"2024-09-19T09:25:27.271821+0000","last_became_peered":"2024-09-19T09:25:27.271821+0000","last_unstale":"2024-09-19T09:25:42.985617+0000","last_undegraded":"2024-09-19T09:25:42.985617+0000","last_fullsized":"2024-09-19T09:25:42.985617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:30.615730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.329744+0000","last_change":"2024-09-19T09:25:29.289347+0000","last_active":"2024-09-19T09:27:11.329744+0000","last_peered":"2024-09-19T09:27:11.329744+0000","last_clean":"2024-09-19T09:27:11.329744+0000","last_became_active":"2024-09-19T09:25:29.289015+0000","last_became_peered":"2024-09-19T09:25:29.289015+0000","last_unstale":"2024-09-19T09:27:11.329744+0000","last_undegraded":"2024-09-19T09:27:11.329744+0000","last_fullsized":"2024-09-19T09:27:11.329744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:18.419334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995432+0000","last_change":"2024-09-19T09:25:27.277342+0000","last_active":"2024-09-19T09:25:42.995432+0000","last_peered":"2024-09-19T09:25:42.995432+0000","last_clean":"2024-09-19T09:25:42.995432+0000","last_became_active":"2024-09-19T09:25:27.277016+0000","last_became_peered":"2024-09-19T09:25:27.277016+0000","last_unstale":"2024-09-19T09:25:42.995432+0000","last_undegraded":"2024-09-19T09:25:42.995432+0000","last_fullsized":"2024-09-19T09:25:42.995432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:56.736156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336629+0000","last_change":"2024-09-19T09:25:29.290924+0000","last_active":"2024-09-19T09:27:11.336629+0000","last_peered":"2024-09-19T09:27:11.336629+0000","last_clean":"2024-09-19T09:27:11.336629+0000","last_became_active":"2024-09-19T09:25:29.290482+0000","last_became_peered":"2024-09-19T09:25:29.290482+0000","last_unstale":"2024-09-19T09:27:11.336629+0000","last_undegraded":"2024-09-19T09:27:11.336629+0000","last_fullsized":"2024-09-19T09:27:11.336629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:16.265040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'6","reported_seq":242,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:26.121271+0000","last_change":"2024-09-19T09:25:27.277604+0000","last_active":"2024-09-19T09:27:26.121271+0000","last_peered":"2024-09-19T09:27:26.121271+0000","last_clean":"2024-09-19T09:27:26.121271+0000","last_became_active":"2024-09-19T09:25:27.277271+0000","last_became_peered":"2024-09-19T09:25:27.277271+0000","last_unstale":"2024-09-19T09:27:26.121271+0000","last_undegraded":"2024-09-19T09:27:26.121271+0000","last_fullsized":"2024-09-19T09:27:26.121271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:32.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347371+0000","last_change":"2024-09-19T09:25:29.290537+0000","last_active":"2024-09-19T09:27:11.347371+0000","last_peered":"2024-09-19T09:27:11.347371+0000","last_clean":"2024-09-19T09:27:11.347371+0000","last_became_active":"2024-09-19T09:25:29.287984+0000","last_became_peered":"2024-09-19T09:25:29.287984+0000","last_unstale":"2024-09-19T09:27:11.347371+0000","last_undegraded":"2024-09-19T09:27:11.347371+0000","last_fullsized":"2024-09-19T09:27:11.347371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:26.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990018+0000","last_change":"2024-09-19T09:25:27.278560+0000","last_active":"2024-09-19T09:25:42.990018+0000","last_peered":"2024-09-19T09:25:42.990018+0000","last_clean":"2024-09-19T09:25:42.990018+0000","last_became_active":"2024-09-19T09:25:27.277950+0000","last_became_peered":"2024-09-19T09:25:27.277950+0000","last_unstale":"2024-09-19T09:25:42.990018+0000","last_undegraded":"2024-09-19T09:25:42.990018+0000","last_fullsized":"2024-09-19T09:25:42.990018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:28.387725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.339317+0000","last_change":"2024-09-19T09:25:29.298897+0000","last_active":"2024-09-19T09:27:11.339317+0000","last_peered":"2024-09-19T09:27:11.339317+0000","last_clean":"2024-09-19T09:27:11.339317+0000","last_became_active":"2024-09-19T09:25:29.298617+0000","last_became_peered":"2024-09-19T09:25:29.298617+0000","last_unstale":"2024-09-19T09:27:11.339317+0000","last_undegraded":"2024-09-19T09:27:11.339317+0000","last_fullsized":"2024-09-19T09:27:11.339317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:53.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994870+0000","last_change":"2024-09-19T09:25:27.324871+0000","last_active":"2024-09-19T09:25:42.994870+0000","last_peered":"2024-09-19T09:25:42.994870+0000","last_clean":"2024-09-19T09:25:42.994870+0000","last_became_active":"2024-09-19T09:25:27.324471+0000","last_became_peered":"2024-09-19T09:25:27.324471+0000","last_unstale":"2024-09-19T09:25:42.994870+0000","last_undegraded":"2024-09-19T09:25:42.994870+0000","last_fullsized":"2024-09-19T09:25:42.994870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:35.936413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347260+0000","last_change":"2024-09-19T09:25:29.298332+0000","last_active":"2024-09-19T09:27:11.347260+0000","last_peered":"2024-09-19T09:27:11.347260+0000","last_clean":"2024-09-19T09:27:11.347260+0000","last_became_active":"2024-09-19T09:25:29.298006+0000","last_became_peered":"2024-09-19T09:25:29.298006+0000","last_unstale":"2024-09-19T09:27:11.347260+0000","last_undegraded":"2024-09-19T09:27:11.347260+0000","last_fullsized":"2024-09-19T09:27:11.347260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:02.621821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'35","reported_seq":285,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:26.121266+0000","last_change":"2024-09-19T09:25:27.279528+0000","last_active":"2024-09-19T09:27:26.121266+0000","last_peered":"2024-09-19T09:27:26.121266+0000","last_clean":"2024-09-19T09:27:26.121266+0000","last_became_active":"2024-09-19T09:25:27.279149+0000","last_became_peered":"2024-09-19T09:25:27.279149+0000","last_unstale":"2024-09-19T09:27:26.121266+0000","last_undegraded":"2024-09-19T09:27:26.121266+0000","last_fullsized":"2024-09-19T09:27:26.121266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:47.030736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65694,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":228,"num_read_kb":178,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.324455+0000","last_change":"2024-09-19T09:25:29.297538+0000","last_active":"2024-09-19T09:27:11.324455+0000","last_peered":"2024-09-19T09:27:11.324455+0000","last_clean":"2024-09-19T09:27:11.324455+0000","last_became_active":"2024-09-19T09:25:29.297181+0000","last_became_peered":"2024-09-19T09:25:29.297181+0000","last_unstale":"2024-09-19T09:27:11.324455+0000","last_undegraded":"2024-09-19T09:27:11.324455+0000","last_fullsized":"2024-09-19T09:27:11.324455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:53.135438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992860+0000","last_change":"2024-09-19T09:25:27.277090+0000","last_active":"2024-09-19T09:25:42.992860+0000","last_peered":"2024-09-19T09:25:42.992860+0000","last_clean":"2024-09-19T09:25:42.992860+0000","last_became_active":"2024-09-19T09:25:27.276718+0000","last_became_peered":"2024-09-19T09:25:27.276718+0000","last_unstale":"2024-09-19T09:25:42.992860+0000","last_undegraded":"2024-09-19T09:25:42.992860+0000","last_fullsized":"2024-09-19T09:25:42.992860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:25.664261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.325840+0000","last_change":"2024-09-19T09:25:29.294393+0000","last_active":"2024-09-19T09:27:11.325840+0000","last_peered":"2024-09-19T09:27:11.325840+0000","last_clean":"2024-09-19T09:27:11.325840+0000","last_became_active":"2024-09-19T09:25:29.294099+0000","last_became_peered":"2024-09-19T09:25:29.294099+0000","last_unstale":"2024-09-19T09:27:11.325840+0000","last_undegraded":"2024-09-19T09:27:11.325840+0000","last_fullsized":"2024-09-19T09:27:11.325840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:17.333736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995398+0000","last_change":"2024-09-19T09:25:27.278593+0000","last_active":"2024-09-19T09:25:42.995398+0000","last_peered":"2024-09-19T09:25:42.995398+0000","last_clean":"2024-09-19T09:25:42.995398+0000","last_became_active":"2024-09-19T09:25:27.278232+0000","last_became_peered":"2024-09-19T09:25:27.278232+0000","last_unstale":"2024-09-19T09:25:42.995398+0000","last_undegraded":"2024-09-19T09:25:42.995398+0000","last_fullsized":"2024-09-19T09:25:42.995398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:05.852671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343581+0000","last_change":"2024-09-19T09:25:29.300024+0000","last_active":"2024-09-19T09:27:11.343581+0000","last_peered":"2024-09-19T09:27:11.343581+0000","last_clean":"2024-09-19T09:27:11.343581+0000","last_became_active":"2024-09-19T09:25:29.299683+0000","last_became_peered":"2024-09-19T09:25:29.299683+0000","last_unstale":"2024-09-19T09:27:11.343581+0000","last_undegraded":"2024-09-19T09:27:11.343581+0000","last_fullsized":"2024-09-19T09:27:11.343581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:42.249399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'7","reported_seq":141,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:25.120627+0000","last_change":"2024-09-19T09:25:27.272739+0000","last_active":"2024-09-19T09:27:25.120627+0000","last_peered":"2024-09-19T09:27:25.120627+0000","last_clean":"2024-09-19T09:27:25.120627+0000","last_became_active":"2024-09-19T09:25:27.272354+0000","last_became_peered":"2024-09-19T09:25:27.272354+0000","last_unstale":"2024-09-19T09:27:25.120627+0000","last_undegraded":"2024-09-19T09:27:25.120627+0000","last_fullsized":"2024-09-19T09:27:25.120627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:27.863840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343011+0000","last_change":"2024-09-19T09:25:29.303460+0000","last_active":"2024-09-19T09:27:11.343011+0000","last_peered":"2024-09-19T09:27:11.343011+0000","last_clean":"2024-09-19T09:27:11.343011+0000","last_became_active":"2024-09-19T09:25:29.303127+0000","last_became_peered":"2024-09-19T09:25:29.303127+0000","last_unstale":"2024-09-19T09:27:11.343011+0000","last_undegraded":"2024-09-19T09:27:11.343011+0000","last_fullsized":"2024-09-19T09:27:11.343011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:35.313924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990876+0000","last_change":"2024-09-19T09:25:27.323238+0000","last_active":"2024-09-19T09:25:42.990876+0000","last_peered":"2024-09-19T09:25:42.990876+0000","last_clean":"2024-09-19T09:25:42.990876+0000","last_became_active":"2024-09-19T09:25:27.322868+0000","last_became_peered":"2024-09-19T09:25:27.322868+0000","last_unstale":"2024-09-19T09:25:42.990876+0000","last_undegraded":"2024-09-19T09:25:42.990876+0000","last_fullsized":"2024-09-19T09:25:42.990876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:22.890280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338463+0000","last_change":"2024-09-19T09:25:43.005569+0000","last_active":"2024-09-19T09:27:11.338463+0000","last_peered":"2024-09-19T09:27:11.338463+0000","last_clean":"2024-09-19T09:27:11.338463+0000","last_became_active":"2024-09-19T09:25:43.005143+0000","last_became_peered":"2024-09-19T09:25:43.005143+0000","last_unstale":"2024-09-19T09:27:11.338463+0000","last_undegraded":"2024-09-19T09:27:11.338463+0000","last_fullsized":"2024-09-19T09:27:11.338463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:26.297013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991438+0000","last_change":"2024-09-19T09:25:27.282737+0000","last_active":"2024-09-19T09:25:42.991438+0000","last_peered":"2024-09-19T09:25:42.991438+0000","last_clean":"2024-09-19T09:25:42.991438+0000","last_became_active":"2024-09-19T09:25:27.281785+0000","last_became_peered":"2024-09-19T09:25:27.281785+0000","last_unstale":"2024-09-19T09:25:42.991438+0000","last_undegraded":"2024-09-19T09:25:42.991438+0000","last_fullsized":"2024-09-19T09:25:42.991438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:41.552206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.340095+0000","last_change":"2024-09-19T09:25:29.300168+0000","last_active":"2024-09-19T09:27:11.340095+0000","last_peered":"2024-09-19T09:27:11.340095+0000","last_clean":"2024-09-19T09:27:11.340095+0000","last_became_active":"2024-09-19T09:25:29.299811+0000","last_became_peered":"2024-09-19T09:25:29.299811+0000","last_unstale":"2024-09-19T09:27:11.340095+0000","last_undegraded":"2024-09-19T09:27:11.340095+0000","last_fullsized":"2024-09-19T09:27:11.340095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:06.691259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985783+0000","last_change":"2024-09-19T09:25:27.278433+0000","last_active":"2024-09-19T09:25:42.985783+0000","last_peered":"2024-09-19T09:25:42.985783+0000","last_clean":"2024-09-19T09:25:42.985783+0000","last_became_active":"2024-09-19T09:25:27.278076+0000","last_became_peered":"2024-09-19T09:25:27.278076+0000","last_unstale":"2024-09-19T09:25:42.985783+0000","last_undegraded":"2024-09-19T09:25:42.985783+0000","last_fullsized":"2024-09-19T09:25:42.985783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:53.799265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338515+0000","last_change":"2024-09-19T09:25:29.295650+0000","last_active":"2024-09-19T09:27:11.338515+0000","last_peered":"2024-09-19T09:27:11.338515+0000","last_clean":"2024-09-19T09:27:11.338515+0000","last_became_active":"2024-09-19T09:25:29.295277+0000","last_became_peered":"2024-09-19T09:25:29.295277+0000","last_unstale":"2024-09-19T09:27:11.338515+0000","last_undegraded":"2024-09-19T09:27:11.338515+0000","last_fullsized":"2024-09-19T09:27:11.338515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:57.690699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996123+0000","last_change":"2024-09-19T09:25:27.279239+0000","last_active":"2024-09-19T09:25:42.996123+0000","last_peered":"2024-09-19T09:25:42.996123+0000","last_clean":"2024-09-19T09:25:42.996123+0000","last_became_active":"2024-09-19T09:25:27.278868+0000","last_became_peered":"2024-09-19T09:25:27.278868+0000","last_unstale":"2024-09-19T09:25:42.996123+0000","last_undegraded":"2024-09-19T09:25:42.996123+0000","last_fullsized":"2024-09-19T09:25:42.996123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:41.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341911+0000","last_change":"2024-09-19T09:25:29.300833+0000","last_active":"2024-09-19T09:27:11.341911+0000","last_peered":"2024-09-19T09:27:11.341911+0000","last_clean":"2024-09-19T09:27:11.341911+0000","last_became_active":"2024-09-19T09:25:29.300472+0000","last_became_peered":"2024-09-19T09:25:29.300472+0000","last_unstale":"2024-09-19T09:27:11.341911+0000","last_undegraded":"2024-09-19T09:27:11.341911+0000","last_fullsized":"2024-09-19T09:27:11.341911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:50.591276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'8","reported_seq":129,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:25.156716+0000","last_change":"2024-09-19T09:25:43.001722+0000","last_active":"2024-09-19T09:27:25.156716+0000","last_peered":"2024-09-19T09:27:25.156716+0000","last_clean":"2024-09-19T09:27:25.156716+0000","last_became_active":"2024-09-19T09:25:42.992796+0000","last_became_peered":"2024-09-19T09:25:42.992796+0000","last_unstale":"2024-09-19T09:27:25.156716+0000","last_undegraded":"2024-09-19T09:27:25.156716+0000","last_fullsized":"2024-09-19T09:27:25.156716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:59.639010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2012,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346495+0000","last_change":"2024-09-19T09:25:29.295906+0000","last_active":"2024-09-19T09:27:11.346495+0000","last_peered":"2024-09-19T09:27:11.346495+0000","last_clean":"2024-09-19T09:27:11.346495+0000","last_became_active":"2024-09-19T09:25:29.295565+0000","last_became_peered":"2024-09-19T09:25:29.295565+0000","last_unstale":"2024-09-19T09:27:11.346495+0000","last_undegraded":"2024-09-19T09:27:11.346495+0000","last_fullsized":"2024-09-19T09:27:11.346495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:24.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989907+0000","last_change":"2024-09-19T09:24:19.256300+0000","last_active":"2024-09-19T09:25:42.989907+0000","last_peered":"2024-09-19T09:25:42.989907+0000","last_clean":"2024-09-19T09:25:42.989907+0000","last_became_active":"2024-09-19T09:24:19.255833+0000","last_became_peered":"2024-09-19T09:24:19.255833+0000","last_unstale":"2024-09-19T09:25:42.989907+0000","last_undegraded":"2024-09-19T09:25:42.989907+0000","last_fullsized":"2024-09-19T09:25:42.989907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_clean_scrub_stamp":"2024-09-19T09:20:57.798838+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:14:33.343748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989637+0000","last_change":"2024-09-19T09:25:27.272090+0000","last_active":"2024-09-19T09:25:42.989637+0000","last_peered":"2024-09-19T09:25:42.989637+0000","last_clean":"2024-09-19T09:25:42.989637+0000","last_became_active":"2024-09-19T09:25:27.271651+0000","last_became_peered":"2024-09-19T09:25:27.271651+0000","last_unstale":"2024-09-19T09:25:42.989637+0000","last_undegraded":"2024-09-19T09:25:42.989637+0000","last_fullsized":"2024-09-19T09:25:42.989637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:45.847082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344375+0000","last_change":"2024-09-19T09:25:29.300718+0000","last_active":"2024-09-19T09:27:11.344375+0000","last_peered":"2024-09-19T09:27:11.344375+0000","last_clean":"2024-09-19T09:27:11.344375+0000","last_became_active":"2024-09-19T09:25:29.299460+0000","last_became_peered":"2024-09-19T09:25:29.299460+0000","last_unstale":"2024-09-19T09:27:11.344375+0000","last_undegraded":"2024-09-19T09:27:11.344375+0000","last_fullsized":"2024-09-19T09:27:11.344375+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:43.063909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003919+0000","last_change":"2024-09-19T09:25:43.003299+0000","last_active":"2024-09-19T09:25:43.003919+0000","last_peered":"2024-09-19T09:25:43.003919+0000","last_clean":"2024-09-19T09:25:43.003919+0000","last_became_active":"2024-09-19T09:25:42.999686+0000","last_became_peered":"2024-09-19T09:25:42.999686+0000","last_unstale":"2024-09-19T09:25:43.003919+0000","last_undegraded":"2024-09-19T09:25:43.003919+0000","last_fullsized":"2024-09-19T09:25:43.003919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:05.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337734+0000","last_change":"2024-09-19T09:25:29.297833+0000","last_active":"2024-09-19T09:27:11.337734+0000","last_peered":"2024-09-19T09:27:11.337734+0000","last_clean":"2024-09-19T09:27:11.337734+0000","last_became_active":"2024-09-19T09:25:29.297534+0000","last_became_peered":"2024-09-19T09:25:29.297534+0000","last_unstale":"2024-09-19T09:27:11.337734+0000","last_undegraded":"2024-09-19T09:27:11.337734+0000","last_fullsized":"2024-09-19T09:27:11.337734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:47.156253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":134,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:22.116052+0000","last_change":"2024-09-19T09:25:27.322759+0000","last_active":"2024-09-19T09:27:22.116052+0000","last_peered":"2024-09-19T09:27:22.116052+0000","last_clean":"2024-09-19T09:27:22.116052+0000","last_became_active":"2024-09-19T09:25:27.322402+0000","last_became_peered":"2024-09-19T09:25:27.322402+0000","last_unstale":"2024-09-19T09:27:22.116052+0000","last_undegraded":"2024-09-19T09:27:22.116052+0000","last_fullsized":"2024-09-19T09:27:22.116052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:06.276957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335476+0000","last_change":"2024-09-19T09:25:42.999834+0000","last_active":"2024-09-19T09:27:11.335476+0000","last_peered":"2024-09-19T09:27:11.335476+0000","last_clean":"2024-09-19T09:27:11.335476+0000","last_became_active":"2024-09-19T09:25:42.999450+0000","last_became_peered":"2024-09-19T09:25:42.999450+0000","last_unstale":"2024-09-19T09:27:11.335476+0000","last_undegraded":"2024-09-19T09:27:11.335476+0000","last_fullsized":"2024-09-19T09:27:11.335476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:00.509844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'7","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:26.129531+0000","last_change":"2024-09-19T09:25:27.275212+0000","last_active":"2024-09-19T09:27:26.129531+0000","last_peered":"2024-09-19T09:27:26.129531+0000","last_clean":"2024-09-19T09:27:26.129531+0000","last_became_active":"2024-09-19T09:25:27.274802+0000","last_became_peered":"2024-09-19T09:25:27.274802+0000","last_unstale":"2024-09-19T09:27:26.129531+0000","last_undegraded":"2024-09-19T09:27:26.129531+0000","last_fullsized":"2024-09-19T09:27:26.129531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:48.392026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.333364+0000","last_change":"2024-09-19T09:25:29.304747+0000","last_active":"2024-09-19T09:27:11.333364+0000","last_peered":"2024-09-19T09:27:11.333364+0000","last_clean":"2024-09-19T09:27:11.333364+0000","last_became_active":"2024-09-19T09:25:29.304386+0000","last_became_peered":"2024-09-19T09:25:29.304386+0000","last_unstale":"2024-09-19T09:27:11.333364+0000","last_undegraded":"2024-09-19T09:27:11.333364+0000","last_fullsized":"2024-09-19T09:27:11.333364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.788697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988854+0000","last_change":"2024-09-19T09:25:27.323083+0000","last_active":"2024-09-19T09:25:42.988854+0000","last_peered":"2024-09-19T09:25:42.988854+0000","last_clean":"2024-09-19T09:25:42.988854+0000","last_became_active":"2024-09-19T09:25:27.322707+0000","last_became_peered":"2024-09-19T09:25:27.322707+0000","last_unstale":"2024-09-19T09:25:42.988854+0000","last_undegraded":"2024-09-19T09:25:42.988854+0000","last_fullsized":"2024-09-19T09:25:42.988854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:24.415116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334132+0000","last_change":"2024-09-19T09:25:29.292472+0000","last_active":"2024-09-19T09:27:11.334132+0000","last_peered":"2024-09-19T09:27:11.334132+0000","last_clean":"2024-09-19T09:27:11.334132+0000","last_became_active":"2024-09-19T09:25:29.292079+0000","last_became_peered":"2024-09-19T09:25:29.292079+0000","last_unstale":"2024-09-19T09:27:11.334132+0000","last_undegraded":"2024-09-19T09:27:11.334132+0000","last_fullsized":"2024-09-19T09:27:11.334132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:35.801594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991224+0000","last_change":"2024-09-19T09:25:27.324839+0000","last_active":"2024-09-19T09:25:42.991224+0000","last_peered":"2024-09-19T09:25:42.991224+0000","last_clean":"2024-09-19T09:25:42.991224+0000","last_became_active":"2024-09-19T09:25:27.324475+0000","last_became_peered":"2024-09-19T09:25:27.324475+0000","last_unstale":"2024-09-19T09:25:42.991224+0000","last_undegraded":"2024-09-19T09:25:42.991224+0000","last_fullsized":"2024-09-19T09:25:42.991224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:32.251753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338007+0000","last_change":"2024-09-19T09:25:29.293025+0000","last_active":"2024-09-19T09:27:11.338007+0000","last_peered":"2024-09-19T09:27:11.338007+0000","last_clean":"2024-09-19T09:27:11.338007+0000","last_became_active":"2024-09-19T09:25:29.292570+0000","last_became_peered":"2024-09-19T09:25:29.292570+0000","last_unstale":"2024-09-19T09:27:11.338007+0000","last_undegraded":"2024-09-19T09:27:11.338007+0000","last_fullsized":"2024-09-19T09:27:11.338007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:16.394524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'17","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.329192+0000","last_change":"2024-09-19T09:25:27.278100+0000","last_active":"2024-09-19T09:27:11.329192+0000","last_peered":"2024-09-19T09:27:11.329192+0000","last_clean":"2024-09-19T09:27:11.329192+0000","last_became_active":"2024-09-19T09:25:27.277769+0000","last_became_peered":"2024-09-19T09:25:27.277769+0000","last_unstale":"2024-09-19T09:27:11.329192+0000","last_undegraded":"2024-09-19T09:27:11.329192+0000","last_fullsized":"2024-09-19T09:27:11.329192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:31.885142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338230+0000","last_change":"2024-09-19T09:25:29.293651+0000","last_active":"2024-09-19T09:27:11.338230+0000","last_peered":"2024-09-19T09:27:11.338230+0000","last_clean":"2024-09-19T09:27:11.338230+0000","last_became_active":"2024-09-19T09:25:29.293317+0000","last_became_peered":"2024-09-19T09:25:29.293317+0000","last_unstale":"2024-09-19T09:27:11.338230+0000","last_undegraded":"2024-09-19T09:27:11.338230+0000","last_fullsized":"2024-09-19T09:27:11.338230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:40.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990030+0000","last_change":"2024-09-19T09:25:27.269367+0000","last_active":"2024-09-19T09:25:42.990030+0000","last_peered":"2024-09-19T09:25:42.990030+0000","last_clean":"2024-09-19T09:25:42.990030+0000","last_became_active":"2024-09-19T09:25:27.269011+0000","last_became_peered":"2024-09-19T09:25:27.269011+0000","last_unstale":"2024-09-19T09:25:42.990030+0000","last_undegraded":"2024-09-19T09:25:42.990030+0000","last_fullsized":"2024-09-19T09:25:42.990030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:00.930752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334968+0000","last_change":"2024-09-19T09:25:29.299134+0000","last_active":"2024-09-19T09:27:11.334968+0000","last_peered":"2024-09-19T09:27:11.334968+0000","last_clean":"2024-09-19T09:27:11.334968+0000","last_became_active":"2024-09-19T09:25:29.298648+0000","last_became_peered":"2024-09-19T09:25:29.298648+0000","last_unstale":"2024-09-19T09:27:11.334968+0000","last_undegraded":"2024-09-19T09:27:11.334968+0000","last_fullsized":"2024-09-19T09:27:11.334968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:39.627852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003686+0000","last_change":"2024-09-19T09:25:27.322861+0000","last_active":"2024-09-19T09:25:43.003686+0000","last_peered":"2024-09-19T09:25:43.003686+0000","last_clean":"2024-09-19T09:25:43.003686+0000","last_became_active":"2024-09-19T09:25:27.322560+0000","last_became_peered":"2024-09-19T09:25:27.322560+0000","last_unstale":"2024-09-19T09:25:43.003686+0000","last_undegraded":"2024-09-19T09:25:43.003686+0000","last_fullsized":"2024-09-19T09:25:43.003686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:35.993093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.301522+0000","last_change":"2024-09-19T09:25:29.301680+0000","last_active":"2024-09-19T09:27:11.301522+0000","last_peered":"2024-09-19T09:27:11.301522+0000","last_clean":"2024-09-19T09:27:11.301522+0000","last_became_active":"2024-09-19T09:25:29.301331+0000","last_became_peered":"2024-09-19T09:25:29.301331+0000","last_unstale":"2024-09-19T09:27:11.301522+0000","last_undegraded":"2024-09-19T09:27:11.301522+0000","last_fullsized":"2024-09-19T09:27:11.301522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:57.364245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:27.272609+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:27.272169+0000","last_became_peered":"2024-09-19T09:25:27.272169+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.879474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341200+0000","last_change":"2024-09-19T09:25:29.296847+0000","last_active":"2024-09-19T09:27:11.341200+0000","last_peered":"2024-09-19T09:27:11.341200+0000","last_clean":"2024-09-19T09:27:11.341200+0000","last_became_active":"2024-09-19T09:25:29.296465+0000","last_became_peered":"2024-09-19T09:25:29.296465+0000","last_unstale":"2024-09-19T09:27:11.341200+0000","last_undegraded":"2024-09-19T09:27:11.341200+0000","last_fullsized":"2024-09-19T09:27:11.341200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:05:48.553252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.012954+0000","last_change":"2024-09-19T09:25:43.012532+0000","last_active":"2024-09-19T09:25:43.012954+0000","last_peered":"2024-09-19T09:25:43.012954+0000","last_clean":"2024-09-19T09:25:43.012954+0000","last_became_active":"2024-09-19T09:25:42.994996+0000","last_became_peered":"2024-09-19T09:25:42.994996+0000","last_unstale":"2024-09-19T09:25:43.012954+0000","last_undegraded":"2024-09-19T09:25:43.012954+0000","last_fullsized":"2024-09-19T09:25:43.012954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:25.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337187+0000","last_change":"2024-09-19T09:25:29.297633+0000","last_active":"2024-09-19T09:27:11.337187+0000","last_peered":"2024-09-19T09:27:11.337187+0000","last_clean":"2024-09-19T09:27:11.337187+0000","last_became_active":"2024-09-19T09:25:29.297284+0000","last_became_peered":"2024-09-19T09:25:29.297284+0000","last_unstale":"2024-09-19T09:27:11.337187+0000","last_undegraded":"2024-09-19T09:27:11.337187+0000","last_fullsized":"2024-09-19T09:27:11.337187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:50.582546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991172+0000","last_change":"2024-09-19T09:25:27.281778+0000","last_active":"2024-09-19T09:25:42.991172+0000","last_peered":"2024-09-19T09:25:42.991172+0000","last_clean":"2024-09-19T09:25:42.991172+0000","last_became_active":"2024-09-19T09:25:27.281280+0000","last_became_peered":"2024-09-19T09:25:27.281280+0000","last_unstale":"2024-09-19T09:25:42.991172+0000","last_undegraded":"2024-09-19T09:25:42.991172+0000","last_fullsized":"2024-09-19T09:25:42.991172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:46.811968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343342+0000","last_change":"2024-09-19T09:25:29.295663+0000","last_active":"2024-09-19T09:27:11.343342+0000","last_peered":"2024-09-19T09:27:11.343342+0000","last_clean":"2024-09-19T09:27:11.343342+0000","last_became_active":"2024-09-19T09:25:29.295146+0000","last_became_peered":"2024-09-19T09:25:29.295146+0000","last_unstale":"2024-09-19T09:27:11.343342+0000","last_undegraded":"2024-09-19T09:27:11.343342+0000","last_fullsized":"2024-09-19T09:27:11.343342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:36.749339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.345435+0000","last_change":"2024-09-19T09:25:29.298004+0000","last_active":"2024-09-19T09:27:11.345435+0000","last_peered":"2024-09-19T09:27:11.345435+0000","last_clean":"2024-09-19T09:27:11.345435+0000","last_became_active":"2024-09-19T09:25:29.297600+0000","last_became_peered":"2024-09-19T09:25:29.297600+0000","last_unstale":"2024-09-19T09:27:11.345435+0000","last_undegraded":"2024-09-19T09:27:11.345435+0000","last_fullsized":"2024-09-19T09:27:11.345435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:09.738615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992323+0000","last_change":"2024-09-19T09:25:27.282876+0000","last_active":"2024-09-19T09:25:42.992323+0000","last_peered":"2024-09-19T09:25:42.992323+0000","last_clean":"2024-09-19T09:25:42.992323+0000","last_became_active":"2024-09-19T09:25:27.282201+0000","last_became_peered":"2024-09-19T09:25:27.282201+0000","last_unstale":"2024-09-19T09:25:42.992323+0000","last_undegraded":"2024-09-19T09:25:42.992323+0000","last_fullsized":"2024-09-19T09:25:42.992323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:08.508662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347692+0000","last_change":"2024-09-19T09:25:29.288379+0000","last_active":"2024-09-19T09:27:11.347692+0000","last_peered":"2024-09-19T09:27:11.347692+0000","last_clean":"2024-09-19T09:27:11.347692+0000","last_became_active":"2024-09-19T09:25:29.287917+0000","last_became_peered":"2024-09-19T09:25:29.287917+0000","last_unstale":"2024-09-19T09:27:11.347692+0000","last_undegraded":"2024-09-19T09:27:11.347692+0000","last_fullsized":"2024-09-19T09:27:11.347692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:03.628233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991076+0000","last_change":"2024-09-19T09:25:27.279551+0000","last_active":"2024-09-19T09:25:42.991076+0000","last_peered":"2024-09-19T09:25:42.991076+0000","last_clean":"2024-09-19T09:25:42.991076+0000","last_became_active":"2024-09-19T09:25:27.278956+0000","last_became_peered":"2024-09-19T09:25:27.278956+0000","last_unstale":"2024-09-19T09:25:42.991076+0000","last_undegraded":"2024-09-19T09:25:42.991076+0000","last_fullsized":"2024-09-19T09:25:42.991076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:17.186453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347269+0000","last_change":"2024-09-19T09:25:29.288657+0000","last_active":"2024-09-19T09:27:11.347269+0000","last_peered":"2024-09-19T09:27:11.347269+0000","last_clean":"2024-09-19T09:27:11.347269+0000","last_became_active":"2024-09-19T09:25:29.288355+0000","last_became_peered":"2024-09-19T09:25:29.288355+0000","last_unstale":"2024-09-19T09:27:11.347269+0000","last_undegraded":"2024-09-19T09:27:11.347269+0000","last_fullsized":"2024-09-19T09:27:11.347269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:46.237558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991373+0000","last_change":"2024-09-19T09:25:27.323312+0000","last_active":"2024-09-19T09:25:42.991373+0000","last_peered":"2024-09-19T09:25:42.991373+0000","last_clean":"2024-09-19T09:25:42.991373+0000","last_became_active":"2024-09-19T09:25:27.322984+0000","last_became_peered":"2024-09-19T09:25:27.322984+0000","last_unstale":"2024-09-19T09:25:42.991373+0000","last_undegraded":"2024-09-19T09:25:42.991373+0000","last_fullsized":"2024-09-19T09:25:42.991373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:48.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.327987+0000","last_change":"2024-09-19T09:25:29.301832+0000","last_active":"2024-09-19T09:27:11.327987+0000","last_peered":"2024-09-19T09:27:11.327987+0000","last_clean":"2024-09-19T09:27:11.327987+0000","last_became_active":"2024-09-19T09:25:29.301539+0000","last_became_peered":"2024-09-19T09:25:29.301539+0000","last_unstale":"2024-09-19T09:27:11.327987+0000","last_undegraded":"2024-09-19T09:27:11.327987+0000","last_fullsized":"2024-09-19T09:27:11.327987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:00.244948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'14","reported_seq":159,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:21.122751+0000","last_change":"2024-09-19T09:25:43.009668+0000","last_active":"2024-09-19T09:27:21.122751+0000","last_peered":"2024-09-19T09:27:21.122751+0000","last_clean":"2024-09-19T09:27:21.122751+0000","last_became_active":"2024-09-19T09:25:42.994966+0000","last_became_peered":"2024-09-19T09:25:42.994966+0000","last_unstale":"2024-09-19T09:27:21.122751+0000","last_undegraded":"2024-09-19T09:27:21.122751+0000","last_fullsized":"2024-09-19T09:27:21.122751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:04:05.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74560,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1108,"num_read_kb":635,"num_write":263,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1842,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":223680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612645,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":431506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841578,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1597440,"data_stored":1087249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005101,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":917504,"data_stored":429922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234035,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":430300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462969,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1023357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593789,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":901120,"data_stored":430173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1022639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":432046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":495830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":496780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":431820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:27:29.073 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:27:29.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:28 smithi142 ceph-mon[35032]: pgmap v334: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T09:27:29.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:28 smithi142 ceph-mon[35032]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:29.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:28 smithi097 ceph-mon[34864]: pgmap v334: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T09:27:29.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:28 smithi097 ceph-mon[34864]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.548+0000 7f52e77e6640 1 -- 172.21.15.16:0/4215039045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52e0100100 msgr2=0x7f52e0100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.548+0000 7f52e77e6640 1 --2- 172.21.15.16:0/4215039045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52e0100100 0x7f52e0100560 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009900 tx=0x7f52d002a410 comp rx=0 tx=0).stop 2024-09-19T09:27:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.548+0000 7f52e77e6640 1 -- 172.21.15.16:0/4215039045 shutdown_connections 2024-09-19T09:27:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.548+0000 7f52e77e6640 1 --2- 172.21.15.16:0/4215039045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e0100aa0 0x7f52e010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.548+0000 7f52e77e6640 1 --2- 172.21.15.16:0/4215039045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52e0100100 0x7f52e0100560 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.548+0000 7f52e77e6640 1 --2- 172.21.15.16:0/4215039045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52e0106b80 0x7f52e0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.548+0000 7f52e77e6640 1 -- 172.21.15.16:0/4215039045 >> 172.21.15.16:0/4215039045 conn(0x7f52e0077230 msgr2=0x7f52e00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.548+0000 7f52e77e6640 1 -- 172.21.15.16:0/4215039045 shutdown_connections 2024-09-19T09:27:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.548+0000 7f52e77e6640 1 -- 172.21.15.16:0/4215039045 wait complete. 2024-09-19T09:27:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.549+0000 7f52e77e6640 1 Processor -- start 2024-09-19T09:27:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.549+0000 7f52e77e6640 1 -- start start 2024-09-19T09:27:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.549+0000 7f52e77e6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52e0100100 0x7f52e019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.549+0000 7f52e77e6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e0100aa0 0x7f52e019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.549+0000 7f52e77e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52e0106b80 0x7f52e019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.549+0000 7f52e77e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52e0112050 con 0x7f52e0106b80 2024-09-19T09:27:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.549+0000 7f52e77e6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f52e0111ed0 con 0x7f52e0100aa0 2024-09-19T09:27:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.549+0000 7f52e77e6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01121d0 con 0x7f52e0100100 2024-09-19T09:27:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.549+0000 7f52e5d83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52e0106b80 0x7f52e019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.549+0000 7f52e5d83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52e0106b80 0x7f52e019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42646/0 (socket says 172.21.15.16:42646) 2024-09-19T09:27:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.549+0000 7f52e5d83640 1 -- 172.21.15.16:0/3376241598 learned_addr learned my addr 172.21.15.16:0/3376241598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.549+0000 7f52e4d81640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e0100aa0 0x7f52e019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.550+0000 7f52e5582640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52e0100100 0x7f52e019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.550+0000 7f52e5d83640 1 -- 172.21.15.16:0/3376241598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52e0100100 msgr2=0x7f52e019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.550+0000 7f52e5d83640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52e0100100 0x7f52e019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.550+0000 7f52e5d83640 1 -- 172.21.15.16:0/3376241598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e0100aa0 msgr2=0x7f52e019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.550+0000 7f52e5d83640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e0100aa0 0x7f52e019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.550+0000 7f52e5d83640 1 -- 172.21.15.16:0/3376241598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e019f670 con 0x7f52e0106b80 2024-09-19T09:27:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.550+0000 7f52e5582640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52e0100100 0x7f52e019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.550+0000 7f52e5d83640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52e0106b80 0x7f52e019ef90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f52d800e9d0 tx=0x7f52d800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.550+0000 7f52d67fc640 1 -- 172.21.15.16:0/3376241598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d800cd80 con 0x7f52e0106b80 2024-09-19T09:27:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.550+0000 7f52e77e6640 1 -- 172.21.15.16:0/3376241598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e019f960 con 0x7f52e0106b80 2024-09-19T09:27:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.550+0000 7f52d67fc640 1 -- 172.21.15.16:0/3376241598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52d8005020 con 0x7f52e0106b80 2024-09-19T09:27:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.550+0000 7f52d67fc640 1 -- 172.21.15.16:0/3376241598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d80116d0 con 0x7f52e0106b80 2024-09-19T09:27:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.550+0000 7f52e77e6640 1 -- 172.21.15.16:0/3376241598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e01a7240 con 0x7f52e0106b80 2024-09-19T09:27:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.553+0000 7f52d67fc640 1 -- 172.21.15.16:0/3376241598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f52d80051d0 con 0x7f52e0106b80 2024-09-19T09:27:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.553+0000 7f52e9bbf740 1 Processor -- start 2024-09-19T09:27:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52e9bbf740 1 -- start start 2024-09-19T09:27:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52e9bbf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a54a0 0x560b703a5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52e9bbf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a5dc0 0x560b703b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52e9bbf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703b0680 0x560b703b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52e9bbf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b7030b2e0 con 0x560b703a54a0 2024-09-19T09:27:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52e9bbf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560b702b2ce0 con 0x560b703b0680 2024-09-19T09:27:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52e9bbf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560b703072a0 con 0x560b703a5dc0 2024-09-19T09:27:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52d67fc640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f52cc0a3240 0x7f52cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52e4d81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a54a0 0x560b703a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52e4d81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a54a0 0x560b703a5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42652/0 (socket says 172.21.15.16:42652) 2024-09-19T09:27:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52e4d81640 1 -- 172.21.15.16:0/3776436844 learned_addr learned my addr 172.21.15.16:0/3776436844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52e5d83640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f52cc0a3240 0x7f52cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52e5582640 1 --2- 172.21.15.16:0/3776436844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a5dc0 0x560b703b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52e4d81640 1 --2- 172.21.15.16:0/3776436844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703b0680 0x560b703b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.554+0000 7f52d67fc640 1 -- 172.21.15.16:0/3376241598 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f52d80cff40 con 0x7f52e0106b80 2024-09-19T09:27:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.555+0000 7f52e4d81640 1 -- 172.21.15.16:0/3776436844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a5dc0 msgr2=0x560b703b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.555+0000 7f52e4d81640 1 --2- 172.21.15.16:0/3776436844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a5dc0 0x560b703b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.555+0000 7f52e4d81640 1 -- 172.21.15.16:0/3776436844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703b0680 msgr2=0x560b703b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.555+0000 7f52e4d81640 1 --2- 172.21.15.16:0/3776436844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703b0680 0x560b703b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.555+0000 7f52e4d81640 1 -- 172.21.15.16:0/3776436844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b70297440 con 0x560b703a54a0 2024-09-19T09:27:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.555+0000 7f52e5d83640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f52cc0a3240 0x7f52cc0a5700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f52d8098980 tx=0x7f52d8015b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.555+0000 7f52e4d81640 1 --2- 172.21.15.16:0/3776436844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a54a0 0x560b703a5880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f52d002a930 tx=0x7f52d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7da694fc909df70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.555+0000 7f52cb7fe640 1 -- 172.21.15.16:0/3776436844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0035040 con 0x560b703a54a0 2024-09-19T09:27:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.555+0000 7f52cb7fe640 1 -- 172.21.15.16:0/3776436844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52d0005500 con 0x560b703a54a0 2024-09-19T09:27:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.555+0000 7f52cb7fe640 1 -- 172.21.15.16:0/3776436844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0004d10 con 0x560b703a54a0 2024-09-19T09:27:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.556+0000 7f52e9bbf740 1 -- 172.21.15.16:0/3776436844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a54a0 msgr2=0x560b703a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.556+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/3776436844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a54a0 0x560b703a5880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f52d002a930 tx=0x7f52d0000bc0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.556+0000 7f52e9bbf740 1 -- 172.21.15.16:0/3776436844 shutdown_connections 2024-09-19T09:27:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.556+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/3776436844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a5dc0 0x560b703b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.556+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/3776436844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703b0680 0x560b703b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.556+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/3776436844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a54a0 0x560b703a5880 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.556+0000 7f52e9bbf740 1 -- 172.21.15.16:0/3776436844 >> 172.21.15.16:0/3776436844 conn(0x560b703a1a30 msgr2=0x560b703a1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.556+0000 7f52e9bbf740 1 -- 172.21.15.16:0/3776436844 shutdown_connections 2024-09-19T09:27:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.556+0000 7f52e9bbf740 1 -- 172.21.15.16:0/3776436844 wait complete. 2024-09-19T09:27:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.557+0000 7f52e9bbf740 1 -- start start 2024-09-19T09:27:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.557+0000 7f52e9bbf740 1 Processor -- start 2024-09-19T09:27:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.557+0000 7f52e9bbf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703a54a0 0x560b703affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.557+0000 7f52e9bbf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a5dc0 0x560b703ad280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.557+0000 7f52e9bbf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703b0680 0x560b703ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.557+0000 7f52e9bbf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560b703b3250 con 0x560b703a5dc0 2024-09-19T09:27:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.557+0000 7f52e9bbf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560b702c6050 con 0x560b703a54a0 2024-09-19T09:27:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.557+0000 7f52e9bbf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560b703b33d0 con 0x560b703b0680 2024-09-19T09:27:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.557+0000 7f52e9bbf740 10 client.0 _ll_register_callbacks cb 0x560b70306160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.557+0000 7f52e5582640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a5dc0 0x560b703ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52e5582640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a5dc0 0x560b703ad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42658/0 (socket says 172.21.15.16:42658) 2024-09-19T09:27:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52e5582640 1 -- 172.21.15.16:0/4279931630 learned_addr learned my addr 172.21.15.16:0/4279931630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52e4d81640 1 --2- 172.21.15.16:0/4279931630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703b0680 0x560b703ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52e4d81640 1 --2- 172.21.15.16:0/4279931630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703a54a0 0x560b703affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52e5582640 1 -- 172.21.15.16:0/4279931630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703b0680 msgr2=0x560b703ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52e5582640 1 --2- 172.21.15.16:0/4279931630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703b0680 0x560b703ad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52e5582640 1 -- 172.21.15.16:0/4279931630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703a54a0 msgr2=0x560b703affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52e5582640 1 --2- 172.21.15.16:0/4279931630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703a54a0 0x560b703affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52e5582640 1 -- 172.21.15.16:0/4279931630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b7007c3c0 con 0x560b703a5dc0 2024-09-19T09:27:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52e5582640 1 --2- 172.21.15.16:0/4279931630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a5dc0 0x560b703ad280 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f52dc011850 tx=0x7f52dc011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f396b9608c3c47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52c8ff9640 1 -- 172.21.15.16:0/4279931630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc00fdf0 con 0x560b703a5dc0 2024-09-19T09:27:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52c8ff9640 1 -- 172.21.15.16:0/4279931630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f52dc004020 con 0x560b703a5dc0 2024-09-19T09:27:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.558+0000 7f52e9bbf740 1 -- 172.21.15.16:0/4279931630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52d0005500 con 0x560b703a5dc0 2024-09-19T09:27:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.559+0000 7f52c8ff9640 1 -- 172.21.15.16:0/4279931630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f52dc004220 con 0x560b703a5dc0 2024-09-19T09:27:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.559+0000 7f52e9bbf740 10 client.15285 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.559+0000 7f52e9bbf740 1 -- 172.21.15.16:0/4279931630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b703bec80 con 0x560b703a5dc0 2024-09-19T09:27:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.559+0000 7f52c8ff9640 1 -- 172.21.15.16:0/4279931630 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52d0005500 con 0x560b703a5dc0 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.559+0000 7f52c8ff9640 1 -- 172.21.15.16:0/4279931630 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f52dc022020 con 0x560b703a5dc0 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.559+0000 7f52e9bbf740 10 client.15285 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.559+0000 7f52e9bbf740 10 client.15285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.559+0000 7f52e9bbf740 20 client.15285 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.559+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/4279931630 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560b703c0150 0x560b703c2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.559+0000 7f52e9bbf740 4 client.15285 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.559+0000 7f52e9bbf740 1 -- 172.21.15.16:0/4279931630 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b70318850 con 0x560b703c0150 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.559+0000 7f52e4d81640 1 --2- 172.21.15.16:0/4279931630 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560b703c0150 0x560b703c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.560+0000 7f52c8ff9640 10 client.15285 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.560+0000 7f52e4d81640 1 --2- 172.21.15.16:0/4279931630 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560b703c0150 0x560b703c2590 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d6409fdb00943d1 server_cookie=aa1e2b129cc5a689 in_seq=0 out_seq=0 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52c8ff9640 1 -- 172.21.15.16:0/4279931630 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b70318850 con 0x560b703c0150 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52c8ff9640 10 client.15285 handle_command_reply: tid=0 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52e9bbf740 1 client.15285 shutdown 2024-09-19T09:27:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52e9bbf740 1 -- 172.21.15.16:0/4279931630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a5dc0 msgr2=0x560b703ad280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/4279931630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a5dc0 0x560b703ad280 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f52dc011850 tx=0x7f52dc011d10 comp rx=0 tx=0).stop 2024-09-19T09:27:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52e9bbf740 1 -- 172.21.15.16:0/4279931630 shutdown_connections 2024-09-19T09:27:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/4279931630 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560b703c0150 0x560b703c2590 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/4279931630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703b0680 0x560b703ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52c8ff9640 0 client.15285 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/4279931630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a5dc0 0x560b703ad280 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/4279931630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703a54a0 0x560b703affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52e9bbf740 1 -- 172.21.15.16:0/4279931630 >> 172.21.15.16:0/4279931630 conn(0x560b703a1a30 msgr2=0x560b703a7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52e9bbf740 1 -- 172.21.15.16:0/4279931630 shutdown_connections 2024-09-19T09:27:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52e9bbf740 1 -- 172.21.15.16:0/4279931630 wait complete. 2024-09-19T09:27:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.561+0000 7f52e9bbf740 20 client.15285 trim_cache size 0 max 16384 2024-09-19T09:27:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.582+0000 7f52e9bbf740 1 Processor -- start 2024-09-19T09:27:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.582+0000 7f52e9bbf740 1 -- start start 2024-09-19T09:27:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.582+0000 7f52e9bbf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a54a0 0x560b703bc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.582+0000 7f52e9bbf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703a5dc0 0x560b703c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.582+0000 7f52e9bbf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703b0680 0x560b703c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.582+0000 7f52e9bbf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01af730 con 0x560b703b0680 2024-09-19T09:27:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.582+0000 7f52e9bbf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01b1270 con 0x560b703a5dc0 2024-09-19T09:27:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.582+0000 7f52e9bbf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01afe00 con 0x560b703a54a0 2024-09-19T09:27:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.583+0000 7f52e4d81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703b0680 0x560b703c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.583+0000 7f52e4d81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703b0680 0x560b703c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42668/0 (socket says 172.21.15.16:42668) 2024-09-19T09:27:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.583+0000 7f52e4d81640 1 -- 172.21.15.16:0/3943969331 learned_addr learned my addr 172.21.15.16:0/3943969331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.583+0000 7f52e4d81640 1 --2- 172.21.15.16:0/3943969331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a54a0 0x560b703bc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.583+0000 7f52e5582640 1 --2- 172.21.15.16:0/3943969331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703a5dc0 0x560b703c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.583+0000 7f52e4d81640 1 -- 172.21.15.16:0/3943969331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a54a0 msgr2=0x560b703bc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.583+0000 7f52e4d81640 1 --2- 172.21.15.16:0/3943969331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a54a0 0x560b703bc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.583+0000 7f52e4d81640 1 -- 172.21.15.16:0/3943969331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703a5dc0 msgr2=0x560b703c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.583+0000 7f52e4d81640 1 --2- 172.21.15.16:0/3943969331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703a5dc0 0x560b703c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.583+0000 7f52e4d81640 1 -- 172.21.15.16:0/3943969331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b703b0340 con 0x560b703b0680 2024-09-19T09:27:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.583+0000 7f52e4d81640 1 --2- 172.21.15.16:0/3943969331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703b0680 0x560b703c83a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f52d0031770 tx=0x7f52d0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103927b06fd51cd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.584+0000 7f52caffd640 1 -- 172.21.15.16:0/3943969331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0035040 con 0x560b703b0680 2024-09-19T09:27:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.584+0000 7f52caffd640 1 -- 172.21.15.16:0/3943969331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52d005d020 con 0x560b703b0680 2024-09-19T09:27:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.584+0000 7f52caffd640 1 -- 172.21.15.16:0/3943969331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0003ad0 con 0x560b703b0680 2024-09-19T09:27:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.584+0000 7f52e9bbf740 1 -- 172.21.15.16:0/3943969331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703b0680 msgr2=0x560b703c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.584+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/3943969331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703b0680 0x560b703c83a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f52d0031770 tx=0x7f52d0005fd0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.584+0000 7f52e9bbf740 1 -- 172.21.15.16:0/3943969331 shutdown_connections 2024-09-19T09:27:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.584+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/3943969331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703b0680 0x560b703c83a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.584+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/3943969331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703a5dc0 0x560b703c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.585+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/3943969331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a54a0 0x560b703bc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.585+0000 7f52e9bbf740 1 -- 172.21.15.16:0/3943969331 >> 172.21.15.16:0/3943969331 conn(0x560b703a1a30 msgr2=0x560b703a8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.585+0000 7f52e9bbf740 1 -- 172.21.15.16:0/3943969331 shutdown_connections 2024-09-19T09:27:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.585+0000 7f52e9bbf740 1 -- 172.21.15.16:0/3943969331 wait complete. 2024-09-19T09:27:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.585+0000 7f52e9bbf740 1 -- start start 2024-09-19T09:27:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.585+0000 7f52e9bbf740 1 Processor -- start 2024-09-19T09:27:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.586+0000 7f52e9bbf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a54a0 0x560b703b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.586+0000 7f52e9bbf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a5dc0 0x560b703b8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.586+0000 7f52e9bbf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703b0680 0x560b703b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.586+0000 7f52e9bbf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01b0ba0 con 0x560b703a54a0 2024-09-19T09:27:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.586+0000 7f52e9bbf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01b04d0 con 0x560b703b0680 2024-09-19T09:27:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.586+0000 7f52e9bbf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01ad6f0 con 0x560b703a5dc0 2024-09-19T09:27:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.586+0000 7f52e9bbf740 10 client.0 _ll_register_callbacks cb 0x7f52dc00c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.586+0000 7f52e4d81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a54a0 0x560b703b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.586+0000 7f52e4d81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a54a0 0x560b703b8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42684/0 (socket says 172.21.15.16:42684) 2024-09-19T09:27:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.586+0000 7f52e4d81640 1 -- 172.21.15.16:0/2217796711 learned_addr learned my addr 172.21.15.16:0/2217796711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.586+0000 7f52e5582640 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a5dc0 0x560b703b8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.586+0000 7f52e4d81640 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703b0680 0x560b703b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52e5582640 1 -- 172.21.15.16:0/2217796711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703b0680 msgr2=0x560b703b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52e5582640 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703b0680 0x560b703b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52e5582640 1 -- 172.21.15.16:0/2217796711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a54a0 msgr2=0x560b703b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52e5582640 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a54a0 0x560b703b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52e5582640 1 -- 172.21.15.16:0/2217796711 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b703b4520 con 0x560b703a5dc0 2024-09-19T09:27:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52e4d81640 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703b0680 0x560b703b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52e5582640 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a5dc0 0x560b703b8e70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f52dc000c00 tx=0x7f52dc00fe20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df2f54a93c04aaeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-19T09:27:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52c8ff9640 1 -- 172.21.15.16:0/2217796711 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52dc011230 con 0x560b703a5dc0 2024-09-19T09:27:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52c8ff9640 1 -- 172.21.15.16:0/2217796711 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f52dc005040 con 0x560b703a5dc0 2024-09-19T09:27:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52e9bbf740 1 -- 172.21.15.16:0/2217796711 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52d005d020 con 0x560b703a5dc0 2024-09-19T09:27:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52c8ff9640 1 -- 172.21.15.16:0/2217796711 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f52dc0113d0 con 0x560b703a5dc0 2024-09-19T09:27:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52e9bbf740 10 client.24940 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.587+0000 7f52e9bbf740 1 -- 172.21.15.16:0/2217796711 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b7042aa00 con 0x560b703a5dc0 2024-09-19T09:27:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.588+0000 7f52c8ff9640 1 -- 172.21.15.16:0/2217796711 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52dc02b040 con 0x560b703a5dc0 2024-09-19T09:27:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.588+0000 7f52c8ff9640 1 -- 172.21.15.16:0/2217796711 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f52dc02e070 con 0x560b703a5dc0 2024-09-19T09:27:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.588+0000 7f52e9bbf740 10 client.24940 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.588+0000 7f52e9bbf740 10 client.24940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.588+0000 7f52e9bbf740 20 client.24940 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.588+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560b7042b1b0 0x560b7042d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.588+0000 7f52e9bbf740 4 client.24940 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:27:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.588+0000 7f52e9bbf740 1 -- 172.21.15.16:0/2217796711 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f52d005d020 con 0x560b7042b1b0 2024-09-19T09:27:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.588+0000 7f52e4d81640 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560b7042b1b0 0x560b7042d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.589+0000 7f52c8ff9640 10 client.24940 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.589+0000 7f52e4d81640 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560b7042b1b0 0x560b7042d5d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d7308cb60c213c7 server_cookie=c80f913bcbd4cb66 in_seq=0 out_seq=0 2024-09-19T09:27:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.589+0000 7f52c8ff9640 1 -- 172.21.15.16:0/2217796711 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f52d005d020 con 0x560b7042b1b0 2024-09-19T09:27:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.589+0000 7f52c8ff9640 10 client.24940 handle_command_reply: tid=0 2024-09-19T09:27:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.589+0000 7f52e9bbf740 1 client.24940 shutdown 2024-09-19T09:27:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.589+0000 7f52e9bbf740 1 -- 172.21.15.16:0/2217796711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a5dc0 msgr2=0x560b703b8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.589+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a5dc0 0x560b703b8e70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f52dc000c00 tx=0x7f52dc00fe20 comp rx=0 tx=0).stop 2024-09-19T09:27:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.589+0000 7f52e9bbf740 1 -- 172.21.15.16:0/2217796711 shutdown_connections 2024-09-19T09:27:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.589+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560b7042b1b0 0x560b7042d5d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.589+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560b703a5dc0 0x560b703b8e70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.589+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560b703b0680 0x560b703b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.589+0000 7f52e9bbf740 1 --2- 172.21.15.16:0/2217796711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560b703a54a0 0x560b703b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.589+0000 7f52e9bbf740 1 -- 172.21.15.16:0/2217796711 >> 172.21.15.16:0/2217796711 conn(0x560b703a1a30 msgr2=0x560b703a0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:29.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52c8ff9640 0 client.24940 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:29.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52e9bbf740 1 -- 172.21.15.16:0/2217796711 shutdown_connections 2024-09-19T09:27:29.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52e9bbf740 1 -- 172.21.15.16:0/2217796711 wait complete. 2024-09-19T09:27:29.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52e9bbf740 20 client.24940 trim_cache size 0 max 16384 2024-09-19T09:27:29.596 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:27:29.596 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:27:29.596 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-19T09:27:29.596 INFO:teuthology.orchestra.run.smithi016.stdout: "a58cc1a9-d204-4dd7-9f0b-6d170fc938b4": { 2024-09-19T09:27:29.597 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-19T09:27:29.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "a58cc1a9-d204-4dd7-9f0b-6d170fc938b4", 2024-09-19T09:27:29.597 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-19T09:27:29.597 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:27:29.597 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:27:29.597 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:27:29.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52e77e6640 1 -- 172.21.15.16:0/3376241598 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f52cc0a3240 msgr2=0x7f52cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52e77e6640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f52cc0a3240 0x7f52cc0a5700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f52d8098980 tx=0x7f52d8015b80 comp rx=0 tx=0).stop 2024-09-19T09:27:29.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52e77e6640 1 -- 172.21.15.16:0/3376241598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52e0106b80 msgr2=0x7f52e019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:29.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52e77e6640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52e0106b80 0x7f52e019ef90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f52d800e9d0 tx=0x7f52d800ee90 comp rx=0 tx=0).stop 2024-09-19T09:27:29.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52e77e6640 1 -- 172.21.15.16:0/3376241598 shutdown_connections 2024-09-19T09:27:29.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52e77e6640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52e0106b80 0x7f52e019ef90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52e77e6640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52e0100aa0 0x7f52e019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52e77e6640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f52cc0a3240 0x7f52cc0a5700 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52e77e6640 1 --2- 172.21.15.16:0/3376241598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52e0100100 0x7f52e019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.590+0000 7f52e77e6640 1 -- 172.21.15.16:0/3376241598 >> 172.21.15.16:0/3376241598 conn(0x7f52e0077230 msgr2=0x7f52e0104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.591+0000 7f52e77e6640 1 -- 172.21.15.16:0/3376241598 shutdown_connections 2024-09-19T09:27:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:29.591+0000 7f52e77e6640 1 -- 172.21.15.16:0/3376241598 wait complete. 2024-09-19T09:27:30.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.250+0000 7f02b63a1640 1 -- 172.21.15.16:0/2867467404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b01002b0 msgr2=0x7f02b0100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.250+0000 7f02b63a1640 1 --2- 172.21.15.16:0/2867467404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b01002b0 0x7f02b0100690 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009900 tx=0x7f02ac02a410 comp rx=0 tx=0).stop 2024-09-19T09:27:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.250+0000 7f02b63a1640 1 -- 172.21.15.16:0/2867467404 shutdown_connections 2024-09-19T09:27:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.250+0000 7f02b63a1640 1 --2- 172.21.15.16:0/2867467404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b01053e0 0x7f02b010cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.250+0000 7f02b63a1640 1 --2- 172.21.15.16:0/2867467404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02b0100c60 0x7f02b0104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.250+0000 7f02b63a1640 1 --2- 172.21.15.16:0/2867467404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b01002b0 0x7f02b0100690 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.250+0000 7f02b63a1640 1 -- 172.21.15.16:0/2867467404 >> 172.21.15.16:0/2867467404 conn(0x7f02b0077070 msgr2=0x7f02b00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.250+0000 7f02b63a1640 1 -- 172.21.15.16:0/2867467404 shutdown_connections 2024-09-19T09:27:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.250+0000 7f02b63a1640 1 -- 172.21.15.16:0/2867467404 wait complete. 2024-09-19T09:27:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.251+0000 7f02b63a1640 1 Processor -- start 2024-09-19T09:27:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.251+0000 7f02b63a1640 1 -- start start 2024-09-19T09:27:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.251+0000 7f02b63a1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b01002b0 0x7f02b01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.251+0000 7f02b63a1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02b0100c60 0x7f02b01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.251+0000 7f02b63a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b01053e0 0x7f02b019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.251+0000 7f02b63a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02b0110c80 con 0x7f02b01053e0 2024-09-19T09:27:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.251+0000 7f02b63a1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f02b0110b00 con 0x7f02b0100c60 2024-09-19T09:27:30.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.251+0000 7f02b63a1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02b0110e00 con 0x7f02b01002b0 2024-09-19T09:27:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.251+0000 7f02b5ba0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b01053e0 0x7f02b019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.251+0000 7f02b539f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b01002b0 0x7f02b01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.251+0000 7f02b5ba0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b01053e0 0x7f02b019cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42710/0 (socket says 172.21.15.16:42710) 2024-09-19T09:27:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.251+0000 7f02b5ba0640 1 -- 172.21.15.16:0/321142399 learned_addr learned my addr 172.21.15.16:0/321142399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.252+0000 7f02b4b9e640 1 --2- 172.21.15.16:0/321142399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02b0100c60 0x7f02b01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.252+0000 7f02b5ba0640 1 -- 172.21.15.16:0/321142399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b01002b0 msgr2=0x7f02b01982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.252+0000 7f02b5ba0640 1 --2- 172.21.15.16:0/321142399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b01002b0 0x7f02b01982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.252+0000 7f02b5ba0640 1 -- 172.21.15.16:0/321142399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02b0100c60 msgr2=0x7f02b01987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.252+0000 7f02b5ba0640 1 --2- 172.21.15.16:0/321142399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02b0100c60 0x7f02b01987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.252+0000 7f02b5ba0640 1 -- 172.21.15.16:0/321142399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b019d260 con 0x7f02b01053e0 2024-09-19T09:27:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.252+0000 7f02b5ba0640 1 --2- 172.21.15.16:0/321142399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b01053e0 0x7f02b019cb80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f02a800eaa0 tx=0x7f02a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.252+0000 7f02a67fc640 1 -- 172.21.15.16:0/321142399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a800cdb0 con 0x7f02b01053e0 2024-09-19T09:27:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.252+0000 7f02a67fc640 1 -- 172.21.15.16:0/321142399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a8005020 con 0x7f02b01053e0 2024-09-19T09:27:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.252+0000 7f02b63a1640 1 -- 172.21.15.16:0/321142399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b019d550 con 0x7f02b01053e0 2024-09-19T09:27:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.252+0000 7f02a67fc640 1 -- 172.21.15.16:0/321142399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a80116b0 con 0x7f02b01053e0 2024-09-19T09:27:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.252+0000 7f02b63a1640 1 -- 172.21.15.16:0/321142399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b01a4e30 con 0x7f02b01053e0 2024-09-19T09:27:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.253+0000 7f02b63a1640 1 -- 172.21.15.16:0/321142399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b0100690 con 0x7f02b01053e0 2024-09-19T09:27:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.255+0000 7f02a67fc640 1 -- 172.21.15.16:0/321142399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f02a8005560 con 0x7f02b01053e0 2024-09-19T09:27:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.255+0000 7f02a67fc640 1 --2- 172.21.15.16:0/321142399 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f029c0a32c0 0x7f029c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.255+0000 7f02b539f640 1 --2- 172.21.15.16:0/321142399 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f029c0a32c0 0x7f029c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.255+0000 7f02a67fc640 1 -- 172.21.15.16:0/321142399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f02a80cf9c0 con 0x7f02b01053e0 2024-09-19T09:27:30.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.256+0000 7f02b539f640 1 --2- 172.21.15.16:0/321142399 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f029c0a32c0 0x7f029c0a5780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009230 tx=0x7f02ac0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:30.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.260+0000 7f02a67fc640 1 -- 172.21.15.16:0/321142399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f02a8016030 con 0x7f02b01053e0 2024-09-19T09:27:30.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.436+0000 7f02b63a1640 1 -- 172.21.15.16:0/321142399 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02b0062630 con 0x7f029c0a32c0 2024-09-19T09:27:30.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.442+0000 7f02a67fc640 1 -- 172.21.15.16:0/321142399 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258892 (secure 0 0 0) 0x7f02b0062630 con 0x7f029c0a32c0 2024-09-19T09:27:30.445 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:30.446 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T09:27:30.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.445+0000 7f02b63a1640 1 -- 172.21.15.16:0/321142399 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f029c0a32c0 msgr2=0x7f029c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:30.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.445+0000 7f02b63a1640 1 --2- 172.21.15.16:0/321142399 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f029c0a32c0 0x7f029c0a5780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009230 tx=0x7f02ac0058d0 comp rx=0 tx=0).stop 2024-09-19T09:27:30.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.445+0000 7f02b63a1640 1 -- 172.21.15.16:0/321142399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b01053e0 msgr2=0x7f02b019cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:30.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.445+0000 7f02b63a1640 1 --2- 172.21.15.16:0/321142399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b01053e0 0x7f02b019cb80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f02a800eaa0 tx=0x7f02a800ef60 comp rx=0 tx=0).stop 2024-09-19T09:27:30.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.445+0000 7f02b63a1640 1 -- 172.21.15.16:0/321142399 shutdown_connections 2024-09-19T09:27:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.445+0000 7f02b63a1640 1 --2- 172.21.15.16:0/321142399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b01053e0 0x7f02b019cb80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.445+0000 7f02b63a1640 1 --2- 172.21.15.16:0/321142399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02b0100c60 0x7f02b01987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.445+0000 7f02b63a1640 1 --2- 172.21.15.16:0/321142399 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f029c0a32c0 0x7f029c0a5780 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.445+0000 7f02b63a1640 1 --2- 172.21.15.16:0/321142399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b01002b0 0x7f02b01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.445+0000 7f02b63a1640 1 -- 172.21.15.16:0/321142399 >> 172.21.15.16:0/321142399 conn(0x7f02b0077070 msgr2=0x7f02b0103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.445+0000 7f02b63a1640 1 -- 172.21.15.16:0/321142399 shutdown_connections 2024-09-19T09:27:30.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:30.445+0000 7f02b63a1640 1 -- 172.21.15.16:0/321142399 wait complete. 2024-09-19T09:27:30.801 INFO:tasks.cephfs.filesystem:scrub status for tag:a58cc1a9-d204-4dd7-9f0b-6d170fc938b4 - {'path': '/', 'tag': 'a58cc1a9-d204-4dd7-9f0b-6d170fc938b4', 'options': 'recursive,force'} 2024-09-19T09:27:31.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:30 smithi016 ceph-mon[30979]: pgmap v335: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T09:27:31.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:30 smithi142 ceph-mon[35032]: pgmap v335: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T09:27:31.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:30 smithi097 ceph-mon[34864]: pgmap v335: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T09:27:31.564 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-19T09:27:29.367385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665162,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1250,"num_read_kb":755,"num_write":446,"num_write_kb":3096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1182578,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391016,"kb_used_data":12648,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550808,"statfs":{"total":1151940427776,"available":1151540027392,"internally_reserved":0,"allocated":12951552,"data_stored":7141086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":69,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002852"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347860+0000","last_change":"2024-09-19T09:25:29.294965+0000","last_active":"2024-09-19T09:27:11.347860+0000","last_peered":"2024-09-19T09:27:11.347860+0000","last_clean":"2024-09-19T09:27:11.347860+0000","last_became_active":"2024-09-19T09:25:29.294157+0000","last_became_peered":"2024-09-19T09:25:29.294157+0000","last_unstale":"2024-09-19T09:27:11.347860+0000","last_undegraded":"2024-09-19T09:27:11.347860+0000","last_fullsized":"2024-09-19T09:27:11.347860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:24.643835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335438+0000","last_change":"2024-09-19T09:25:29.298888+0000","last_active":"2024-09-19T09:27:11.335438+0000","last_peered":"2024-09-19T09:27:11.335438+0000","last_clean":"2024-09-19T09:27:11.335438+0000","last_became_active":"2024-09-19T09:25:29.298245+0000","last_became_peered":"2024-09-19T09:25:29.298245+0000","last_unstale":"2024-09-19T09:27:11.335438+0000","last_undegraded":"2024-09-19T09:27:11.335438+0000","last_fullsized":"2024-09-19T09:27:11.335438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:03.162342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344990+0000","last_change":"2024-09-19T09:25:29.292300+0000","last_active":"2024-09-19T09:27:11.344990+0000","last_peered":"2024-09-19T09:27:11.344990+0000","last_clean":"2024-09-19T09:27:11.344990+0000","last_became_active":"2024-09-19T09:25:29.291973+0000","last_became_peered":"2024-09-19T09:25:29.291973+0000","last_unstale":"2024-09-19T09:27:11.344990+0000","last_undegraded":"2024-09-19T09:27:11.344990+0000","last_fullsized":"2024-09-19T09:27:11.344990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:19.672778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342109+0000","last_change":"2024-09-19T09:25:29.305695+0000","last_active":"2024-09-19T09:27:11.342109+0000","last_peered":"2024-09-19T09:27:11.342109+0000","last_clean":"2024-09-19T09:27:11.342109+0000","last_became_active":"2024-09-19T09:25:29.305244+0000","last_became_peered":"2024-09-19T09:25:29.305244+0000","last_unstale":"2024-09-19T09:27:11.342109+0000","last_undegraded":"2024-09-19T09:27:11.342109+0000","last_fullsized":"2024-09-19T09:27:11.342109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:38.054624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336846+0000","last_change":"2024-09-19T09:25:29.295259+0000","last_active":"2024-09-19T09:27:11.336846+0000","last_peered":"2024-09-19T09:27:11.336846+0000","last_clean":"2024-09-19T09:27:11.336846+0000","last_became_active":"2024-09-19T09:25:29.294936+0000","last_became_peered":"2024-09-19T09:25:29.294936+0000","last_unstale":"2024-09-19T09:27:11.336846+0000","last_undegraded":"2024-09-19T09:27:11.336846+0000","last_fullsized":"2024-09-19T09:27:11.336846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:59.193504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336175+0000","last_change":"2024-09-19T09:25:29.299650+0000","last_active":"2024-09-19T09:27:11.336175+0000","last_peered":"2024-09-19T09:27:11.336175+0000","last_clean":"2024-09-19T09:27:11.336175+0000","last_became_active":"2024-09-19T09:25:29.299182+0000","last_became_peered":"2024-09-19T09:25:29.299182+0000","last_unstale":"2024-09-19T09:27:11.336175+0000","last_undegraded":"2024-09-19T09:27:11.336175+0000","last_fullsized":"2024-09-19T09:27:11.336175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:44.637771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342329+0000","last_change":"2024-09-19T09:25:29.302501+0000","last_active":"2024-09-19T09:27:11.342329+0000","last_peered":"2024-09-19T09:27:11.342329+0000","last_clean":"2024-09-19T09:27:11.342329+0000","last_became_active":"2024-09-19T09:25:29.301419+0000","last_became_peered":"2024-09-19T09:25:29.301419+0000","last_unstale":"2024-09-19T09:27:11.342329+0000","last_undegraded":"2024-09-19T09:27:11.342329+0000","last_fullsized":"2024-09-19T09:27:11.342329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:50.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337306+0000","last_change":"2024-09-19T09:25:29.300212+0000","last_active":"2024-09-19T09:27:11.337306+0000","last_peered":"2024-09-19T09:27:11.337306+0000","last_clean":"2024-09-19T09:27:11.337306+0000","last_became_active":"2024-09-19T09:25:29.299873+0000","last_became_peered":"2024-09-19T09:25:29.299873+0000","last_unstale":"2024-09-19T09:27:11.337306+0000","last_undegraded":"2024-09-19T09:27:11.337306+0000","last_fullsized":"2024-09-19T09:27:11.337306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:34.620314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346891+0000","last_change":"2024-09-19T09:25:29.290214+0000","last_active":"2024-09-19T09:27:11.346891+0000","last_peered":"2024-09-19T09:27:11.346891+0000","last_clean":"2024-09-19T09:27:11.346891+0000","last_became_active":"2024-09-19T09:25:29.289707+0000","last_became_peered":"2024-09-19T09:25:29.289707+0000","last_unstale":"2024-09-19T09:27:11.346891+0000","last_undegraded":"2024-09-19T09:27:11.346891+0000","last_fullsized":"2024-09-19T09:27:11.346891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:16.341340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338183+0000","last_change":"2024-09-19T09:25:29.301774+0000","last_active":"2024-09-19T09:27:11.338183+0000","last_peered":"2024-09-19T09:27:11.338183+0000","last_clean":"2024-09-19T09:27:11.338183+0000","last_became_active":"2024-09-19T09:25:29.301398+0000","last_became_peered":"2024-09-19T09:25:29.301398+0000","last_unstale":"2024-09-19T09:27:11.338183+0000","last_undegraded":"2024-09-19T09:27:11.338183+0000","last_fullsized":"2024-09-19T09:27:11.338183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:36.058038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346172+0000","last_change":"2024-09-19T09:25:29.301475+0000","last_active":"2024-09-19T09:27:11.346172+0000","last_peered":"2024-09-19T09:27:11.346172+0000","last_clean":"2024-09-19T09:27:11.346172+0000","last_became_active":"2024-09-19T09:25:29.300908+0000","last_became_peered":"2024-09-19T09:25:29.300908+0000","last_unstale":"2024-09-19T09:27:11.346172+0000","last_undegraded":"2024-09-19T09:27:11.346172+0000","last_fullsized":"2024-09-19T09:27:11.346172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:44.774127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336219+0000","last_change":"2024-09-19T09:25:42.994395+0000","last_active":"2024-09-19T09:27:11.336219+0000","last_peered":"2024-09-19T09:27:11.336219+0000","last_clean":"2024-09-19T09:27:11.336219+0000","last_became_active":"2024-09-19T09:25:42.993992+0000","last_became_peered":"2024-09-19T09:25:42.993992+0000","last_unstale":"2024-09-19T09:27:11.336219+0000","last_undegraded":"2024-09-19T09:27:11.336219+0000","last_fullsized":"2024-09-19T09:27:11.336219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.486069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.340454+0000","last_change":"2024-09-19T09:25:42.989979+0000","last_active":"2024-09-19T09:27:11.340454+0000","last_peered":"2024-09-19T09:27:11.340454+0000","last_clean":"2024-09-19T09:27:11.340454+0000","last_became_active":"2024-09-19T09:25:42.989578+0000","last_became_peered":"2024-09-19T09:25:42.989578+0000","last_unstale":"2024-09-19T09:27:11.340454+0000","last_undegraded":"2024-09-19T09:27:11.340454+0000","last_fullsized":"2024-09-19T09:27:11.340454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:04.583023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336926+0000","last_change":"2024-09-19T09:25:29.299917+0000","last_active":"2024-09-19T09:27:11.336926+0000","last_peered":"2024-09-19T09:27:11.336926+0000","last_clean":"2024-09-19T09:27:11.336926+0000","last_became_active":"2024-09-19T09:25:29.299571+0000","last_became_peered":"2024-09-19T09:25:29.299571+0000","last_unstale":"2024-09-19T09:27:11.336926+0000","last_undegraded":"2024-09-19T09:27:11.336926+0000","last_fullsized":"2024-09-19T09:27:11.336926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:25.142252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335108+0000","last_change":"2024-09-19T09:25:29.299713+0000","last_active":"2024-09-19T09:27:11.335108+0000","last_peered":"2024-09-19T09:27:11.335108+0000","last_clean":"2024-09-19T09:27:11.335108+0000","last_became_active":"2024-09-19T09:25:29.299234+0000","last_became_peered":"2024-09-19T09:25:29.299234+0000","last_unstale":"2024-09-19T09:27:11.335108+0000","last_undegraded":"2024-09-19T09:27:11.335108+0000","last_fullsized":"2024-09-19T09:27:11.335108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:34.714714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346606+0000","last_change":"2024-09-19T09:25:29.293010+0000","last_active":"2024-09-19T09:27:11.346606+0000","last_peered":"2024-09-19T09:27:11.346606+0000","last_clean":"2024-09-19T09:27:11.346606+0000","last_became_active":"2024-09-19T09:25:29.292645+0000","last_became_peered":"2024-09-19T09:25:29.292645+0000","last_unstale":"2024-09-19T09:27:11.346606+0000","last_undegraded":"2024-09-19T09:27:11.346606+0000","last_fullsized":"2024-09-19T09:27:11.346606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:24.749075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347900+0000","last_change":"2024-09-19T09:25:29.294530+0000","last_active":"2024-09-19T09:27:11.347900+0000","last_peered":"2024-09-19T09:27:11.347900+0000","last_clean":"2024-09-19T09:27:11.347900+0000","last_became_active":"2024-09-19T09:25:29.293699+0000","last_became_peered":"2024-09-19T09:25:29.293699+0000","last_unstale":"2024-09-19T09:27:11.347900+0000","last_undegraded":"2024-09-19T09:27:11.347900+0000","last_fullsized":"2024-09-19T09:27:11.347900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:53.491258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336452+0000","last_change":"2024-09-19T09:25:29.290600+0000","last_active":"2024-09-19T09:27:11.336452+0000","last_peered":"2024-09-19T09:27:11.336452+0000","last_clean":"2024-09-19T09:27:11.336452+0000","last_became_active":"2024-09-19T09:25:29.290305+0000","last_became_peered":"2024-09-19T09:25:29.290305+0000","last_unstale":"2024-09-19T09:27:11.336452+0000","last_undegraded":"2024-09-19T09:27:11.336452+0000","last_fullsized":"2024-09-19T09:27:11.336452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:42.809659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338092+0000","last_change":"2024-09-19T09:25:29.300591+0000","last_active":"2024-09-19T09:27:11.338092+0000","last_peered":"2024-09-19T09:27:11.338092+0000","last_clean":"2024-09-19T09:27:11.338092+0000","last_became_active":"2024-09-19T09:25:29.300251+0000","last_became_peered":"2024-09-19T09:25:29.300251+0000","last_unstale":"2024-09-19T09:27:11.338092+0000","last_undegraded":"2024-09-19T09:27:11.338092+0000","last_fullsized":"2024-09-19T09:27:11.338092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:08.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342064+0000","last_change":"2024-09-19T09:25:29.298452+0000","last_active":"2024-09-19T09:27:11.342064+0000","last_peered":"2024-09-19T09:27:11.342064+0000","last_clean":"2024-09-19T09:27:11.342064+0000","last_became_active":"2024-09-19T09:25:29.298137+0000","last_became_peered":"2024-09-19T09:25:29.298137+0000","last_unstale":"2024-09-19T09:27:11.342064+0000","last_undegraded":"2024-09-19T09:27:11.342064+0000","last_fullsized":"2024-09-19T09:27:11.342064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:11.716228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337968+0000","last_change":"2024-09-19T09:25:29.294278+0000","last_active":"2024-09-19T09:27:11.337968+0000","last_peered":"2024-09-19T09:27:11.337968+0000","last_clean":"2024-09-19T09:27:11.337968+0000","last_became_active":"2024-09-19T09:25:29.293972+0000","last_became_peered":"2024-09-19T09:25:29.293972+0000","last_unstale":"2024-09-19T09:27:11.337968+0000","last_undegraded":"2024-09-19T09:27:11.337968+0000","last_fullsized":"2024-09-19T09:27:11.337968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:27.923839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346634+0000","last_change":"2024-09-19T09:25:29.294000+0000","last_active":"2024-09-19T09:27:11.346634+0000","last_peered":"2024-09-19T09:27:11.346634+0000","last_clean":"2024-09-19T09:27:11.346634+0000","last_became_active":"2024-09-19T09:25:29.293486+0000","last_became_peered":"2024-09-19T09:25:29.293486+0000","last_unstale":"2024-09-19T09:27:11.346634+0000","last_undegraded":"2024-09-19T09:27:11.346634+0000","last_fullsized":"2024-09-19T09:27:11.346634+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:44.053620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334040+0000","last_change":"2024-09-19T09:25:42.994951+0000","last_active":"2024-09-19T09:27:11.334040+0000","last_peered":"2024-09-19T09:27:11.334040+0000","last_clean":"2024-09-19T09:27:11.334040+0000","last_became_active":"2024-09-19T09:25:42.994588+0000","last_became_peered":"2024-09-19T09:25:42.994588+0000","last_unstale":"2024-09-19T09:27:11.334040+0000","last_undegraded":"2024-09-19T09:27:11.334040+0000","last_fullsized":"2024-09-19T09:27:11.334040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:10.141169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334658+0000","last_change":"2024-09-19T09:25:29.286395+0000","last_active":"2024-09-19T09:27:11.334658+0000","last_peered":"2024-09-19T09:27:11.334658+0000","last_clean":"2024-09-19T09:27:11.334658+0000","last_became_active":"2024-09-19T09:25:29.286048+0000","last_became_peered":"2024-09-19T09:25:29.286048+0000","last_unstale":"2024-09-19T09:27:11.334658+0000","last_undegraded":"2024-09-19T09:27:11.334658+0000","last_fullsized":"2024-09-19T09:27:11.334658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:43.121166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.327390+0000","last_change":"2024-09-19T09:25:29.298458+0000","last_active":"2024-09-19T09:27:11.327390+0000","last_peered":"2024-09-19T09:27:11.327390+0000","last_clean":"2024-09-19T09:27:11.327390+0000","last_became_active":"2024-09-19T09:25:29.297993+0000","last_became_peered":"2024-09-19T09:25:29.297993+0000","last_unstale":"2024-09-19T09:27:11.327390+0000","last_undegraded":"2024-09-19T09:27:11.327390+0000","last_fullsized":"2024-09-19T09:27:11.327390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:44.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344574+0000","last_change":"2024-09-19T09:25:29.297167+0000","last_active":"2024-09-19T09:27:11.344574+0000","last_peered":"2024-09-19T09:27:11.344574+0000","last_clean":"2024-09-19T09:27:11.344574+0000","last_became_active":"2024-09-19T09:25:29.296777+0000","last_became_peered":"2024-09-19T09:25:29.296777+0000","last_unstale":"2024-09-19T09:27:11.344574+0000","last_undegraded":"2024-09-19T09:27:11.344574+0000","last_fullsized":"2024-09-19T09:27:11.344574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:40.581643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.339599+0000","last_change":"2024-09-19T09:25:29.296169+0000","last_active":"2024-09-19T09:27:11.339599+0000","last_peered":"2024-09-19T09:27:11.339599+0000","last_clean":"2024-09-19T09:27:11.339599+0000","last_became_active":"2024-09-19T09:25:29.295806+0000","last_became_peered":"2024-09-19T09:25:29.295806+0000","last_unstale":"2024-09-19T09:27:11.339599+0000","last_undegraded":"2024-09-19T09:27:11.339599+0000","last_fullsized":"2024-09-19T09:27:11.339599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:39.639884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334590+0000","last_change":"2024-09-19T09:25:29.297960+0000","last_active":"2024-09-19T09:27:11.334590+0000","last_peered":"2024-09-19T09:27:11.334590+0000","last_clean":"2024-09-19T09:27:11.334590+0000","last_became_active":"2024-09-19T09:25:29.297577+0000","last_became_peered":"2024-09-19T09:25:29.297577+0000","last_unstale":"2024-09-19T09:27:11.334590+0000","last_undegraded":"2024-09-19T09:27:11.334590+0000","last_fullsized":"2024-09-19T09:27:11.334590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:27.035504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335188+0000","last_change":"2024-09-19T09:25:29.297896+0000","last_active":"2024-09-19T09:27:11.335188+0000","last_peered":"2024-09-19T09:27:11.335188+0000","last_clean":"2024-09-19T09:27:11.335188+0000","last_became_active":"2024-09-19T09:25:29.297594+0000","last_became_peered":"2024-09-19T09:25:29.297594+0000","last_unstale":"2024-09-19T09:27:11.335188+0000","last_undegraded":"2024-09-19T09:27:11.335188+0000","last_fullsized":"2024-09-19T09:27:11.335188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.366325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334560+0000","last_change":"2024-09-19T09:25:29.301851+0000","last_active":"2024-09-19T09:27:11.334560+0000","last_peered":"2024-09-19T09:27:11.334560+0000","last_clean":"2024-09-19T09:27:11.334560+0000","last_became_active":"2024-09-19T09:25:29.299282+0000","last_became_peered":"2024-09-19T09:25:29.299282+0000","last_unstale":"2024-09-19T09:27:11.334560+0000","last_undegraded":"2024-09-19T09:27:11.334560+0000","last_fullsized":"2024-09-19T09:27:11.334560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:00.382114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342990+0000","last_change":"2024-09-19T09:25:29.303316+0000","last_active":"2024-09-19T09:27:11.342990+0000","last_peered":"2024-09-19T09:27:11.342990+0000","last_clean":"2024-09-19T09:27:11.342990+0000","last_became_active":"2024-09-19T09:25:29.302994+0000","last_became_peered":"2024-09-19T09:25:29.302994+0000","last_unstale":"2024-09-19T09:27:11.342990+0000","last_undegraded":"2024-09-19T09:27:11.342990+0000","last_fullsized":"2024-09-19T09:27:11.342990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:46.233991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341767+0000","last_change":"2024-09-19T09:25:29.293004+0000","last_active":"2024-09-19T09:27:11.341767+0000","last_peered":"2024-09-19T09:27:11.341767+0000","last_clean":"2024-09-19T09:27:11.341767+0000","last_became_active":"2024-09-19T09:25:29.292657+0000","last_became_peered":"2024-09-19T09:25:29.292657+0000","last_unstale":"2024-09-19T09:27:11.341767+0000","last_undegraded":"2024-09-19T09:27:11.341767+0000","last_fullsized":"2024-09-19T09:27:11.341767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:50.966477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.332473+0000","last_change":"2024-09-19T09:25:29.300434+0000","last_active":"2024-09-19T09:27:11.332473+0000","last_peered":"2024-09-19T09:27:11.332473+0000","last_clean":"2024-09-19T09:27:11.332473+0000","last_became_active":"2024-09-19T09:25:29.300053+0000","last_became_peered":"2024-09-19T09:25:29.300053+0000","last_unstale":"2024-09-19T09:27:11.332473+0000","last_undegraded":"2024-09-19T09:27:11.332473+0000","last_fullsized":"2024-09-19T09:27:11.332473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:13.283298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":142,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:26.121860+0000","last_change":"2024-09-19T09:25:27.280792+0000","last_active":"2024-09-19T09:27:26.121860+0000","last_peered":"2024-09-19T09:27:26.121860+0000","last_clean":"2024-09-19T09:27:26.121860+0000","last_became_active":"2024-09-19T09:25:27.280443+0000","last_became_peered":"2024-09-19T09:25:27.280443+0000","last_unstale":"2024-09-19T09:27:26.121860+0000","last_undegraded":"2024-09-19T09:27:26.121860+0000","last_fullsized":"2024-09-19T09:27:26.121860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.599512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334165+0000","last_change":"2024-09-19T09:25:29.298975+0000","last_active":"2024-09-19T09:27:11.334165+0000","last_peered":"2024-09-19T09:27:11.334165+0000","last_clean":"2024-09-19T09:27:11.334165+0000","last_became_active":"2024-09-19T09:25:29.298476+0000","last_became_peered":"2024-09-19T09:25:29.298476+0000","last_unstale":"2024-09-19T09:27:11.334165+0000","last_undegraded":"2024-09-19T09:27:11.334165+0000","last_fullsized":"2024-09-19T09:27:11.334165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:09.658894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986623+0000","last_change":"2024-09-19T09:25:27.278257+0000","last_active":"2024-09-19T09:25:42.986623+0000","last_peered":"2024-09-19T09:25:42.986623+0000","last_clean":"2024-09-19T09:25:42.986623+0000","last_became_active":"2024-09-19T09:25:27.277897+0000","last_became_peered":"2024-09-19T09:25:27.277897+0000","last_unstale":"2024-09-19T09:25:42.986623+0000","last_undegraded":"2024-09-19T09:25:42.986623+0000","last_fullsized":"2024-09-19T09:25:42.986623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:47.550314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985617+0000","last_change":"2024-09-19T09:25:27.272215+0000","last_active":"2024-09-19T09:25:42.985617+0000","last_peered":"2024-09-19T09:25:42.985617+0000","last_clean":"2024-09-19T09:25:42.985617+0000","last_became_active":"2024-09-19T09:25:27.271821+0000","last_became_peered":"2024-09-19T09:25:27.271821+0000","last_unstale":"2024-09-19T09:25:42.985617+0000","last_undegraded":"2024-09-19T09:25:42.985617+0000","last_fullsized":"2024-09-19T09:25:42.985617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:30.615730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.329744+0000","last_change":"2024-09-19T09:25:29.289347+0000","last_active":"2024-09-19T09:27:11.329744+0000","last_peered":"2024-09-19T09:27:11.329744+0000","last_clean":"2024-09-19T09:27:11.329744+0000","last_became_active":"2024-09-19T09:25:29.289015+0000","last_became_peered":"2024-09-19T09:25:29.289015+0000","last_unstale":"2024-09-19T09:27:11.329744+0000","last_undegraded":"2024-09-19T09:27:11.329744+0000","last_fullsized":"2024-09-19T09:27:11.329744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:18.419334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995432+0000","last_change":"2024-09-19T09:25:27.277342+0000","last_active":"2024-09-19T09:25:42.995432+0000","last_peered":"2024-09-19T09:25:42.995432+0000","last_clean":"2024-09-19T09:25:42.995432+0000","last_became_active":"2024-09-19T09:25:27.277016+0000","last_became_peered":"2024-09-19T09:25:27.277016+0000","last_unstale":"2024-09-19T09:25:42.995432+0000","last_undegraded":"2024-09-19T09:25:42.995432+0000","last_fullsized":"2024-09-19T09:25:42.995432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:56.736156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336629+0000","last_change":"2024-09-19T09:25:29.290924+0000","last_active":"2024-09-19T09:27:11.336629+0000","last_peered":"2024-09-19T09:27:11.336629+0000","last_clean":"2024-09-19T09:27:11.336629+0000","last_became_active":"2024-09-19T09:25:29.290482+0000","last_became_peered":"2024-09-19T09:25:29.290482+0000","last_unstale":"2024-09-19T09:27:11.336629+0000","last_undegraded":"2024-09-19T09:27:11.336629+0000","last_fullsized":"2024-09-19T09:27:11.336629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:16.265040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'6","reported_seq":242,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:26.121271+0000","last_change":"2024-09-19T09:25:27.277604+0000","last_active":"2024-09-19T09:27:26.121271+0000","last_peered":"2024-09-19T09:27:26.121271+0000","last_clean":"2024-09-19T09:27:26.121271+0000","last_became_active":"2024-09-19T09:25:27.277271+0000","last_became_peered":"2024-09-19T09:25:27.277271+0000","last_unstale":"2024-09-19T09:27:26.121271+0000","last_undegraded":"2024-09-19T09:27:26.121271+0000","last_fullsized":"2024-09-19T09:27:26.121271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:32.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347371+0000","last_change":"2024-09-19T09:25:29.290537+0000","last_active":"2024-09-19T09:27:11.347371+0000","last_peered":"2024-09-19T09:27:11.347371+0000","last_clean":"2024-09-19T09:27:11.347371+0000","last_became_active":"2024-09-19T09:25:29.287984+0000","last_became_peered":"2024-09-19T09:25:29.287984+0000","last_unstale":"2024-09-19T09:27:11.347371+0000","last_undegraded":"2024-09-19T09:27:11.347371+0000","last_fullsized":"2024-09-19T09:27:11.347371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:26.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990018+0000","last_change":"2024-09-19T09:25:27.278560+0000","last_active":"2024-09-19T09:25:42.990018+0000","last_peered":"2024-09-19T09:25:42.990018+0000","last_clean":"2024-09-19T09:25:42.990018+0000","last_became_active":"2024-09-19T09:25:27.277950+0000","last_became_peered":"2024-09-19T09:25:27.277950+0000","last_unstale":"2024-09-19T09:25:42.990018+0000","last_undegraded":"2024-09-19T09:25:42.990018+0000","last_fullsized":"2024-09-19T09:25:42.990018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:28.387725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.339317+0000","last_change":"2024-09-19T09:25:29.298897+0000","last_active":"2024-09-19T09:27:11.339317+0000","last_peered":"2024-09-19T09:27:11.339317+0000","last_clean":"2024-09-19T09:27:11.339317+0000","last_became_active":"2024-09-19T09:25:29.298617+0000","last_became_peered":"2024-09-19T09:25:29.298617+0000","last_unstale":"2024-09-19T09:27:11.339317+0000","last_undegraded":"2024-09-19T09:27:11.339317+0000","last_fullsized":"2024-09-19T09:27:11.339317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:53.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994870+0000","last_change":"2024-09-19T09:25:27.324871+0000","last_active":"2024-09-19T09:25:42.994870+0000","last_peered":"2024-09-19T09:25:42.994870+0000","last_clean":"2024-09-19T09:25:42.994870+0000","last_became_active":"2024-09-19T09:25:27.324471+0000","last_became_peered":"2024-09-19T09:25:27.324471+0000","last_unstale":"2024-09-19T09:25:42.994870+0000","last_undegraded":"2024-09-19T09:25:42.994870+0000","last_fullsized":"2024-09-19T09:25:42.994870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:35.936413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347260+0000","last_change":"2024-09-19T09:25:29.298332+0000","last_active":"2024-09-19T09:27:11.347260+0000","last_peered":"2024-09-19T09:27:11.347260+0000","last_clean":"2024-09-19T09:27:11.347260+0000","last_became_active":"2024-09-19T09:25:29.298006+0000","last_became_peered":"2024-09-19T09:25:29.298006+0000","last_unstale":"2024-09-19T09:27:11.347260+0000","last_undegraded":"2024-09-19T09:27:11.347260+0000","last_fullsized":"2024-09-19T09:27:11.347260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:02.621821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'35","reported_seq":285,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:26.121266+0000","last_change":"2024-09-19T09:25:27.279528+0000","last_active":"2024-09-19T09:27:26.121266+0000","last_peered":"2024-09-19T09:27:26.121266+0000","last_clean":"2024-09-19T09:27:26.121266+0000","last_became_active":"2024-09-19T09:25:27.279149+0000","last_became_peered":"2024-09-19T09:25:27.279149+0000","last_unstale":"2024-09-19T09:27:26.121266+0000","last_undegraded":"2024-09-19T09:27:26.121266+0000","last_fullsized":"2024-09-19T09:27:26.121266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:47.030736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65694,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":228,"num_read_kb":178,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.324455+0000","last_change":"2024-09-19T09:25:29.297538+0000","last_active":"2024-09-19T09:27:11.324455+0000","last_peered":"2024-09-19T09:27:11.324455+0000","last_clean":"2024-09-19T09:27:11.324455+0000","last_became_active":"2024-09-19T09:25:29.297181+0000","last_became_peered":"2024-09-19T09:25:29.297181+0000","last_unstale":"2024-09-19T09:27:11.324455+0000","last_undegraded":"2024-09-19T09:27:11.324455+0000","last_fullsized":"2024-09-19T09:27:11.324455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:53.135438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992860+0000","last_change":"2024-09-19T09:25:27.277090+0000","last_active":"2024-09-19T09:25:42.992860+0000","last_peered":"2024-09-19T09:25:42.992860+0000","last_clean":"2024-09-19T09:25:42.992860+0000","last_became_active":"2024-09-19T09:25:27.276718+0000","last_became_peered":"2024-09-19T09:25:27.276718+0000","last_unstale":"2024-09-19T09:25:42.992860+0000","last_undegraded":"2024-09-19T09:25:42.992860+0000","last_fullsized":"2024-09-19T09:25:42.992860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:25.664261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.325840+0000","last_change":"2024-09-19T09:25:29.294393+0000","last_active":"2024-09-19T09:27:11.325840+0000","last_peered":"2024-09-19T09:27:11.325840+0000","last_clean":"2024-09-19T09:27:11.325840+0000","last_became_active":"2024-09-19T09:25:29.294099+0000","last_became_peered":"2024-09-19T09:25:29.294099+0000","last_unstale":"2024-09-19T09:27:11.325840+0000","last_undegraded":"2024-09-19T09:27:11.325840+0000","last_fullsized":"2024-09-19T09:27:11.325840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:17.333736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995398+0000","last_change":"2024-09-19T09:25:27.278593+0000","last_active":"2024-09-19T09:25:42.995398+0000","last_peered":"2024-09-19T09:25:42.995398+0000","last_clean":"2024-09-19T09:25:42.995398+0000","last_became_active":"2024-09-19T09:25:27.278232+0000","last_became_peered":"2024-09-19T09:25:27.278232+0000","last_unstale":"2024-09-19T09:25:42.995398+0000","last_undegraded":"2024-09-19T09:25:42.995398+0000","last_fullsized":"2024-09-19T09:25:42.995398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:05.852671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343581+0000","last_change":"2024-09-19T09:25:29.300024+0000","last_active":"2024-09-19T09:27:11.343581+0000","last_peered":"2024-09-19T09:27:11.343581+0000","last_clean":"2024-09-19T09:27:11.343581+0000","last_became_active":"2024-09-19T09:25:29.299683+0000","last_became_peered":"2024-09-19T09:25:29.299683+0000","last_unstale":"2024-09-19T09:27:11.343581+0000","last_undegraded":"2024-09-19T09:27:11.343581+0000","last_fullsized":"2024-09-19T09:27:11.343581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:42.249399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'7","reported_seq":141,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:25.120627+0000","last_change":"2024-09-19T09:25:27.272739+0000","last_active":"2024-09-19T09:27:25.120627+0000","last_peered":"2024-09-19T09:27:25.120627+0000","last_clean":"2024-09-19T09:27:25.120627+0000","last_became_active":"2024-09-19T09:25:27.272354+0000","last_became_peered":"2024-09-19T09:25:27.272354+0000","last_unstale":"2024-09-19T09:27:25.120627+0000","last_undegraded":"2024-09-19T09:27:25.120627+0000","last_fullsized":"2024-09-19T09:27:25.120627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:27.863840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343011+0000","last_change":"2024-09-19T09:25:29.303460+0000","last_active":"2024-09-19T09:27:11.343011+0000","last_peered":"2024-09-19T09:27:11.343011+0000","last_clean":"2024-09-19T09:27:11.343011+0000","last_became_active":"2024-09-19T09:25:29.303127+0000","last_became_peered":"2024-09-19T09:25:29.303127+0000","last_unstale":"2024-09-19T09:27:11.343011+0000","last_undegraded":"2024-09-19T09:27:11.343011+0000","last_fullsized":"2024-09-19T09:27:11.343011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:35.313924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990876+0000","last_change":"2024-09-19T09:25:27.323238+0000","last_active":"2024-09-19T09:25:42.990876+0000","last_peered":"2024-09-19T09:25:42.990876+0000","last_clean":"2024-09-19T09:25:42.990876+0000","last_became_active":"2024-09-19T09:25:27.322868+0000","last_became_peered":"2024-09-19T09:25:27.322868+0000","last_unstale":"2024-09-19T09:25:42.990876+0000","last_undegraded":"2024-09-19T09:25:42.990876+0000","last_fullsized":"2024-09-19T09:25:42.990876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:22.890280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338463+0000","last_change":"2024-09-19T09:25:43.005569+0000","last_active":"2024-09-19T09:27:11.338463+0000","last_peered":"2024-09-19T09:27:11.338463+0000","last_clean":"2024-09-19T09:27:11.338463+0000","last_became_active":"2024-09-19T09:25:43.005143+0000","last_became_peered":"2024-09-19T09:25:43.005143+0000","last_unstale":"2024-09-19T09:27:11.338463+0000","last_undegraded":"2024-09-19T09:27:11.338463+0000","last_fullsized":"2024-09-19T09:27:11.338463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:26.297013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991438+0000","last_change":"2024-09-19T09:25:27.282737+0000","last_active":"2024-09-19T09:25:42.991438+0000","last_peered":"2024-09-19T09:25:42.991438+0000","last_clean":"2024-09-19T09:25:42.991438+0000","last_became_active":"2024-09-19T09:25:27.281785+0000","last_became_peered":"2024-09-19T09:25:27.281785+0000","last_unstale":"2024-09-19T09:25:42.991438+0000","last_undegraded":"2024-09-19T09:25:42.991438+0000","last_fullsized":"2024-09-19T09:25:42.991438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:41.552206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.340095+0000","last_change":"2024-09-19T09:25:29.300168+0000","last_active":"2024-09-19T09:27:11.340095+0000","last_peered":"2024-09-19T09:27:11.340095+0000","last_clean":"2024-09-19T09:27:11.340095+0000","last_became_active":"2024-09-19T09:25:29.299811+0000","last_became_peered":"2024-09-19T09:25:29.299811+0000","last_unstale":"2024-09-19T09:27:11.340095+0000","last_undegraded":"2024-09-19T09:27:11.340095+0000","last_fullsized":"2024-09-19T09:27:11.340095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:06.691259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985783+0000","last_change":"2024-09-19T09:25:27.278433+0000","last_active":"2024-09-19T09:25:42.985783+0000","last_peered":"2024-09-19T09:25:42.985783+0000","last_clean":"2024-09-19T09:25:42.985783+0000","last_became_active":"2024-09-19T09:25:27.278076+0000","last_became_peered":"2024-09-19T09:25:27.278076+0000","last_unstale":"2024-09-19T09:25:42.985783+0000","last_undegraded":"2024-09-19T09:25:42.985783+0000","last_fullsized":"2024-09-19T09:25:42.985783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:53.799265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338515+0000","last_change":"2024-09-19T09:25:29.295650+0000","last_active":"2024-09-19T09:27:11.338515+0000","last_peered":"2024-09-19T09:27:11.338515+0000","last_clean":"2024-09-19T09:27:11.338515+0000","last_became_active":"2024-09-19T09:25:29.295277+0000","last_became_peered":"2024-09-19T09:25:29.295277+0000","last_unstale":"2024-09-19T09:27:11.338515+0000","last_undegraded":"2024-09-19T09:27:11.338515+0000","last_fullsized":"2024-09-19T09:27:11.338515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:57.690699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996123+0000","last_change":"2024-09-19T09:25:27.279239+0000","last_active":"2024-09-19T09:25:42.996123+0000","last_peered":"2024-09-19T09:25:42.996123+0000","last_clean":"2024-09-19T09:25:42.996123+0000","last_became_active":"2024-09-19T09:25:27.278868+0000","last_became_peered":"2024-09-19T09:25:27.278868+0000","last_unstale":"2024-09-19T09:25:42.996123+0000","last_undegraded":"2024-09-19T09:25:42.996123+0000","last_fullsized":"2024-09-19T09:25:42.996123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:41.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341911+0000","last_change":"2024-09-19T09:25:29.300833+0000","last_active":"2024-09-19T09:27:11.341911+0000","last_peered":"2024-09-19T09:27:11.341911+0000","last_clean":"2024-09-19T09:27:11.341911+0000","last_became_active":"2024-09-19T09:25:29.300472+0000","last_became_peered":"2024-09-19T09:25:29.300472+0000","last_unstale":"2024-09-19T09:27:11.341911+0000","last_undegraded":"2024-09-19T09:27:11.341911+0000","last_fullsized":"2024-09-19T09:27:11.341911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:50.591276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'8","reported_seq":129,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:25.156716+0000","last_change":"2024-09-19T09:25:43.001722+0000","last_active":"2024-09-19T09:27:25.156716+0000","last_peered":"2024-09-19T09:27:25.156716+0000","last_clean":"2024-09-19T09:27:25.156716+0000","last_became_active":"2024-09-19T09:25:42.992796+0000","last_became_peered":"2024-09-19T09:25:42.992796+0000","last_unstale":"2024-09-19T09:27:25.156716+0000","last_undegraded":"2024-09-19T09:27:25.156716+0000","last_fullsized":"2024-09-19T09:27:25.156716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:59.639010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2012,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346495+0000","last_change":"2024-09-19T09:25:29.295906+0000","last_active":"2024-09-19T09:27:11.346495+0000","last_peered":"2024-09-19T09:27:11.346495+0000","last_clean":"2024-09-19T09:27:11.346495+0000","last_became_active":"2024-09-19T09:25:29.295565+0000","last_became_peered":"2024-09-19T09:25:29.295565+0000","last_unstale":"2024-09-19T09:27:11.346495+0000","last_undegraded":"2024-09-19T09:27:11.346495+0000","last_fullsized":"2024-09-19T09:27:11.346495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:24.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989907+0000","last_change":"2024-09-19T09:24:19.256300+0000","last_active":"2024-09-19T09:25:42.989907+0000","last_peered":"2024-09-19T09:25:42.989907+0000","last_clean":"2024-09-19T09:25:42.989907+0000","last_became_active":"2024-09-19T09:24:19.255833+0000","last_became_peered":"2024-09-19T09:24:19.255833+0000","last_unstale":"2024-09-19T09:25:42.989907+0000","last_undegraded":"2024-09-19T09:25:42.989907+0000","last_fullsized":"2024-09-19T09:25:42.989907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_clean_scrub_stamp":"2024-09-19T09:20:57.798838+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:14:33.343748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989637+0000","last_change":"2024-09-19T09:25:27.272090+0000","last_active":"2024-09-19T09:25:42.989637+0000","last_peered":"2024-09-19T09:25:42.989637+0000","last_clean":"2024-09-19T09:25:42.989637+0000","last_became_active":"2024-09-19T09:25:27.271651+0000","last_became_peered":"2024-09-19T09:25:27.271651+0000","last_unstale":"2024-09-19T09:25:42.989637+0000","last_undegraded":"2024-09-19T09:25:42.989637+0000","last_fullsized":"2024-09-19T09:25:42.989637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:45.847082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344375+0000","last_change":"2024-09-19T09:25:29.300718+0000","last_active":"2024-09-19T09:27:11.344375+0000","last_peered":"2024-09-19T09:27:11.344375+0000","last_clean":"2024-09-19T09:27:11.344375+0000","last_became_active":"2024-09-19T09:25:29.299460+0000","last_became_peered":"2024-09-19T09:25:29.299460+0000","last_unstale":"2024-09-19T09:27:11.344375+0000","last_undegraded":"2024-09-19T09:27:11.344375+0000","last_fullsized":"2024-09-19T09:27:11.344375+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:43.063909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003919+0000","last_change":"2024-09-19T09:25:43.003299+0000","last_active":"2024-09-19T09:25:43.003919+0000","last_peered":"2024-09-19T09:25:43.003919+0000","last_clean":"2024-09-19T09:25:43.003919+0000","last_became_active":"2024-09-19T09:25:42.999686+0000","last_became_peered":"2024-09-19T09:25:42.999686+0000","last_unstale":"2024-09-19T09:25:43.003919+0000","last_undegraded":"2024-09-19T09:25:43.003919+0000","last_fullsized":"2024-09-19T09:25:43.003919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:05.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337734+0000","last_change":"2024-09-19T09:25:29.297833+0000","last_active":"2024-09-19T09:27:11.337734+0000","last_peered":"2024-09-19T09:27:11.337734+0000","last_clean":"2024-09-19T09:27:11.337734+0000","last_became_active":"2024-09-19T09:25:29.297534+0000","last_became_peered":"2024-09-19T09:25:29.297534+0000","last_unstale":"2024-09-19T09:27:11.337734+0000","last_undegraded":"2024-09-19T09:27:11.337734+0000","last_fullsized":"2024-09-19T09:27:11.337734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:47.156253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:27.122980+0000","last_change":"2024-09-19T09:25:27.322759+0000","last_active":"2024-09-19T09:27:27.122980+0000","last_peered":"2024-09-19T09:27:27.122980+0000","last_clean":"2024-09-19T09:27:27.122980+0000","last_became_active":"2024-09-19T09:25:27.322402+0000","last_became_peered":"2024-09-19T09:25:27.322402+0000","last_unstale":"2024-09-19T09:27:27.122980+0000","last_undegraded":"2024-09-19T09:27:27.122980+0000","last_fullsized":"2024-09-19T09:27:27.122980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:06.276957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335476+0000","last_change":"2024-09-19T09:25:42.999834+0000","last_active":"2024-09-19T09:27:11.335476+0000","last_peered":"2024-09-19T09:27:11.335476+0000","last_clean":"2024-09-19T09:27:11.335476+0000","last_became_active":"2024-09-19T09:25:42.999450+0000","last_became_peered":"2024-09-19T09:25:42.999450+0000","last_unstale":"2024-09-19T09:27:11.335476+0000","last_undegraded":"2024-09-19T09:27:11.335476+0000","last_fullsized":"2024-09-19T09:27:11.335476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:00.509844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'7","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:26.129531+0000","last_change":"2024-09-19T09:25:27.275212+0000","last_active":"2024-09-19T09:27:26.129531+0000","last_peered":"2024-09-19T09:27:26.129531+0000","last_clean":"2024-09-19T09:27:26.129531+0000","last_became_active":"2024-09-19T09:25:27.274802+0000","last_became_peered":"2024-09-19T09:25:27.274802+0000","last_unstale":"2024-09-19T09:27:26.129531+0000","last_undegraded":"2024-09-19T09:27:26.129531+0000","last_fullsized":"2024-09-19T09:27:26.129531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:48.392026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.333364+0000","last_change":"2024-09-19T09:25:29.304747+0000","last_active":"2024-09-19T09:27:11.333364+0000","last_peered":"2024-09-19T09:27:11.333364+0000","last_clean":"2024-09-19T09:27:11.333364+0000","last_became_active":"2024-09-19T09:25:29.304386+0000","last_became_peered":"2024-09-19T09:25:29.304386+0000","last_unstale":"2024-09-19T09:27:11.333364+0000","last_undegraded":"2024-09-19T09:27:11.333364+0000","last_fullsized":"2024-09-19T09:27:11.333364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.788697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988854+0000","last_change":"2024-09-19T09:25:27.323083+0000","last_active":"2024-09-19T09:25:42.988854+0000","last_peered":"2024-09-19T09:25:42.988854+0000","last_clean":"2024-09-19T09:25:42.988854+0000","last_became_active":"2024-09-19T09:25:27.322707+0000","last_became_peered":"2024-09-19T09:25:27.322707+0000","last_unstale":"2024-09-19T09:25:42.988854+0000","last_undegraded":"2024-09-19T09:25:42.988854+0000","last_fullsized":"2024-09-19T09:25:42.988854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:24.415116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334132+0000","last_change":"2024-09-19T09:25:29.292472+0000","last_active":"2024-09-19T09:27:11.334132+0000","last_peered":"2024-09-19T09:27:11.334132+0000","last_clean":"2024-09-19T09:27:11.334132+0000","last_became_active":"2024-09-19T09:25:29.292079+0000","last_became_peered":"2024-09-19T09:25:29.292079+0000","last_unstale":"2024-09-19T09:27:11.334132+0000","last_undegraded":"2024-09-19T09:27:11.334132+0000","last_fullsized":"2024-09-19T09:27:11.334132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:35.801594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991224+0000","last_change":"2024-09-19T09:25:27.324839+0000","last_active":"2024-09-19T09:25:42.991224+0000","last_peered":"2024-09-19T09:25:42.991224+0000","last_clean":"2024-09-19T09:25:42.991224+0000","last_became_active":"2024-09-19T09:25:27.324475+0000","last_became_peered":"2024-09-19T09:25:27.324475+0000","last_unstale":"2024-09-19T09:25:42.991224+0000","last_undegraded":"2024-09-19T09:25:42.991224+0000","last_fullsized":"2024-09-19T09:25:42.991224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:32.251753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338007+0000","last_change":"2024-09-19T09:25:29.293025+0000","last_active":"2024-09-19T09:27:11.338007+0000","last_peered":"2024-09-19T09:27:11.338007+0000","last_clean":"2024-09-19T09:27:11.338007+0000","last_became_active":"2024-09-19T09:25:29.292570+0000","last_became_peered":"2024-09-19T09:25:29.292570+0000","last_unstale":"2024-09-19T09:27:11.338007+0000","last_undegraded":"2024-09-19T09:27:11.338007+0000","last_fullsized":"2024-09-19T09:27:11.338007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:16.394524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'17","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.329192+0000","last_change":"2024-09-19T09:25:27.278100+0000","last_active":"2024-09-19T09:27:11.329192+0000","last_peered":"2024-09-19T09:27:11.329192+0000","last_clean":"2024-09-19T09:27:11.329192+0000","last_became_active":"2024-09-19T09:25:27.277769+0000","last_became_peered":"2024-09-19T09:25:27.277769+0000","last_unstale":"2024-09-19T09:27:11.329192+0000","last_undegraded":"2024-09-19T09:27:11.329192+0000","last_fullsized":"2024-09-19T09:27:11.329192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:31.885142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338230+0000","last_change":"2024-09-19T09:25:29.293651+0000","last_active":"2024-09-19T09:27:11.338230+0000","last_peered":"2024-09-19T09:27:11.338230+0000","last_clean":"2024-09-19T09:27:11.338230+0000","last_became_active":"2024-09-19T09:25:29.293317+0000","last_became_peered":"2024-09-19T09:25:29.293317+0000","last_unstale":"2024-09-19T09:27:11.338230+0000","last_undegraded":"2024-09-19T09:27:11.338230+0000","last_fullsized":"2024-09-19T09:27:11.338230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:40.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990030+0000","last_change":"2024-09-19T09:25:27.269367+0000","last_active":"2024-09-19T09:25:42.990030+0000","last_peered":"2024-09-19T09:25:42.990030+0000","last_clean":"2024-09-19T09:25:42.990030+0000","last_became_active":"2024-09-19T09:25:27.269011+0000","last_became_peered":"2024-09-19T09:25:27.269011+0000","last_unstale":"2024-09-19T09:25:42.990030+0000","last_undegraded":"2024-09-19T09:25:42.990030+0000","last_fullsized":"2024-09-19T09:25:42.990030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:00.930752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334968+0000","last_change":"2024-09-19T09:25:29.299134+0000","last_active":"2024-09-19T09:27:11.334968+0000","last_peered":"2024-09-19T09:27:11.334968+0000","last_clean":"2024-09-19T09:27:11.334968+0000","last_became_active":"2024-09-19T09:25:29.298648+0000","last_became_peered":"2024-09-19T09:25:29.298648+0000","last_unstale":"2024-09-19T09:27:11.334968+0000","last_undegraded":"2024-09-19T09:27:11.334968+0000","last_fullsized":"2024-09-19T09:27:11.334968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:39.627852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003686+0000","last_change":"2024-09-19T09:25:27.322861+0000","last_active":"2024-09-19T09:25:43.003686+0000","last_peered":"2024-09-19T09:25:43.003686+0000","last_clean":"2024-09-19T09:25:43.003686+0000","last_became_active":"2024-09-19T09:25:27.322560+0000","last_became_peered":"2024-09-19T09:25:27.322560+0000","last_unstale":"2024-09-19T09:25:43.003686+0000","last_undegraded":"2024-09-19T09:25:43.003686+0000","last_fullsized":"2024-09-19T09:25:43.003686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:35.993093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.301522+0000","last_change":"2024-09-19T09:25:29.301680+0000","last_active":"2024-09-19T09:27:11.301522+0000","last_peered":"2024-09-19T09:27:11.301522+0000","last_clean":"2024-09-19T09:27:11.301522+0000","last_became_active":"2024-09-19T09:25:29.301331+0000","last_became_peered":"2024-09-19T09:25:29.301331+0000","last_unstale":"2024-09-19T09:27:11.301522+0000","last_undegraded":"2024-09-19T09:27:11.301522+0000","last_fullsized":"2024-09-19T09:27:11.301522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:57.364245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:27.272609+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:27.272169+0000","last_became_peered":"2024-09-19T09:25:27.272169+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.879474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341200+0000","last_change":"2024-09-19T09:25:29.296847+0000","last_active":"2024-09-19T09:27:11.341200+0000","last_peered":"2024-09-19T09:27:11.341200+0000","last_clean":"2024-09-19T09:27:11.341200+0000","last_became_active":"2024-09-19T09:25:29.296465+0000","last_became_peered":"2024-09-19T09:25:29.296465+0000","last_unstale":"2024-09-19T09:27:11.341200+0000","last_undegraded":"2024-09-19T09:27:11.341200+0000","last_fullsized":"2024-09-19T09:27:11.341200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:05:48.553252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.012954+0000","last_change":"2024-09-19T09:25:43.012532+0000","last_active":"2024-09-19T09:25:43.012954+0000","last_peered":"2024-09-19T09:25:43.012954+0000","last_clean":"2024-09-19T09:25:43.012954+0000","last_became_active":"2024-09-19T09:25:42.994996+0000","last_became_peered":"2024-09-19T09:25:42.994996+0000","last_unstale":"2024-09-19T09:25:43.012954+0000","last_undegraded":"2024-09-19T09:25:43.012954+0000","last_fullsized":"2024-09-19T09:25:43.012954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:25.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337187+0000","last_change":"2024-09-19T09:25:29.297633+0000","last_active":"2024-09-19T09:27:11.337187+0000","last_peered":"2024-09-19T09:27:11.337187+0000","last_clean":"2024-09-19T09:27:11.337187+0000","last_became_active":"2024-09-19T09:25:29.297284+0000","last_became_peered":"2024-09-19T09:25:29.297284+0000","last_unstale":"2024-09-19T09:27:11.337187+0000","last_undegraded":"2024-09-19T09:27:11.337187+0000","last_fullsized":"2024-09-19T09:27:11.337187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:50.582546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991172+0000","last_change":"2024-09-19T09:25:27.281778+0000","last_active":"2024-09-19T09:25:42.991172+0000","last_peered":"2024-09-19T09:25:42.991172+0000","last_clean":"2024-09-19T09:25:42.991172+0000","last_became_active":"2024-09-19T09:25:27.281280+0000","last_became_peered":"2024-09-19T09:25:27.281280+0000","last_unstale":"2024-09-19T09:25:42.991172+0000","last_undegraded":"2024-09-19T09:25:42.991172+0000","last_fullsized":"2024-09-19T09:25:42.991172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:46.811968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343342+0000","last_change":"2024-09-19T09:25:29.295663+0000","last_active":"2024-09-19T09:27:11.343342+0000","last_peered":"2024-09-19T09:27:11.343342+0000","last_clean":"2024-09-19T09:27:11.343342+0000","last_became_active":"2024-09-19T09:25:29.295146+0000","last_became_peered":"2024-09-19T09:25:29.295146+0000","last_unstale":"2024-09-19T09:27:11.343342+0000","last_undegraded":"2024-09-19T09:27:11.343342+0000","last_fullsized":"2024-09-19T09:27:11.343342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:36.749339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.345435+0000","last_change":"2024-09-19T09:25:29.298004+0000","last_active":"2024-09-19T09:27:11.345435+0000","last_peered":"2024-09-19T09:27:11.345435+0000","last_clean":"2024-09-19T09:27:11.345435+0000","last_became_active":"2024-09-19T09:25:29.297600+0000","last_became_peered":"2024-09-19T09:25:29.297600+0000","last_unstale":"2024-09-19T09:27:11.345435+0000","last_undegraded":"2024-09-19T09:27:11.345435+0000","last_fullsized":"2024-09-19T09:27:11.345435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:09.738615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992323+0000","last_change":"2024-09-19T09:25:27.282876+0000","last_active":"2024-09-19T09:25:42.992323+0000","last_peered":"2024-09-19T09:25:42.992323+0000","last_clean":"2024-09-19T09:25:42.992323+0000","last_became_active":"2024-09-19T09:25:27.282201+0000","last_became_peered":"2024-09-19T09:25:27.282201+0000","last_unstale":"2024-09-19T09:25:42.992323+0000","last_undegraded":"2024-09-19T09:25:42.992323+0000","last_fullsized":"2024-09-19T09:25:42.992323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:08.508662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347692+0000","last_change":"2024-09-19T09:25:29.288379+0000","last_active":"2024-09-19T09:27:11.347692+0000","last_peered":"2024-09-19T09:27:11.347692+0000","last_clean":"2024-09-19T09:27:11.347692+0000","last_became_active":"2024-09-19T09:25:29.287917+0000","last_became_peered":"2024-09-19T09:25:29.287917+0000","last_unstale":"2024-09-19T09:27:11.347692+0000","last_undegraded":"2024-09-19T09:27:11.347692+0000","last_fullsized":"2024-09-19T09:27:11.347692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:03.628233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991076+0000","last_change":"2024-09-19T09:25:27.279551+0000","last_active":"2024-09-19T09:25:42.991076+0000","last_peered":"2024-09-19T09:25:42.991076+0000","last_clean":"2024-09-19T09:25:42.991076+0000","last_became_active":"2024-09-19T09:25:27.278956+0000","last_became_peered":"2024-09-19T09:25:27.278956+0000","last_unstale":"2024-09-19T09:25:42.991076+0000","last_undegraded":"2024-09-19T09:25:42.991076+0000","last_fullsized":"2024-09-19T09:25:42.991076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:17.186453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347269+0000","last_change":"2024-09-19T09:25:29.288657+0000","last_active":"2024-09-19T09:27:11.347269+0000","last_peered":"2024-09-19T09:27:11.347269+0000","last_clean":"2024-09-19T09:27:11.347269+0000","last_became_active":"2024-09-19T09:25:29.288355+0000","last_became_peered":"2024-09-19T09:25:29.288355+0000","last_unstale":"2024-09-19T09:27:11.347269+0000","last_undegraded":"2024-09-19T09:27:11.347269+0000","last_fullsized":"2024-09-19T09:27:11.347269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:46.237558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991373+0000","last_change":"2024-09-19T09:25:27.323312+0000","last_active":"2024-09-19T09:25:42.991373+0000","last_peered":"2024-09-19T09:25:42.991373+0000","last_clean":"2024-09-19T09:25:42.991373+0000","last_became_active":"2024-09-19T09:25:27.322984+0000","last_became_peered":"2024-09-19T09:25:27.322984+0000","last_unstale":"2024-09-19T09:25:42.991373+0000","last_undegraded":"2024-09-19T09:25:42.991373+0000","last_fullsized":"2024-09-19T09:25:42.991373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:48.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.327987+0000","last_change":"2024-09-19T09:25:29.301832+0000","last_active":"2024-09-19T09:27:11.327987+0000","last_peered":"2024-09-19T09:27:11.327987+0000","last_clean":"2024-09-19T09:27:11.327987+0000","last_became_active":"2024-09-19T09:25:29.301539+0000","last_became_peered":"2024-09-19T09:25:29.301539+0000","last_unstale":"2024-09-19T09:27:11.327987+0000","last_undegraded":"2024-09-19T09:27:11.327987+0000","last_fullsized":"2024-09-19T09:27:11.327987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:00.244948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'14","reported_seq":165,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:26.982182+0000","last_change":"2024-09-19T09:25:43.009668+0000","last_active":"2024-09-19T09:27:26.982182+0000","last_peered":"2024-09-19T09:27:26.982182+0000","last_clean":"2024-09-19T09:27:26.982182+0000","last_became_active":"2024-09-19T09:25:42.994966+0000","last_became_peered":"2024-09-19T09:25:42.994966+0000","last_unstale":"2024-09-19T09:27:26.982182+0000","last_undegraded":"2024-09-19T09:27:26.982182+0000","last_fullsized":"2024-09-19T09:27:26.982182+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:04:05.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74560,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1124,"num_read_kb":646,"num_write":263,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1842,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":223680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612645,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":431506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841578,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1597440,"data_stored":1087249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005102,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":917504,"data_stored":429922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234035,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":430300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462969,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1023357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":901120,"data_stored":430173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822723,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1022639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":432046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":495830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":496780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":431820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:27:31.571 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T09:27:31.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T09:27:31.838 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:32.172 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:31 smithi016 ceph-mon[30979]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:32.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.253+0000 7f39e427b640 1 -- 172.21.15.16:0/3719798368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc100e30 msgr2=0x7f39dc104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:32.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.253+0000 7f39e427b640 1 --2- 172.21.15.16:0/3719798368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc100e30 0x7f39dc104e80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009900 tx=0x7f39d002a410 comp rx=0 tx=0).stop 2024-09-19T09:27:32.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.253+0000 7f39e427b640 1 -- 172.21.15.16:0/3719798368 shutdown_connections 2024-09-19T09:27:32.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.253+0000 7f39e427b640 1 --2- 172.21.15.16:0/3719798368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc1055b0 0x7f39dc10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:32.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.253+0000 7f39e427b640 1 --2- 172.21.15.16:0/3719798368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc100e30 0x7f39dc104e80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.253+0000 7f39e427b640 1 --2- 172.21.15.16:0/3719798368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc100480 0x7f39dc100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.253+0000 7f39e427b640 1 -- 172.21.15.16:0/3719798368 >> 172.21.15.16:0/3719798368 conn(0x7f39dc077210 msgr2=0x7f39dc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.253+0000 7f39e427b640 1 -- 172.21.15.16:0/3719798368 shutdown_connections 2024-09-19T09:27:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.253+0000 7f39e427b640 1 -- 172.21.15.16:0/3719798368 wait complete. 2024-09-19T09:27:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.253+0000 7f39e427b640 1 Processor -- start 2024-09-19T09:27:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e427b640 1 -- start start 2024-09-19T09:27:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e427b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc100480 0x7f39dc198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e427b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc100e30 0x7f39dc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e427b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc1055b0 0x7f39dc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e427b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc10fd90 con 0x7f39dc100480 2024-09-19T09:27:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e427b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc10fc10 con 0x7f39dc100e30 2024-09-19T09:27:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e427b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc10ff10 con 0x7f39dc1055b0 2024-09-19T09:27:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e2017640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc100480 0x7f39dc198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:32.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e2017640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc100480 0x7f39dc198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42726/0 (socket says 172.21.15.16:42726) 2024-09-19T09:27:32.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e2017640 1 -- 172.21.15.16:0/137356326 learned_addr learned my addr 172.21.15.16:0/137356326 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:32.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e2818640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc1055b0 0x7f39dc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:32.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e1816640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc100e30 0x7f39dc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:32.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e2017640 1 -- 172.21.15.16:0/137356326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc1055b0 msgr2=0x7f39dc19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:32.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e2017640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc1055b0 0x7f39dc19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:32.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e2017640 1 -- 172.21.15.16:0/137356326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc100e30 msgr2=0x7f39dc1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:32.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e2017640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc100e30 0x7f39dc1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:32.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e2017640 1 -- 172.21.15.16:0/137356326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc19d430 con 0x7f39dc100480 2024-09-19T09:27:32.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.254+0000 7f39e2017640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc100480 0x7f39dc198480 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00c890 tx=0x7f39cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:32.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.255+0000 7f39db7fe640 1 -- 172.21.15.16:0/137356326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc004220 con 0x7f39dc100480 2024-09-19T09:27:32.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.255+0000 7f39db7fe640 1 -- 172.21.15.16:0/137356326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39cc0043c0 con 0x7f39dc100480 2024-09-19T09:27:32.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.255+0000 7f39e427b640 1 -- 172.21.15.16:0/137356326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc19d720 con 0x7f39dc100480 2024-09-19T09:27:32.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.255+0000 7f39db7fe640 1 -- 172.21.15.16:0/137356326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0049e0 con 0x7f39dc100480 2024-09-19T09:27:32.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.255+0000 7f39e427b640 1 -- 172.21.15.16:0/137356326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc1a5000 con 0x7f39dc100480 2024-09-19T09:27:32.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.256+0000 7f39e427b640 1 -- 172.21.15.16:0/137356326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39dc100860 con 0x7f39dc100480 2024-09-19T09:27:32.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.257+0000 7f39db7fe640 1 -- 172.21.15.16:0/137356326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f39cc0079b0 con 0x7f39dc100480 2024-09-19T09:27:32.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.258+0000 7f39db7fe640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f39c40a32c0 0x7f39c40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:32.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.258+0000 7f39e1816640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f39c40a32c0 0x7f39c40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:32.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.258+0000 7f39db7fe640 1 -- 172.21.15.16:0/137356326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f39cc0cf920 con 0x7f39dc100480 2024-09-19T09:27:32.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.258+0000 7f39e1816640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f39c40a32c0 0x7f39c40a5780 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f39dc102e20 tx=0x7f39d0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:32.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.261+0000 7f39db7fe640 1 -- 172.21.15.16:0/137356326 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f39cc014030 con 0x7f39dc100480 2024-09-19T09:27:32.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:31 smithi142 ceph-mon[35032]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:32.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:31 smithi097 ceph-mon[34864]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.436+0000 7f39e427b640 1 -- 172.21.15.16:0/137356326 --> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f39dc062630 con 0x7f39c40a32c0 2024-09-19T09:27:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.443+0000 7f39db7fe640 1 -- 172.21.15.16:0/137356326 <== mgr.14150 v2:172.21.15.16:6800/4287557979 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258892 (secure 0 0 0) 0x7f39dc062630 con 0x7f39c40a32c0 2024-09-19T09:27:32.448 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:32.449 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-19T09:27:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.446+0000 7f39e427b640 1 -- 172.21.15.16:0/137356326 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f39c40a32c0 msgr2=0x7f39c40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.446+0000 7f39e427b640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f39c40a32c0 0x7f39c40a5780 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f39dc102e20 tx=0x7f39d0005a20 comp rx=0 tx=0).stop 2024-09-19T09:27:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.446+0000 7f39e427b640 1 -- 172.21.15.16:0/137356326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc100480 msgr2=0x7f39dc198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.446+0000 7f39e427b640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc100480 0x7f39dc198480 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00c890 tx=0x7f39cc00cd50 comp rx=0 tx=0).stop 2024-09-19T09:27:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.446+0000 7f39e427b640 1 -- 172.21.15.16:0/137356326 shutdown_connections 2024-09-19T09:27:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.446+0000 7f39e427b640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f39c40a32c0 0x7f39c40a5780 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.446+0000 7f39e427b640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39dc1055b0 0x7f39dc19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.446+0000 7f39e427b640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc100e30 0x7f39dc1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.446+0000 7f39e427b640 1 --2- 172.21.15.16:0/137356326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39dc100480 0x7f39dc198480 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.446+0000 7f39e427b640 1 -- 172.21.15.16:0/137356326 >> 172.21.15.16:0/137356326 conn(0x7f39dc077210 msgr2=0x7f39dc103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.447+0000 7f39e427b640 1 -- 172.21.15.16:0/137356326 shutdown_connections 2024-09-19T09:27:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:32.447+0000 7f39e427b640 1 -- 172.21.15.16:0/137356326 wait complete. 2024-09-19T09:27:33.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:32 smithi142 ceph-mon[35032]: pgmap v336: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T09:27:33.129 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:32 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:33.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:32 smithi016 ceph-mon[30979]: pgmap v336: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T09:27:33.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:32 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:33.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:32 smithi097 ceph-mon[34864]: pgmap v336: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-19T09:27:33.172 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:32 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:27:33.192 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-19T09:27:31.367868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665162,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1250,"num_read_kb":755,"num_write":446,"num_write_kb":3096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1182578,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":391016,"kb_used_data":12648,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124550808,"statfs":{"total":1151940427776,"available":1151540027392,"internally_reserved":0,"allocated":12951552,"data_stored":7141086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":59,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002852"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347860+0000","last_change":"2024-09-19T09:25:29.294965+0000","last_active":"2024-09-19T09:27:11.347860+0000","last_peered":"2024-09-19T09:27:11.347860+0000","last_clean":"2024-09-19T09:27:11.347860+0000","last_became_active":"2024-09-19T09:25:29.294157+0000","last_became_peered":"2024-09-19T09:25:29.294157+0000","last_unstale":"2024-09-19T09:27:11.347860+0000","last_undegraded":"2024-09-19T09:27:11.347860+0000","last_fullsized":"2024-09-19T09:27:11.347860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:24.643835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335438+0000","last_change":"2024-09-19T09:25:29.298888+0000","last_active":"2024-09-19T09:27:11.335438+0000","last_peered":"2024-09-19T09:27:11.335438+0000","last_clean":"2024-09-19T09:27:11.335438+0000","last_became_active":"2024-09-19T09:25:29.298245+0000","last_became_peered":"2024-09-19T09:25:29.298245+0000","last_unstale":"2024-09-19T09:27:11.335438+0000","last_undegraded":"2024-09-19T09:27:11.335438+0000","last_fullsized":"2024-09-19T09:27:11.335438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:03.162342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344990+0000","last_change":"2024-09-19T09:25:29.292300+0000","last_active":"2024-09-19T09:27:11.344990+0000","last_peered":"2024-09-19T09:27:11.344990+0000","last_clean":"2024-09-19T09:27:11.344990+0000","last_became_active":"2024-09-19T09:25:29.291973+0000","last_became_peered":"2024-09-19T09:25:29.291973+0000","last_unstale":"2024-09-19T09:27:11.344990+0000","last_undegraded":"2024-09-19T09:27:11.344990+0000","last_fullsized":"2024-09-19T09:27:11.344990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:19.672778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342109+0000","last_change":"2024-09-19T09:25:29.305695+0000","last_active":"2024-09-19T09:27:11.342109+0000","last_peered":"2024-09-19T09:27:11.342109+0000","last_clean":"2024-09-19T09:27:11.342109+0000","last_became_active":"2024-09-19T09:25:29.305244+0000","last_became_peered":"2024-09-19T09:25:29.305244+0000","last_unstale":"2024-09-19T09:27:11.342109+0000","last_undegraded":"2024-09-19T09:27:11.342109+0000","last_fullsized":"2024-09-19T09:27:11.342109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:49:38.054624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336846+0000","last_change":"2024-09-19T09:25:29.295259+0000","last_active":"2024-09-19T09:27:11.336846+0000","last_peered":"2024-09-19T09:27:11.336846+0000","last_clean":"2024-09-19T09:27:11.336846+0000","last_became_active":"2024-09-19T09:25:29.294936+0000","last_became_peered":"2024-09-19T09:25:29.294936+0000","last_unstale":"2024-09-19T09:27:11.336846+0000","last_undegraded":"2024-09-19T09:27:11.336846+0000","last_fullsized":"2024-09-19T09:27:11.336846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:59.193504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336175+0000","last_change":"2024-09-19T09:25:29.299650+0000","last_active":"2024-09-19T09:27:11.336175+0000","last_peered":"2024-09-19T09:27:11.336175+0000","last_clean":"2024-09-19T09:27:11.336175+0000","last_became_active":"2024-09-19T09:25:29.299182+0000","last_became_peered":"2024-09-19T09:25:29.299182+0000","last_unstale":"2024-09-19T09:27:11.336175+0000","last_undegraded":"2024-09-19T09:27:11.336175+0000","last_fullsized":"2024-09-19T09:27:11.336175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:44.637771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342329+0000","last_change":"2024-09-19T09:25:29.302501+0000","last_active":"2024-09-19T09:27:11.342329+0000","last_peered":"2024-09-19T09:27:11.342329+0000","last_clean":"2024-09-19T09:27:11.342329+0000","last_became_active":"2024-09-19T09:25:29.301419+0000","last_became_peered":"2024-09-19T09:25:29.301419+0000","last_unstale":"2024-09-19T09:27:11.342329+0000","last_undegraded":"2024-09-19T09:27:11.342329+0000","last_fullsized":"2024-09-19T09:27:11.342329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:50.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337306+0000","last_change":"2024-09-19T09:25:29.300212+0000","last_active":"2024-09-19T09:27:11.337306+0000","last_peered":"2024-09-19T09:27:11.337306+0000","last_clean":"2024-09-19T09:27:11.337306+0000","last_became_active":"2024-09-19T09:25:29.299873+0000","last_became_peered":"2024-09-19T09:25:29.299873+0000","last_unstale":"2024-09-19T09:27:11.337306+0000","last_undegraded":"2024-09-19T09:27:11.337306+0000","last_fullsized":"2024-09-19T09:27:11.337306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:34.620314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346891+0000","last_change":"2024-09-19T09:25:29.290214+0000","last_active":"2024-09-19T09:27:11.346891+0000","last_peered":"2024-09-19T09:27:11.346891+0000","last_clean":"2024-09-19T09:27:11.346891+0000","last_became_active":"2024-09-19T09:25:29.289707+0000","last_became_peered":"2024-09-19T09:25:29.289707+0000","last_unstale":"2024-09-19T09:27:11.346891+0000","last_undegraded":"2024-09-19T09:27:11.346891+0000","last_fullsized":"2024-09-19T09:27:11.346891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:16.341340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338183+0000","last_change":"2024-09-19T09:25:29.301774+0000","last_active":"2024-09-19T09:27:11.338183+0000","last_peered":"2024-09-19T09:27:11.338183+0000","last_clean":"2024-09-19T09:27:11.338183+0000","last_became_active":"2024-09-19T09:25:29.301398+0000","last_became_peered":"2024-09-19T09:25:29.301398+0000","last_unstale":"2024-09-19T09:27:11.338183+0000","last_undegraded":"2024-09-19T09:27:11.338183+0000","last_fullsized":"2024-09-19T09:27:11.338183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:36.058038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346172+0000","last_change":"2024-09-19T09:25:29.301475+0000","last_active":"2024-09-19T09:27:11.346172+0000","last_peered":"2024-09-19T09:27:11.346172+0000","last_clean":"2024-09-19T09:27:11.346172+0000","last_became_active":"2024-09-19T09:25:29.300908+0000","last_became_peered":"2024-09-19T09:25:29.300908+0000","last_unstale":"2024-09-19T09:27:11.346172+0000","last_undegraded":"2024-09-19T09:27:11.346172+0000","last_fullsized":"2024-09-19T09:27:11.346172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:44.774127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336219+0000","last_change":"2024-09-19T09:25:42.994395+0000","last_active":"2024-09-19T09:27:11.336219+0000","last_peered":"2024-09-19T09:27:11.336219+0000","last_clean":"2024-09-19T09:27:11.336219+0000","last_became_active":"2024-09-19T09:25:42.993992+0000","last_became_peered":"2024-09-19T09:25:42.993992+0000","last_unstale":"2024-09-19T09:27:11.336219+0000","last_undegraded":"2024-09-19T09:27:11.336219+0000","last_fullsized":"2024-09-19T09:27:11.336219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:24:44.486069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.340454+0000","last_change":"2024-09-19T09:25:42.989979+0000","last_active":"2024-09-19T09:27:11.340454+0000","last_peered":"2024-09-19T09:27:11.340454+0000","last_clean":"2024-09-19T09:27:11.340454+0000","last_became_active":"2024-09-19T09:25:42.989578+0000","last_became_peered":"2024-09-19T09:25:42.989578+0000","last_unstale":"2024-09-19T09:27:11.340454+0000","last_undegraded":"2024-09-19T09:27:11.340454+0000","last_fullsized":"2024-09-19T09:27:11.340454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:04.583023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336926+0000","last_change":"2024-09-19T09:25:29.299917+0000","last_active":"2024-09-19T09:27:11.336926+0000","last_peered":"2024-09-19T09:27:11.336926+0000","last_clean":"2024-09-19T09:27:11.336926+0000","last_became_active":"2024-09-19T09:25:29.299571+0000","last_became_peered":"2024-09-19T09:25:29.299571+0000","last_unstale":"2024-09-19T09:27:11.336926+0000","last_undegraded":"2024-09-19T09:27:11.336926+0000","last_fullsized":"2024-09-19T09:27:11.336926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:25.142252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335108+0000","last_change":"2024-09-19T09:25:29.299713+0000","last_active":"2024-09-19T09:27:11.335108+0000","last_peered":"2024-09-19T09:27:11.335108+0000","last_clean":"2024-09-19T09:27:11.335108+0000","last_became_active":"2024-09-19T09:25:29.299234+0000","last_became_peered":"2024-09-19T09:25:29.299234+0000","last_unstale":"2024-09-19T09:27:11.335108+0000","last_undegraded":"2024-09-19T09:27:11.335108+0000","last_fullsized":"2024-09-19T09:27:11.335108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:34.714714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346606+0000","last_change":"2024-09-19T09:25:29.293010+0000","last_active":"2024-09-19T09:27:11.346606+0000","last_peered":"2024-09-19T09:27:11.346606+0000","last_clean":"2024-09-19T09:27:11.346606+0000","last_became_active":"2024-09-19T09:25:29.292645+0000","last_became_peered":"2024-09-19T09:25:29.292645+0000","last_unstale":"2024-09-19T09:27:11.346606+0000","last_undegraded":"2024-09-19T09:27:11.346606+0000","last_fullsized":"2024-09-19T09:27:11.346606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:24.749075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347900+0000","last_change":"2024-09-19T09:25:29.294530+0000","last_active":"2024-09-19T09:27:11.347900+0000","last_peered":"2024-09-19T09:27:11.347900+0000","last_clean":"2024-09-19T09:27:11.347900+0000","last_became_active":"2024-09-19T09:25:29.293699+0000","last_became_peered":"2024-09-19T09:25:29.293699+0000","last_unstale":"2024-09-19T09:27:11.347900+0000","last_undegraded":"2024-09-19T09:27:11.347900+0000","last_fullsized":"2024-09-19T09:27:11.347900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:53.491258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336452+0000","last_change":"2024-09-19T09:25:29.290600+0000","last_active":"2024-09-19T09:27:11.336452+0000","last_peered":"2024-09-19T09:27:11.336452+0000","last_clean":"2024-09-19T09:27:11.336452+0000","last_became_active":"2024-09-19T09:25:29.290305+0000","last_became_peered":"2024-09-19T09:25:29.290305+0000","last_unstale":"2024-09-19T09:27:11.336452+0000","last_undegraded":"2024-09-19T09:27:11.336452+0000","last_fullsized":"2024-09-19T09:27:11.336452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:42.809659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338092+0000","last_change":"2024-09-19T09:25:29.300591+0000","last_active":"2024-09-19T09:27:11.338092+0000","last_peered":"2024-09-19T09:27:11.338092+0000","last_clean":"2024-09-19T09:27:11.338092+0000","last_became_active":"2024-09-19T09:25:29.300251+0000","last_became_peered":"2024-09-19T09:25:29.300251+0000","last_unstale":"2024-09-19T09:27:11.338092+0000","last_undegraded":"2024-09-19T09:27:11.338092+0000","last_fullsized":"2024-09-19T09:27:11.338092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:08.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342064+0000","last_change":"2024-09-19T09:25:29.298452+0000","last_active":"2024-09-19T09:27:11.342064+0000","last_peered":"2024-09-19T09:27:11.342064+0000","last_clean":"2024-09-19T09:27:11.342064+0000","last_became_active":"2024-09-19T09:25:29.298137+0000","last_became_peered":"2024-09-19T09:25:29.298137+0000","last_unstale":"2024-09-19T09:27:11.342064+0000","last_undegraded":"2024-09-19T09:27:11.342064+0000","last_fullsized":"2024-09-19T09:27:11.342064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:37:11.716228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337968+0000","last_change":"2024-09-19T09:25:29.294278+0000","last_active":"2024-09-19T09:27:11.337968+0000","last_peered":"2024-09-19T09:27:11.337968+0000","last_clean":"2024-09-19T09:27:11.337968+0000","last_became_active":"2024-09-19T09:25:29.293972+0000","last_became_peered":"2024-09-19T09:25:29.293972+0000","last_unstale":"2024-09-19T09:27:11.337968+0000","last_undegraded":"2024-09-19T09:27:11.337968+0000","last_fullsized":"2024-09-19T09:27:11.337968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:05:27.923839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346634+0000","last_change":"2024-09-19T09:25:29.294000+0000","last_active":"2024-09-19T09:27:11.346634+0000","last_peered":"2024-09-19T09:27:11.346634+0000","last_clean":"2024-09-19T09:27:11.346634+0000","last_became_active":"2024-09-19T09:25:29.293486+0000","last_became_peered":"2024-09-19T09:25:29.293486+0000","last_unstale":"2024-09-19T09:27:11.346634+0000","last_undegraded":"2024-09-19T09:27:11.346634+0000","last_fullsized":"2024-09-19T09:27:11.346634+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:44.053620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334040+0000","last_change":"2024-09-19T09:25:42.994951+0000","last_active":"2024-09-19T09:27:11.334040+0000","last_peered":"2024-09-19T09:27:11.334040+0000","last_clean":"2024-09-19T09:27:11.334040+0000","last_became_active":"2024-09-19T09:25:42.994588+0000","last_became_peered":"2024-09-19T09:25:42.994588+0000","last_unstale":"2024-09-19T09:27:11.334040+0000","last_undegraded":"2024-09-19T09:27:11.334040+0000","last_fullsized":"2024-09-19T09:27:11.334040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:22:10.141169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334658+0000","last_change":"2024-09-19T09:25:29.286395+0000","last_active":"2024-09-19T09:27:11.334658+0000","last_peered":"2024-09-19T09:27:11.334658+0000","last_clean":"2024-09-19T09:27:11.334658+0000","last_became_active":"2024-09-19T09:25:29.286048+0000","last_became_peered":"2024-09-19T09:25:29.286048+0000","last_unstale":"2024-09-19T09:27:11.334658+0000","last_undegraded":"2024-09-19T09:27:11.334658+0000","last_fullsized":"2024-09-19T09:27:11.334658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:43.121166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.327390+0000","last_change":"2024-09-19T09:25:29.298458+0000","last_active":"2024-09-19T09:27:11.327390+0000","last_peered":"2024-09-19T09:27:11.327390+0000","last_clean":"2024-09-19T09:27:11.327390+0000","last_became_active":"2024-09-19T09:25:29.297993+0000","last_became_peered":"2024-09-19T09:25:29.297993+0000","last_unstale":"2024-09-19T09:27:11.327390+0000","last_undegraded":"2024-09-19T09:27:11.327390+0000","last_fullsized":"2024-09-19T09:27:11.327390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:04:44.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344574+0000","last_change":"2024-09-19T09:25:29.297167+0000","last_active":"2024-09-19T09:27:11.344574+0000","last_peered":"2024-09-19T09:27:11.344574+0000","last_clean":"2024-09-19T09:27:11.344574+0000","last_became_active":"2024-09-19T09:25:29.296777+0000","last_became_peered":"2024-09-19T09:25:29.296777+0000","last_unstale":"2024-09-19T09:27:11.344574+0000","last_undegraded":"2024-09-19T09:27:11.344574+0000","last_fullsized":"2024-09-19T09:27:11.344574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:40.581643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.339599+0000","last_change":"2024-09-19T09:25:29.296169+0000","last_active":"2024-09-19T09:27:11.339599+0000","last_peered":"2024-09-19T09:27:11.339599+0000","last_clean":"2024-09-19T09:27:11.339599+0000","last_became_active":"2024-09-19T09:25:29.295806+0000","last_became_peered":"2024-09-19T09:25:29.295806+0000","last_unstale":"2024-09-19T09:27:11.339599+0000","last_undegraded":"2024-09-19T09:27:11.339599+0000","last_fullsized":"2024-09-19T09:27:11.339599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:39.639884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334590+0000","last_change":"2024-09-19T09:25:29.297960+0000","last_active":"2024-09-19T09:27:11.334590+0000","last_peered":"2024-09-19T09:27:11.334590+0000","last_clean":"2024-09-19T09:27:11.334590+0000","last_became_active":"2024-09-19T09:25:29.297577+0000","last_became_peered":"2024-09-19T09:25:29.297577+0000","last_unstale":"2024-09-19T09:27:11.334590+0000","last_undegraded":"2024-09-19T09:27:11.334590+0000","last_fullsized":"2024-09-19T09:27:11.334590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:27.035504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335188+0000","last_change":"2024-09-19T09:25:29.297896+0000","last_active":"2024-09-19T09:27:11.335188+0000","last_peered":"2024-09-19T09:27:11.335188+0000","last_clean":"2024-09-19T09:27:11.335188+0000","last_became_active":"2024-09-19T09:25:29.297594+0000","last_became_peered":"2024-09-19T09:25:29.297594+0000","last_unstale":"2024-09-19T09:27:11.335188+0000","last_undegraded":"2024-09-19T09:27:11.335188+0000","last_fullsized":"2024-09-19T09:27:11.335188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:14.366325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334560+0000","last_change":"2024-09-19T09:25:29.301851+0000","last_active":"2024-09-19T09:27:11.334560+0000","last_peered":"2024-09-19T09:27:11.334560+0000","last_clean":"2024-09-19T09:27:11.334560+0000","last_became_active":"2024-09-19T09:25:29.299282+0000","last_became_peered":"2024-09-19T09:25:29.299282+0000","last_unstale":"2024-09-19T09:27:11.334560+0000","last_undegraded":"2024-09-19T09:27:11.334560+0000","last_fullsized":"2024-09-19T09:27:11.334560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:00.382114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.342990+0000","last_change":"2024-09-19T09:25:29.303316+0000","last_active":"2024-09-19T09:27:11.342990+0000","last_peered":"2024-09-19T09:27:11.342990+0000","last_clean":"2024-09-19T09:27:11.342990+0000","last_became_active":"2024-09-19T09:25:29.302994+0000","last_became_peered":"2024-09-19T09:25:29.302994+0000","last_unstale":"2024-09-19T09:27:11.342990+0000","last_undegraded":"2024-09-19T09:27:11.342990+0000","last_fullsized":"2024-09-19T09:27:11.342990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:46.233991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341767+0000","last_change":"2024-09-19T09:25:29.293004+0000","last_active":"2024-09-19T09:27:11.341767+0000","last_peered":"2024-09-19T09:27:11.341767+0000","last_clean":"2024-09-19T09:27:11.341767+0000","last_became_active":"2024-09-19T09:25:29.292657+0000","last_became_peered":"2024-09-19T09:25:29.292657+0000","last_unstale":"2024-09-19T09:27:11.341767+0000","last_undegraded":"2024-09-19T09:27:11.341767+0000","last_fullsized":"2024-09-19T09:27:11.341767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:50.966477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.332473+0000","last_change":"2024-09-19T09:25:29.300434+0000","last_active":"2024-09-19T09:27:11.332473+0000","last_peered":"2024-09-19T09:27:11.332473+0000","last_clean":"2024-09-19T09:27:11.332473+0000","last_became_active":"2024-09-19T09:25:29.300053+0000","last_became_peered":"2024-09-19T09:25:29.300053+0000","last_unstale":"2024-09-19T09:27:11.332473+0000","last_undegraded":"2024-09-19T09:27:11.332473+0000","last_fullsized":"2024-09-19T09:27:11.332473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:13.283298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":142,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:26.121860+0000","last_change":"2024-09-19T09:25:27.280792+0000","last_active":"2024-09-19T09:27:26.121860+0000","last_peered":"2024-09-19T09:27:26.121860+0000","last_clean":"2024-09-19T09:27:26.121860+0000","last_became_active":"2024-09-19T09:25:27.280443+0000","last_became_peered":"2024-09-19T09:25:27.280443+0000","last_unstale":"2024-09-19T09:27:26.121860+0000","last_undegraded":"2024-09-19T09:27:26.121860+0000","last_fullsized":"2024-09-19T09:27:26.121860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.599512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334165+0000","last_change":"2024-09-19T09:25:29.298975+0000","last_active":"2024-09-19T09:27:11.334165+0000","last_peered":"2024-09-19T09:27:11.334165+0000","last_clean":"2024-09-19T09:27:11.334165+0000","last_became_active":"2024-09-19T09:25:29.298476+0000","last_became_peered":"2024-09-19T09:25:29.298476+0000","last_unstale":"2024-09-19T09:27:11.334165+0000","last_undegraded":"2024-09-19T09:27:11.334165+0000","last_fullsized":"2024-09-19T09:27:11.334165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:09.658894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.986623+0000","last_change":"2024-09-19T09:25:27.278257+0000","last_active":"2024-09-19T09:25:42.986623+0000","last_peered":"2024-09-19T09:25:42.986623+0000","last_clean":"2024-09-19T09:25:42.986623+0000","last_became_active":"2024-09-19T09:25:27.277897+0000","last_became_peered":"2024-09-19T09:25:27.277897+0000","last_unstale":"2024-09-19T09:25:42.986623+0000","last_undegraded":"2024-09-19T09:25:42.986623+0000","last_fullsized":"2024-09-19T09:25:42.986623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:47.550314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985617+0000","last_change":"2024-09-19T09:25:27.272215+0000","last_active":"2024-09-19T09:25:42.985617+0000","last_peered":"2024-09-19T09:25:42.985617+0000","last_clean":"2024-09-19T09:25:42.985617+0000","last_became_active":"2024-09-19T09:25:27.271821+0000","last_became_peered":"2024-09-19T09:25:27.271821+0000","last_unstale":"2024-09-19T09:25:42.985617+0000","last_undegraded":"2024-09-19T09:25:42.985617+0000","last_fullsized":"2024-09-19T09:25:42.985617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:30.615730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.329744+0000","last_change":"2024-09-19T09:25:29.289347+0000","last_active":"2024-09-19T09:27:11.329744+0000","last_peered":"2024-09-19T09:27:11.329744+0000","last_clean":"2024-09-19T09:27:11.329744+0000","last_became_active":"2024-09-19T09:25:29.289015+0000","last_became_peered":"2024-09-19T09:25:29.289015+0000","last_unstale":"2024-09-19T09:27:11.329744+0000","last_undegraded":"2024-09-19T09:27:11.329744+0000","last_fullsized":"2024-09-19T09:27:11.329744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:18.419334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995432+0000","last_change":"2024-09-19T09:25:27.277342+0000","last_active":"2024-09-19T09:25:42.995432+0000","last_peered":"2024-09-19T09:25:42.995432+0000","last_clean":"2024-09-19T09:25:42.995432+0000","last_became_active":"2024-09-19T09:25:27.277016+0000","last_became_peered":"2024-09-19T09:25:27.277016+0000","last_unstale":"2024-09-19T09:25:42.995432+0000","last_undegraded":"2024-09-19T09:25:42.995432+0000","last_fullsized":"2024-09-19T09:25:42.995432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:56.736156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.336629+0000","last_change":"2024-09-19T09:25:29.290924+0000","last_active":"2024-09-19T09:27:11.336629+0000","last_peered":"2024-09-19T09:27:11.336629+0000","last_clean":"2024-09-19T09:27:11.336629+0000","last_became_active":"2024-09-19T09:25:29.290482+0000","last_became_peered":"2024-09-19T09:25:29.290482+0000","last_unstale":"2024-09-19T09:27:11.336629+0000","last_undegraded":"2024-09-19T09:27:11.336629+0000","last_fullsized":"2024-09-19T09:27:11.336629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:16.265040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'6","reported_seq":242,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:26.121271+0000","last_change":"2024-09-19T09:25:27.277604+0000","last_active":"2024-09-19T09:27:26.121271+0000","last_peered":"2024-09-19T09:27:26.121271+0000","last_clean":"2024-09-19T09:27:26.121271+0000","last_became_active":"2024-09-19T09:25:27.277271+0000","last_became_peered":"2024-09-19T09:25:27.277271+0000","last_unstale":"2024-09-19T09:27:26.121271+0000","last_undegraded":"2024-09-19T09:27:26.121271+0000","last_fullsized":"2024-09-19T09:27:26.121271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:32.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347371+0000","last_change":"2024-09-19T09:25:29.290537+0000","last_active":"2024-09-19T09:27:11.347371+0000","last_peered":"2024-09-19T09:27:11.347371+0000","last_clean":"2024-09-19T09:27:11.347371+0000","last_became_active":"2024-09-19T09:25:29.287984+0000","last_became_peered":"2024-09-19T09:25:29.287984+0000","last_unstale":"2024-09-19T09:27:11.347371+0000","last_undegraded":"2024-09-19T09:27:11.347371+0000","last_fullsized":"2024-09-19T09:27:11.347371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:26.133461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990018+0000","last_change":"2024-09-19T09:25:27.278560+0000","last_active":"2024-09-19T09:25:42.990018+0000","last_peered":"2024-09-19T09:25:42.990018+0000","last_clean":"2024-09-19T09:25:42.990018+0000","last_became_active":"2024-09-19T09:25:27.277950+0000","last_became_peered":"2024-09-19T09:25:27.277950+0000","last_unstale":"2024-09-19T09:25:42.990018+0000","last_undegraded":"2024-09-19T09:25:42.990018+0000","last_fullsized":"2024-09-19T09:25:42.990018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:28.387725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.339317+0000","last_change":"2024-09-19T09:25:29.298897+0000","last_active":"2024-09-19T09:27:11.339317+0000","last_peered":"2024-09-19T09:27:11.339317+0000","last_clean":"2024-09-19T09:27:11.339317+0000","last_became_active":"2024-09-19T09:25:29.298617+0000","last_became_peered":"2024-09-19T09:25:29.298617+0000","last_unstale":"2024-09-19T09:27:11.339317+0000","last_undegraded":"2024-09-19T09:27:11.339317+0000","last_fullsized":"2024-09-19T09:27:11.339317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:53.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.994870+0000","last_change":"2024-09-19T09:25:27.324871+0000","last_active":"2024-09-19T09:25:42.994870+0000","last_peered":"2024-09-19T09:25:42.994870+0000","last_clean":"2024-09-19T09:25:42.994870+0000","last_became_active":"2024-09-19T09:25:27.324471+0000","last_became_peered":"2024-09-19T09:25:27.324471+0000","last_unstale":"2024-09-19T09:25:42.994870+0000","last_undegraded":"2024-09-19T09:25:42.994870+0000","last_fullsized":"2024-09-19T09:25:42.994870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:45:35.936413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347260+0000","last_change":"2024-09-19T09:25:29.298332+0000","last_active":"2024-09-19T09:27:11.347260+0000","last_peered":"2024-09-19T09:27:11.347260+0000","last_clean":"2024-09-19T09:27:11.347260+0000","last_became_active":"2024-09-19T09:25:29.298006+0000","last_became_peered":"2024-09-19T09:25:29.298006+0000","last_unstale":"2024-09-19T09:27:11.347260+0000","last_undegraded":"2024-09-19T09:27:11.347260+0000","last_fullsized":"2024-09-19T09:27:11.347260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:02.621821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'35","reported_seq":285,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:26.121266+0000","last_change":"2024-09-19T09:25:27.279528+0000","last_active":"2024-09-19T09:27:26.121266+0000","last_peered":"2024-09-19T09:27:26.121266+0000","last_clean":"2024-09-19T09:27:26.121266+0000","last_became_active":"2024-09-19T09:25:27.279149+0000","last_became_peered":"2024-09-19T09:25:27.279149+0000","last_unstale":"2024-09-19T09:27:26.121266+0000","last_undegraded":"2024-09-19T09:27:26.121266+0000","last_fullsized":"2024-09-19T09:27:26.121266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:47.030736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65694,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":228,"num_read_kb":178,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.324455+0000","last_change":"2024-09-19T09:25:29.297538+0000","last_active":"2024-09-19T09:27:11.324455+0000","last_peered":"2024-09-19T09:27:11.324455+0000","last_clean":"2024-09-19T09:27:11.324455+0000","last_became_active":"2024-09-19T09:25:29.297181+0000","last_became_peered":"2024-09-19T09:25:29.297181+0000","last_unstale":"2024-09-19T09:27:11.324455+0000","last_undegraded":"2024-09-19T09:27:11.324455+0000","last_fullsized":"2024-09-19T09:27:11.324455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:53.135438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'9","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992860+0000","last_change":"2024-09-19T09:25:27.277090+0000","last_active":"2024-09-19T09:25:42.992860+0000","last_peered":"2024-09-19T09:25:42.992860+0000","last_clean":"2024-09-19T09:25:42.992860+0000","last_became_active":"2024-09-19T09:25:27.276718+0000","last_became_peered":"2024-09-19T09:25:27.276718+0000","last_unstale":"2024-09-19T09:25:42.992860+0000","last_undegraded":"2024-09-19T09:25:42.992860+0000","last_fullsized":"2024-09-19T09:25:42.992860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:25.664261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.325840+0000","last_change":"2024-09-19T09:25:29.294393+0000","last_active":"2024-09-19T09:27:11.325840+0000","last_peered":"2024-09-19T09:27:11.325840+0000","last_clean":"2024-09-19T09:27:11.325840+0000","last_became_active":"2024-09-19T09:25:29.294099+0000","last_became_peered":"2024-09-19T09:25:29.294099+0000","last_unstale":"2024-09-19T09:27:11.325840+0000","last_undegraded":"2024-09-19T09:27:11.325840+0000","last_fullsized":"2024-09-19T09:27:11.325840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:17.333736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.995398+0000","last_change":"2024-09-19T09:25:27.278593+0000","last_active":"2024-09-19T09:25:42.995398+0000","last_peered":"2024-09-19T09:25:42.995398+0000","last_clean":"2024-09-19T09:25:42.995398+0000","last_became_active":"2024-09-19T09:25:27.278232+0000","last_became_peered":"2024-09-19T09:25:27.278232+0000","last_unstale":"2024-09-19T09:25:42.995398+0000","last_undegraded":"2024-09-19T09:25:42.995398+0000","last_fullsized":"2024-09-19T09:25:42.995398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:05.852671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343581+0000","last_change":"2024-09-19T09:25:29.300024+0000","last_active":"2024-09-19T09:27:11.343581+0000","last_peered":"2024-09-19T09:27:11.343581+0000","last_clean":"2024-09-19T09:27:11.343581+0000","last_became_active":"2024-09-19T09:25:29.299683+0000","last_became_peered":"2024-09-19T09:25:29.299683+0000","last_unstale":"2024-09-19T09:27:11.343581+0000","last_undegraded":"2024-09-19T09:27:11.343581+0000","last_fullsized":"2024-09-19T09:27:11.343581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:42.249399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'7","reported_seq":141,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:25.120627+0000","last_change":"2024-09-19T09:25:27.272739+0000","last_active":"2024-09-19T09:27:25.120627+0000","last_peered":"2024-09-19T09:27:25.120627+0000","last_clean":"2024-09-19T09:27:25.120627+0000","last_became_active":"2024-09-19T09:25:27.272354+0000","last_became_peered":"2024-09-19T09:25:27.272354+0000","last_unstale":"2024-09-19T09:27:25.120627+0000","last_undegraded":"2024-09-19T09:27:25.120627+0000","last_fullsized":"2024-09-19T09:27:25.120627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:27.863840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343011+0000","last_change":"2024-09-19T09:25:29.303460+0000","last_active":"2024-09-19T09:27:11.343011+0000","last_peered":"2024-09-19T09:27:11.343011+0000","last_clean":"2024-09-19T09:27:11.343011+0000","last_became_active":"2024-09-19T09:25:29.303127+0000","last_became_peered":"2024-09-19T09:25:29.303127+0000","last_unstale":"2024-09-19T09:27:11.343011+0000","last_undegraded":"2024-09-19T09:27:11.343011+0000","last_fullsized":"2024-09-19T09:27:11.343011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:35.313924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'17","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990876+0000","last_change":"2024-09-19T09:25:27.323238+0000","last_active":"2024-09-19T09:25:42.990876+0000","last_peered":"2024-09-19T09:25:42.990876+0000","last_clean":"2024-09-19T09:25:42.990876+0000","last_became_active":"2024-09-19T09:25:27.322868+0000","last_became_peered":"2024-09-19T09:25:27.322868+0000","last_unstale":"2024-09-19T09:25:42.990876+0000","last_undegraded":"2024-09-19T09:25:42.990876+0000","last_fullsized":"2024-09-19T09:25:42.990876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:22.890280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338463+0000","last_change":"2024-09-19T09:25:43.005569+0000","last_active":"2024-09-19T09:27:11.338463+0000","last_peered":"2024-09-19T09:27:11.338463+0000","last_clean":"2024-09-19T09:27:11.338463+0000","last_became_active":"2024-09-19T09:25:43.005143+0000","last_became_peered":"2024-09-19T09:25:43.005143+0000","last_unstale":"2024-09-19T09:27:11.338463+0000","last_undegraded":"2024-09-19T09:27:11.338463+0000","last_fullsized":"2024-09-19T09:27:11.338463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:26.297013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991438+0000","last_change":"2024-09-19T09:25:27.282737+0000","last_active":"2024-09-19T09:25:42.991438+0000","last_peered":"2024-09-19T09:25:42.991438+0000","last_clean":"2024-09-19T09:25:42.991438+0000","last_became_active":"2024-09-19T09:25:27.281785+0000","last_became_peered":"2024-09-19T09:25:27.281785+0000","last_unstale":"2024-09-19T09:25:42.991438+0000","last_undegraded":"2024-09-19T09:25:42.991438+0000","last_fullsized":"2024-09-19T09:25:42.991438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:41.552206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.340095+0000","last_change":"2024-09-19T09:25:29.300168+0000","last_active":"2024-09-19T09:27:11.340095+0000","last_peered":"2024-09-19T09:27:11.340095+0000","last_clean":"2024-09-19T09:27:11.340095+0000","last_became_active":"2024-09-19T09:25:29.299811+0000","last_became_peered":"2024-09-19T09:25:29.299811+0000","last_unstale":"2024-09-19T09:27:11.340095+0000","last_undegraded":"2024-09-19T09:27:11.340095+0000","last_fullsized":"2024-09-19T09:27:11.340095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:06.691259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.985783+0000","last_change":"2024-09-19T09:25:27.278433+0000","last_active":"2024-09-19T09:25:42.985783+0000","last_peered":"2024-09-19T09:25:42.985783+0000","last_clean":"2024-09-19T09:25:42.985783+0000","last_became_active":"2024-09-19T09:25:27.278076+0000","last_became_peered":"2024-09-19T09:25:27.278076+0000","last_unstale":"2024-09-19T09:25:42.985783+0000","last_undegraded":"2024-09-19T09:25:42.985783+0000","last_fullsized":"2024-09-19T09:25:42.985783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:53.799265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338515+0000","last_change":"2024-09-19T09:25:29.295650+0000","last_active":"2024-09-19T09:27:11.338515+0000","last_peered":"2024-09-19T09:27:11.338515+0000","last_clean":"2024-09-19T09:27:11.338515+0000","last_became_active":"2024-09-19T09:25:29.295277+0000","last_became_peered":"2024-09-19T09:25:29.295277+0000","last_unstale":"2024-09-19T09:27:11.338515+0000","last_undegraded":"2024-09-19T09:27:11.338515+0000","last_fullsized":"2024-09-19T09:27:11.338515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:52:57.690699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.996123+0000","last_change":"2024-09-19T09:25:27.279239+0000","last_active":"2024-09-19T09:25:42.996123+0000","last_peered":"2024-09-19T09:25:42.996123+0000","last_clean":"2024-09-19T09:25:42.996123+0000","last_became_active":"2024-09-19T09:25:27.278868+0000","last_became_peered":"2024-09-19T09:25:27.278868+0000","last_unstale":"2024-09-19T09:25:42.996123+0000","last_undegraded":"2024-09-19T09:25:42.996123+0000","last_fullsized":"2024-09-19T09:25:42.996123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:41.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341911+0000","last_change":"2024-09-19T09:25:29.300833+0000","last_active":"2024-09-19T09:27:11.341911+0000","last_peered":"2024-09-19T09:27:11.341911+0000","last_clean":"2024-09-19T09:27:11.341911+0000","last_became_active":"2024-09-19T09:25:29.300472+0000","last_became_peered":"2024-09-19T09:25:29.300472+0000","last_unstale":"2024-09-19T09:27:11.341911+0000","last_undegraded":"2024-09-19T09:27:11.341911+0000","last_fullsized":"2024-09-19T09:27:11.341911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:50.591276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'8","reported_seq":129,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:25.156716+0000","last_change":"2024-09-19T09:25:43.001722+0000","last_active":"2024-09-19T09:27:25.156716+0000","last_peered":"2024-09-19T09:27:25.156716+0000","last_clean":"2024-09-19T09:27:25.156716+0000","last_became_active":"2024-09-19T09:25:42.992796+0000","last_became_peered":"2024-09-19T09:25:42.992796+0000","last_unstale":"2024-09-19T09:27:25.156716+0000","last_undegraded":"2024-09-19T09:27:25.156716+0000","last_fullsized":"2024-09-19T09:27:25.156716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:59.639010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2012,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.346495+0000","last_change":"2024-09-19T09:25:29.295906+0000","last_active":"2024-09-19T09:27:11.346495+0000","last_peered":"2024-09-19T09:27:11.346495+0000","last_clean":"2024-09-19T09:27:11.346495+0000","last_became_active":"2024-09-19T09:25:29.295565+0000","last_became_peered":"2024-09-19T09:25:29.295565+0000","last_unstale":"2024-09-19T09:27:11.346495+0000","last_undegraded":"2024-09-19T09:27:11.346495+0000","last_fullsized":"2024-09-19T09:27:11.346495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:24.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":335,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989907+0000","last_change":"2024-09-19T09:24:19.256300+0000","last_active":"2024-09-19T09:25:42.989907+0000","last_peered":"2024-09-19T09:25:42.989907+0000","last_clean":"2024-09-19T09:25:42.989907+0000","last_became_active":"2024-09-19T09:24:19.255833+0000","last_became_peered":"2024-09-19T09:24:19.255833+0000","last_unstale":"2024-09-19T09:25:42.989907+0000","last_undegraded":"2024-09-19T09:25:42.989907+0000","last_fullsized":"2024-09-19T09:25:42.989907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:20:57.798838+0000","last_clean_scrub_stamp":"2024-09-19T09:20:57.798838+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:14:33.343748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'6","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.989637+0000","last_change":"2024-09-19T09:25:27.272090+0000","last_active":"2024-09-19T09:25:42.989637+0000","last_peered":"2024-09-19T09:25:42.989637+0000","last_clean":"2024-09-19T09:25:42.989637+0000","last_became_active":"2024-09-19T09:25:27.271651+0000","last_became_peered":"2024-09-19T09:25:27.271651+0000","last_unstale":"2024-09-19T09:25:42.989637+0000","last_undegraded":"2024-09-19T09:25:42.989637+0000","last_fullsized":"2024-09-19T09:25:42.989637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:45.847082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.344375+0000","last_change":"2024-09-19T09:25:29.300718+0000","last_active":"2024-09-19T09:27:11.344375+0000","last_peered":"2024-09-19T09:27:11.344375+0000","last_clean":"2024-09-19T09:27:11.344375+0000","last_became_active":"2024-09-19T09:25:29.299460+0000","last_became_peered":"2024-09-19T09:25:29.299460+0000","last_unstale":"2024-09-19T09:27:11.344375+0000","last_undegraded":"2024-09-19T09:27:11.344375+0000","last_fullsized":"2024-09-19T09:27:11.344375+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:43.063909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'4","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003919+0000","last_change":"2024-09-19T09:25:43.003299+0000","last_active":"2024-09-19T09:25:43.003919+0000","last_peered":"2024-09-19T09:25:43.003919+0000","last_clean":"2024-09-19T09:25:43.003919+0000","last_became_active":"2024-09-19T09:25:42.999686+0000","last_became_peered":"2024-09-19T09:25:42.999686+0000","last_unstale":"2024-09-19T09:25:43.003919+0000","last_undegraded":"2024-09-19T09:25:43.003919+0000","last_fullsized":"2024-09-19T09:25:43.003919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:05.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337734+0000","last_change":"2024-09-19T09:25:29.297833+0000","last_active":"2024-09-19T09:27:11.337734+0000","last_peered":"2024-09-19T09:27:11.337734+0000","last_clean":"2024-09-19T09:27:11.337734+0000","last_became_active":"2024-09-19T09:25:29.297534+0000","last_became_peered":"2024-09-19T09:25:29.297534+0000","last_unstale":"2024-09-19T09:27:11.337734+0000","last_undegraded":"2024-09-19T09:27:11.337734+0000","last_fullsized":"2024-09-19T09:27:11.337734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:47.156253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'6","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:27.122980+0000","last_change":"2024-09-19T09:25:27.322759+0000","last_active":"2024-09-19T09:27:27.122980+0000","last_peered":"2024-09-19T09:27:27.122980+0000","last_clean":"2024-09-19T09:27:27.122980+0000","last_became_active":"2024-09-19T09:25:27.322402+0000","last_became_peered":"2024-09-19T09:25:27.322402+0000","last_unstale":"2024-09-19T09:27:27.122980+0000","last_undegraded":"2024-09-19T09:27:27.122980+0000","last_fullsized":"2024-09-19T09:27:27.122980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:06.276957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.335476+0000","last_change":"2024-09-19T09:25:42.999834+0000","last_active":"2024-09-19T09:27:11.335476+0000","last_peered":"2024-09-19T09:27:11.335476+0000","last_clean":"2024-09-19T09:27:11.335476+0000","last_became_active":"2024-09-19T09:25:42.999450+0000","last_became_peered":"2024-09-19T09:25:42.999450+0000","last_unstale":"2024-09-19T09:27:11.335476+0000","last_undegraded":"2024-09-19T09:27:11.335476+0000","last_fullsized":"2024-09-19T09:27:11.335476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:00.509844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'7","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:26.129531+0000","last_change":"2024-09-19T09:25:27.275212+0000","last_active":"2024-09-19T09:27:26.129531+0000","last_peered":"2024-09-19T09:27:26.129531+0000","last_clean":"2024-09-19T09:27:26.129531+0000","last_became_active":"2024-09-19T09:25:27.274802+0000","last_became_peered":"2024-09-19T09:25:27.274802+0000","last_unstale":"2024-09-19T09:27:26.129531+0000","last_undegraded":"2024-09-19T09:27:26.129531+0000","last_fullsized":"2024-09-19T09:27:26.129531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:48.392026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.333364+0000","last_change":"2024-09-19T09:25:29.304747+0000","last_active":"2024-09-19T09:27:11.333364+0000","last_peered":"2024-09-19T09:27:11.333364+0000","last_clean":"2024-09-19T09:27:11.333364+0000","last_became_active":"2024-09-19T09:25:29.304386+0000","last_became_peered":"2024-09-19T09:25:29.304386+0000","last_unstale":"2024-09-19T09:27:11.333364+0000","last_undegraded":"2024-09-19T09:27:11.333364+0000","last_fullsized":"2024-09-19T09:27:11.333364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:03.788697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.988854+0000","last_change":"2024-09-19T09:25:27.323083+0000","last_active":"2024-09-19T09:25:42.988854+0000","last_peered":"2024-09-19T09:25:42.988854+0000","last_clean":"2024-09-19T09:25:42.988854+0000","last_became_active":"2024-09-19T09:25:27.322707+0000","last_became_peered":"2024-09-19T09:25:27.322707+0000","last_unstale":"2024-09-19T09:25:42.988854+0000","last_undegraded":"2024-09-19T09:25:42.988854+0000","last_fullsized":"2024-09-19T09:25:42.988854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:24.415116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334132+0000","last_change":"2024-09-19T09:25:29.292472+0000","last_active":"2024-09-19T09:27:11.334132+0000","last_peered":"2024-09-19T09:27:11.334132+0000","last_clean":"2024-09-19T09:27:11.334132+0000","last_became_active":"2024-09-19T09:25:29.292079+0000","last_became_peered":"2024-09-19T09:25:29.292079+0000","last_unstale":"2024-09-19T09:27:11.334132+0000","last_undegraded":"2024-09-19T09:27:11.334132+0000","last_fullsized":"2024-09-19T09:27:11.334132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:35.801594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991224+0000","last_change":"2024-09-19T09:25:27.324839+0000","last_active":"2024-09-19T09:25:42.991224+0000","last_peered":"2024-09-19T09:25:42.991224+0000","last_clean":"2024-09-19T09:25:42.991224+0000","last_became_active":"2024-09-19T09:25:27.324475+0000","last_became_peered":"2024-09-19T09:25:27.324475+0000","last_unstale":"2024-09-19T09:25:42.991224+0000","last_undegraded":"2024-09-19T09:25:42.991224+0000","last_fullsized":"2024-09-19T09:25:42.991224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:32.251753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338007+0000","last_change":"2024-09-19T09:25:29.293025+0000","last_active":"2024-09-19T09:27:11.338007+0000","last_peered":"2024-09-19T09:27:11.338007+0000","last_clean":"2024-09-19T09:27:11.338007+0000","last_became_active":"2024-09-19T09:25:29.292570+0000","last_became_peered":"2024-09-19T09:25:29.292570+0000","last_unstale":"2024-09-19T09:27:11.338007+0000","last_undegraded":"2024-09-19T09:27:11.338007+0000","last_fullsized":"2024-09-19T09:27:11.338007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:16.394524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'17","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.329192+0000","last_change":"2024-09-19T09:25:27.278100+0000","last_active":"2024-09-19T09:27:11.329192+0000","last_peered":"2024-09-19T09:27:11.329192+0000","last_clean":"2024-09-19T09:27:11.329192+0000","last_became_active":"2024-09-19T09:25:27.277769+0000","last_became_peered":"2024-09-19T09:25:27.277769+0000","last_unstale":"2024-09-19T09:27:11.329192+0000","last_undegraded":"2024-09-19T09:27:11.329192+0000","last_fullsized":"2024-09-19T09:27:11.329192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:31.885142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.338230+0000","last_change":"2024-09-19T09:25:29.293651+0000","last_active":"2024-09-19T09:27:11.338230+0000","last_peered":"2024-09-19T09:27:11.338230+0000","last_clean":"2024-09-19T09:27:11.338230+0000","last_became_active":"2024-09-19T09:25:29.293317+0000","last_became_peered":"2024-09-19T09:25:29.293317+0000","last_unstale":"2024-09-19T09:27:11.338230+0000","last_undegraded":"2024-09-19T09:27:11.338230+0000","last_fullsized":"2024-09-19T09:27:11.338230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:40.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.990030+0000","last_change":"2024-09-19T09:25:27.269367+0000","last_active":"2024-09-19T09:25:42.990030+0000","last_peered":"2024-09-19T09:25:42.990030+0000","last_clean":"2024-09-19T09:25:42.990030+0000","last_became_active":"2024-09-19T09:25:27.269011+0000","last_became_peered":"2024-09-19T09:25:27.269011+0000","last_unstale":"2024-09-19T09:25:42.990030+0000","last_undegraded":"2024-09-19T09:25:42.990030+0000","last_fullsized":"2024-09-19T09:25:42.990030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:00.930752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.334968+0000","last_change":"2024-09-19T09:25:29.299134+0000","last_active":"2024-09-19T09:27:11.334968+0000","last_peered":"2024-09-19T09:27:11.334968+0000","last_clean":"2024-09-19T09:27:11.334968+0000","last_became_active":"2024-09-19T09:25:29.298648+0000","last_became_peered":"2024-09-19T09:25:29.298648+0000","last_unstale":"2024-09-19T09:27:11.334968+0000","last_undegraded":"2024-09-19T09:27:11.334968+0000","last_fullsized":"2024-09-19T09:27:11.334968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:39.627852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.003686+0000","last_change":"2024-09-19T09:25:27.322861+0000","last_active":"2024-09-19T09:25:43.003686+0000","last_peered":"2024-09-19T09:25:43.003686+0000","last_clean":"2024-09-19T09:25:43.003686+0000","last_became_active":"2024-09-19T09:25:27.322560+0000","last_became_peered":"2024-09-19T09:25:27.322560+0000","last_unstale":"2024-09-19T09:25:43.003686+0000","last_undegraded":"2024-09-19T09:25:43.003686+0000","last_fullsized":"2024-09-19T09:25:43.003686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:35.993093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.301522+0000","last_change":"2024-09-19T09:25:29.301680+0000","last_active":"2024-09-19T09:27:11.301522+0000","last_peered":"2024-09-19T09:27:11.301522+0000","last_clean":"2024-09-19T09:27:11.301522+0000","last_became_active":"2024-09-19T09:25:29.301331+0000","last_became_peered":"2024-09-19T09:25:29.301331+0000","last_unstale":"2024-09-19T09:27:11.301522+0000","last_undegraded":"2024-09-19T09:27:11.301522+0000","last_fullsized":"2024-09-19T09:27:11.301522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:57.364245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991324+0000","last_change":"2024-09-19T09:25:27.272609+0000","last_active":"2024-09-19T09:25:42.991324+0000","last_peered":"2024-09-19T09:25:42.991324+0000","last_clean":"2024-09-19T09:25:42.991324+0000","last_became_active":"2024-09-19T09:25:27.272169+0000","last_became_peered":"2024-09-19T09:25:27.272169+0000","last_unstale":"2024-09-19T09:25:42.991324+0000","last_undegraded":"2024-09-19T09:25:42.991324+0000","last_fullsized":"2024-09-19T09:25:42.991324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:18.879474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.341200+0000","last_change":"2024-09-19T09:25:29.296847+0000","last_active":"2024-09-19T09:27:11.341200+0000","last_peered":"2024-09-19T09:27:11.341200+0000","last_clean":"2024-09-19T09:27:11.341200+0000","last_became_active":"2024-09-19T09:25:29.296465+0000","last_became_peered":"2024-09-19T09:25:29.296465+0000","last_unstale":"2024-09-19T09:27:11.341200+0000","last_undegraded":"2024-09-19T09:27:11.341200+0000","last_fullsized":"2024-09-19T09:27:11.341200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:05:48.553252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:43.012954+0000","last_change":"2024-09-19T09:25:43.012532+0000","last_active":"2024-09-19T09:25:43.012954+0000","last_peered":"2024-09-19T09:25:43.012954+0000","last_clean":"2024-09-19T09:25:43.012954+0000","last_became_active":"2024-09-19T09:25:42.994996+0000","last_became_peered":"2024-09-19T09:25:42.994996+0000","last_unstale":"2024-09-19T09:25:43.012954+0000","last_undegraded":"2024-09-19T09:25:43.012954+0000","last_fullsized":"2024-09-19T09:25:43.012954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:25.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.337187+0000","last_change":"2024-09-19T09:25:29.297633+0000","last_active":"2024-09-19T09:27:11.337187+0000","last_peered":"2024-09-19T09:27:11.337187+0000","last_clean":"2024-09-19T09:27:11.337187+0000","last_became_active":"2024-09-19T09:25:29.297284+0000","last_became_peered":"2024-09-19T09:25:29.297284+0000","last_unstale":"2024-09-19T09:27:11.337187+0000","last_undegraded":"2024-09-19T09:27:11.337187+0000","last_fullsized":"2024-09-19T09:27:11.337187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:50.582546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991172+0000","last_change":"2024-09-19T09:25:27.281778+0000","last_active":"2024-09-19T09:25:42.991172+0000","last_peered":"2024-09-19T09:25:42.991172+0000","last_clean":"2024-09-19T09:25:42.991172+0000","last_became_active":"2024-09-19T09:25:27.281280+0000","last_became_peered":"2024-09-19T09:25:27.281280+0000","last_unstale":"2024-09-19T09:25:42.991172+0000","last_undegraded":"2024-09-19T09:25:42.991172+0000","last_fullsized":"2024-09-19T09:25:42.991172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:46.811968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.343342+0000","last_change":"2024-09-19T09:25:29.295663+0000","last_active":"2024-09-19T09:27:11.343342+0000","last_peered":"2024-09-19T09:27:11.343342+0000","last_clean":"2024-09-19T09:27:11.343342+0000","last_became_active":"2024-09-19T09:25:29.295146+0000","last_became_peered":"2024-09-19T09:25:29.295146+0000","last_unstale":"2024-09-19T09:27:11.343342+0000","last_undegraded":"2024-09-19T09:27:11.343342+0000","last_fullsized":"2024-09-19T09:27:11.343342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:36.749339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.345435+0000","last_change":"2024-09-19T09:25:29.298004+0000","last_active":"2024-09-19T09:27:11.345435+0000","last_peered":"2024-09-19T09:27:11.345435+0000","last_clean":"2024-09-19T09:27:11.345435+0000","last_became_active":"2024-09-19T09:25:29.297600+0000","last_became_peered":"2024-09-19T09:25:29.297600+0000","last_unstale":"2024-09-19T09:27:11.345435+0000","last_undegraded":"2024-09-19T09:27:11.345435+0000","last_fullsized":"2024-09-19T09:27:11.345435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:09.738615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.992323+0000","last_change":"2024-09-19T09:25:27.282876+0000","last_active":"2024-09-19T09:25:42.992323+0000","last_peered":"2024-09-19T09:25:42.992323+0000","last_clean":"2024-09-19T09:25:42.992323+0000","last_became_active":"2024-09-19T09:25:27.282201+0000","last_became_peered":"2024-09-19T09:25:27.282201+0000","last_unstale":"2024-09-19T09:25:42.992323+0000","last_undegraded":"2024-09-19T09:25:42.992323+0000","last_fullsized":"2024-09-19T09:25:42.992323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:08.508662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347692+0000","last_change":"2024-09-19T09:25:29.288379+0000","last_active":"2024-09-19T09:27:11.347692+0000","last_peered":"2024-09-19T09:27:11.347692+0000","last_clean":"2024-09-19T09:27:11.347692+0000","last_became_active":"2024-09-19T09:25:29.287917+0000","last_became_peered":"2024-09-19T09:25:29.287917+0000","last_unstale":"2024-09-19T09:27:11.347692+0000","last_undegraded":"2024-09-19T09:27:11.347692+0000","last_fullsized":"2024-09-19T09:27:11.347692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:03.628233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991076+0000","last_change":"2024-09-19T09:25:27.279551+0000","last_active":"2024-09-19T09:25:42.991076+0000","last_peered":"2024-09-19T09:25:42.991076+0000","last_clean":"2024-09-19T09:25:42.991076+0000","last_became_active":"2024-09-19T09:25:27.278956+0000","last_became_peered":"2024-09-19T09:25:27.278956+0000","last_unstale":"2024-09-19T09:25:42.991076+0000","last_undegraded":"2024-09-19T09:25:42.991076+0000","last_fullsized":"2024-09-19T09:25:42.991076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:17.186453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.347269+0000","last_change":"2024-09-19T09:25:29.288657+0000","last_active":"2024-09-19T09:27:11.347269+0000","last_peered":"2024-09-19T09:27:11.347269+0000","last_clean":"2024-09-19T09:27:11.347269+0000","last_became_active":"2024-09-19T09:25:29.288355+0000","last_became_peered":"2024-09-19T09:25:29.288355+0000","last_unstale":"2024-09-19T09:27:11.347269+0000","last_undegraded":"2024-09-19T09:27:11.347269+0000","last_fullsized":"2024-09-19T09:27:11.347269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:46.237558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:25:42.991373+0000","last_change":"2024-09-19T09:25:27.323312+0000","last_active":"2024-09-19T09:25:42.991373+0000","last_peered":"2024-09-19T09:25:42.991373+0000","last_clean":"2024-09-19T09:25:42.991373+0000","last_became_active":"2024-09-19T09:25:27.322984+0000","last_became_peered":"2024-09-19T09:25:27.322984+0000","last_unstale":"2024-09-19T09:25:42.991373+0000","last_undegraded":"2024-09-19T09:25:42.991373+0000","last_fullsized":"2024-09-19T09:25:42.991373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:48.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:11.327987+0000","last_change":"2024-09-19T09:25:29.301832+0000","last_active":"2024-09-19T09:27:11.327987+0000","last_peered":"2024-09-19T09:27:11.327987+0000","last_clean":"2024-09-19T09:27:11.327987+0000","last_became_active":"2024-09-19T09:25:29.301539+0000","last_became_peered":"2024-09-19T09:25:29.301539+0000","last_unstale":"2024-09-19T09:27:11.327987+0000","last_undegraded":"2024-09-19T09:27:11.327987+0000","last_fullsized":"2024-09-19T09:27:11.327987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:28.270247+0000","last_clean_scrub_stamp":"2024-09-19T09:25:28.270247+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:00.244948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'14","reported_seq":165,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T09:27:26.982182+0000","last_change":"2024-09-19T09:25:43.009668+0000","last_active":"2024-09-19T09:27:26.982182+0000","last_peered":"2024-09-19T09:27:26.982182+0000","last_clean":"2024-09-19T09:27:26.982182+0000","last_became_active":"2024-09-19T09:25:42.994966+0000","last_became_peered":"2024-09-19T09:25:42.994966+0000","last_unstale":"2024-09-19T09:27:26.982182+0000","last_undegraded":"2024-09-19T09:27:26.982182+0000","last_fullsized":"2024-09-19T09:27:26.982182+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T09:25:26.256190+0000","last_clean_scrub_stamp":"2024-09-19T09:25:26.256190+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:04:05.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74560,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1124,"num_read_kb":646,"num_write":263,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1842,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":223680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612646,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":431506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841578,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1597440,"data_stored":1087249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005102,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":917504,"data_stored":429922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234035,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":430300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462969,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1023357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":901120,"data_stored":430173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822723,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1523712,"data_stored":1022639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":432046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":429464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":495830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":496780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":431820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T09:27:33.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:27:34.114 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:34.238 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:33 smithi016 ceph-mon[30979]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:34.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:33 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:34.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:33 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:34.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:33 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:34.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:33 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:34.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:33 smithi142 ceph-mon[35032]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:34.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:33 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:34.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:33 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:34.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:33 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:34.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:33 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:34.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:33 smithi097 ceph-mon[34864]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T09:27:34.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:33 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:34.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:33 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:34.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:33 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:34.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:33 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.617+0000 7f108e658640 1 -- 172.21.15.16:0/1662321386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1088102ac0 msgr2=0x7f10881000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.617+0000 7f108e658640 1 --2- 172.21.15.16:0/1662321386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1088102ac0 0x7f10881000b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1078009900 tx=0x7f107802a480 comp rx=0 tx=0).stop 2024-09-19T09:27:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.617+0000 7f108e658640 1 -- 172.21.15.16:0/1662321386 shutdown_connections 2024-09-19T09:27:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.617+0000 7f108e658640 1 --2- 172.21.15.16:0/1662321386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1088102ea0 0x7f10881005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.617+0000 7f108e658640 1 --2- 172.21.15.16:0/1662321386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088100c10 0x7f1088110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.617+0000 7f108e658640 1 --2- 172.21.15.16:0/1662321386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1088102ac0 0x7f10881000b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.617+0000 7f108e658640 1 -- 172.21.15.16:0/1662321386 >> 172.21.15.16:0/1662321386 conn(0x7f1088077210 msgr2=0x7f10880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.617+0000 7f108e658640 1 -- 172.21.15.16:0/1662321386 shutdown_connections 2024-09-19T09:27:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.617+0000 7f108e658640 1 -- 172.21.15.16:0/1662321386 wait complete. 2024-09-19T09:27:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f108e658640 1 Processor -- start 2024-09-19T09:27:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f108e658640 1 -- start start 2024-09-19T09:27:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f108e658640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1088100c10 0x7f108819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:34.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f108e658640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1088102ac0 0x7f108819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f108e658640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088102ea0 0x7f108819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f108e658640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1088114150 con 0x7f1088100c10 2024-09-19T09:27:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f108e658640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1088113fd0 con 0x7f1088102ea0 2024-09-19T09:27:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f108e658640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f10881142d0 con 0x7f1088102ac0 2024-09-19T09:27:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f1087fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1088100c10 0x7f108819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f1087fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1088100c10 0x7f108819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42752/0 (socket says 172.21.15.16:42752) 2024-09-19T09:27:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f1087fff640 1 -- 172.21.15.16:0/4041026682 learned_addr learned my addr 172.21.15.16:0/4041026682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f108cbf5640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088102ea0 0x7f108819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:34.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f10877fe640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1088102ac0 0x7f108819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f1087fff640 1 -- 172.21.15.16:0/4041026682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1088102ac0 msgr2=0x7f108819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f1087fff640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1088102ac0 0x7f108819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f1087fff640 1 -- 172.21.15.16:0/4041026682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088102ea0 msgr2=0x7f108819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.618+0000 7f1087fff640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088102ea0 0x7f108819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.619+0000 7f1087fff640 1 -- 172.21.15.16:0/4041026682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108819f620 con 0x7f1088100c10 2024-09-19T09:27:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.619+0000 7f108cbf5640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088102ea0 0x7f108819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.619+0000 7f1087fff640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1088100c10 0x7f108819a670 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1078006010 tx=0x7f1078004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.619+0000 7f10857fa640 1 -- 172.21.15.16:0/4041026682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107803f030 con 0x7f1088100c10 2024-09-19T09:27:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.619+0000 7f108e658640 1 -- 172.21.15.16:0/4041026682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108819f8b0 con 0x7f1088100c10 2024-09-19T09:27:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.619+0000 7f10857fa640 1 -- 172.21.15.16:0/4041026682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1078004440 con 0x7f1088100c10 2024-09-19T09:27:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.619+0000 7f10857fa640 1 -- 172.21.15.16:0/4041026682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078035070 con 0x7f1088100c10 2024-09-19T09:27:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.619+0000 7f108e658640 1 -- 172.21.15.16:0/4041026682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10881a71f0 con 0x7f1088100c10 2024-09-19T09:27:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.620+0000 7f108e658640 1 -- 172.21.15.16:0/4041026682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10881000b0 con 0x7f1088100c10 2024-09-19T09:27:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.621+0000 7f10857fa640 1 -- 172.21.15.16:0/4041026682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f1078042030 con 0x7f1088100c10 2024-09-19T09:27:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.622+0000 7f10857fa640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f10700a3240 0x7f10700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.622+0000 7f10857fa640 1 -- 172.21.15.16:0/4041026682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f10780efff0 con 0x7f1088100c10 2024-09-19T09:27:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.622+0000 7f10877fe640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f10700a3240 0x7f10700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.623+0000 7f10877fe640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f10700a3240 0x7f10700a5700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f10880ff750 tx=0x7f107c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.625+0000 7f10857fa640 1 -- 172.21.15.16:0/4041026682 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f10780bcd10 con 0x7f1088100c10 2024-09-19T09:27:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.853+0000 7f108e658640 1 -- 172.21.15.16:0/4041026682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f108810bf10 con 0x7f1088100c10 2024-09-19T09:27:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.854+0000 7f10857fa640 1 -- 172.21.15.16:0/4041026682 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f10780bc610 con 0x7f1088100c10 2024-09-19T09:27:34.855 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:34.855 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:27:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:27:34.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.857+0000 7f108e658640 1 -- 172.21.15.16:0/4041026682 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f10700a3240 msgr2=0x7f10700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:34.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.857+0000 7f108e658640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f10700a3240 0x7f10700a5700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f10880ff750 tx=0x7f107c005f90 comp rx=0 tx=0).stop 2024-09-19T09:27:34.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.857+0000 7f108e658640 1 -- 172.21.15.16:0/4041026682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1088100c10 msgr2=0x7f108819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:34.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.857+0000 7f108e658640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1088100c10 0x7f108819a670 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1078006010 tx=0x7f1078004040 comp rx=0 tx=0).stop 2024-09-19T09:27:34.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.857+0000 7f108e658640 1 -- 172.21.15.16:0/4041026682 shutdown_connections 2024-09-19T09:27:34.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.857+0000 7f108e658640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f10700a3240 0x7f10700a5700 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.857+0000 7f108e658640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1088102ac0 0x7f108819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.857+0000 7f108e658640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088102ea0 0x7f108819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.857+0000 7f108e658640 1 --2- 172.21.15.16:0/4041026682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1088100c10 0x7f108819a670 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:34.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.857+0000 7f108e658640 1 -- 172.21.15.16:0/4041026682 >> 172.21.15.16:0/4041026682 conn(0x7f1088077210 msgr2=0x7f10880fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:34.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.857+0000 7f108e658640 1 -- 172.21.15.16:0/4041026682 shutdown_connections 2024-09-19T09:27:34.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:34.857+0000 7f108e658640 1 -- 172.21.15.16:0/4041026682 wait complete. 2024-09-19T09:27:35.244 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:34 smithi016 ceph-mon[30979]: pgmap v337: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-19T09:27:35.244 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:34 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/4041026682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:35.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:34 smithi142 ceph-mon[35032]: pgmap v337: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-19T09:27:35.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:34 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/4041026682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:35.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:34 smithi097 ceph-mon[34864]: pgmap v337: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-19T09:27:35.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:34 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/4041026682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:35.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:27:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.332+0000 7f186fb6a640 1 -- 172.21.15.16:0/871300994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1868106f20 msgr2=0x7f186810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.332+0000 7f186fb6a640 1 --2- 172.21.15.16:0/871300994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1868106f20 0x7f186810eaa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1860009ca0 tx=0x7f186002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:27:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.332+0000 7f186fb6a640 1 -- 172.21.15.16:0/871300994 shutdown_connections 2024-09-19T09:27:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.332+0000 7f186fb6a640 1 --2- 172.21.15.16:0/871300994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1868106f20 0x7f186810eaa0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.332+0000 7f186fb6a640 1 --2- 172.21.15.16:0/871300994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18681027a0 0x7f18681067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.332+0000 7f186fb6a640 1 --2- 172.21.15.16:0/871300994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1868101df0 0x7f18681021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.332+0000 7f186fb6a640 1 -- 172.21.15.16:0/871300994 >> 172.21.15.16:0/871300994 conn(0x7f18680fd8f0 msgr2=0x7f18680ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.332+0000 7f186fb6a640 1 -- 172.21.15.16:0/871300994 shutdown_connections 2024-09-19T09:27:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.332+0000 7f186fb6a640 1 -- 172.21.15.16:0/871300994 wait complete. 2024-09-19T09:27:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.333+0000 7f186fb6a640 1 Processor -- start 2024-09-19T09:27:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.333+0000 7f186fb6a640 1 -- start start 2024-09-19T09:27:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.333+0000 7f186fb6a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1868101df0 0x7f18681982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.333+0000 7f186fb6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18681027a0 0x7f1868198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.333+0000 7f186fb6a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1868106f20 0x7f186819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.333+0000 7f186fb6a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1868111f80 con 0x7f18681027a0 2024-09-19T09:27:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.333+0000 7f186fb6a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1868111e00 con 0x7f1868101df0 2024-09-19T09:27:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.333+0000 7f186fb6a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1868112100 con 0x7f1868106f20 2024-09-19T09:27:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.333+0000 7f186d105640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18681027a0 0x7f1868198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.333+0000 7f186d105640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18681027a0 0x7f1868198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55100/0 (socket says 172.21.15.16:55100) 2024-09-19T09:27:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.333+0000 7f186d105640 1 -- 172.21.15.16:0/1694390524 learned_addr learned my addr 172.21.15.16:0/1694390524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:36.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.334+0000 7f186d906640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1868101df0 0x7f18681982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.334+0000 7f186e107640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1868106f20 0x7f186819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.334+0000 7f186d105640 1 -- 172.21.15.16:0/1694390524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1868106f20 msgr2=0x7f186819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.334+0000 7f186d105640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1868106f20 0x7f186819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.334+0000 7f186d105640 1 -- 172.21.15.16:0/1694390524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1868101df0 msgr2=0x7f18681982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.334+0000 7f186d105640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1868101df0 0x7f18681982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.334+0000 7f186d105640 1 -- 172.21.15.16:0/1694390524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186819d280 con 0x7f18681027a0 2024-09-19T09:27:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.334+0000 7f186d906640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1868101df0 0x7f18681982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.334+0000 7f186d105640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18681027a0 0x7f1868198810 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f185800c920 tx=0x7f185800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.334+0000 7f185effd640 1 -- 172.21.15.16:0/1694390524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858007c70 con 0x7f18681027a0 2024-09-19T09:27:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.334+0000 7f185effd640 1 -- 172.21.15.16:0/1694390524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1858007e10 con 0x7f18681027a0 2024-09-19T09:27:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.334+0000 7f185effd640 1 -- 172.21.15.16:0/1694390524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858018890 con 0x7f18681027a0 2024-09-19T09:27:36.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.334+0000 7f186fb6a640 1 -- 172.21.15.16:0/1694390524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186819d570 con 0x7f18681027a0 2024-09-19T09:27:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.335+0000 7f186fb6a640 1 -- 172.21.15.16:0/1694390524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18681a4e50 con 0x7f18681027a0 2024-09-19T09:27:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.336+0000 7f186fb6a640 1 -- 172.21.15.16:0/1694390524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18681021d0 con 0x7f18681027a0 2024-09-19T09:27:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.337+0000 7f185effd640 1 -- 172.21.15.16:0/1694390524 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f1858018a70 con 0x7f18681027a0 2024-09-19T09:27:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.338+0000 7f185effd640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f18540a3390 0x7f18540a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.338+0000 7f186d906640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f18540a3390 0x7f18540a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.338+0000 7f185effd640 1 -- 172.21.15.16:0/1694390524 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f18580cf990 con 0x7f18681027a0 2024-09-19T09:27:36.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.338+0000 7f186d906640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f18540a3390 0x7f18540a5850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f186400abe0 tx=0x7f186400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:36.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.342+0000 7f185effd640 1 -- 172.21.15.16:0/1694390524 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1858014030 con 0x7f18681027a0 2024-09-19T09:27:36.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.556+0000 7f186fb6a640 1 -- 172.21.15.16:0/1694390524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18681a5080 con 0x7f18681027a0 2024-09-19T09:27:36.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.557+0000 7f185effd640 1 -- 172.21.15.16:0/1694390524 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f185801f300 con 0x7f18681027a0 2024-09-19T09:27:36.558 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:36.558 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:27:36.558 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:27:36.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.559+0000 7f186fb6a640 1 -- 172.21.15.16:0/1694390524 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f18540a3390 msgr2=0x7f18540a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:36.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.559+0000 7f186fb6a640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f18540a3390 0x7f18540a5850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f186400abe0 tx=0x7f186400a5c0 comp rx=0 tx=0).stop 2024-09-19T09:27:36.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.559+0000 7f186fb6a640 1 -- 172.21.15.16:0/1694390524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18681027a0 msgr2=0x7f1868198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:36.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.559+0000 7f186fb6a640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18681027a0 0x7f1868198810 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f185800c920 tx=0x7f185800cde0 comp rx=0 tx=0).stop 2024-09-19T09:27:36.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.559+0000 7f186fb6a640 1 -- 172.21.15.16:0/1694390524 shutdown_connections 2024-09-19T09:27:36.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.559+0000 7f186fb6a640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f18540a3390 0x7f18540a5850 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:36.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.559+0000 7f186fb6a640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1868106f20 0x7f186819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:36.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.559+0000 7f186fb6a640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18681027a0 0x7f1868198810 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:36.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.559+0000 7f186fb6a640 1 --2- 172.21.15.16:0/1694390524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1868101df0 0x7f18681982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:36.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.559+0000 7f186fb6a640 1 -- 172.21.15.16:0/1694390524 >> 172.21.15.16:0/1694390524 conn(0x7f18680fd8f0 msgr2=0x7f18680fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:36.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.559+0000 7f186fb6a640 1 -- 172.21.15.16:0/1694390524 shutdown_connections 2024-09-19T09:27:36.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:36.559+0000 7f186fb6a640 1 -- 172.21.15.16:0/1694390524 wait complete. 2024-09-19T09:27:36.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:36 smithi142 ceph-mon[35032]: pgmap v338: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T09:27:36.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:36.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:36.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:36.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:36.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:36 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:36.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:36 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1694390524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:36.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:36 smithi097 ceph-mon[34864]: pgmap v338: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T09:27:36.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:36.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:36.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:36.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:36.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:36 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:36.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:36 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1694390524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:36 smithi016 ceph-mon[30979]: pgmap v338: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-19T09:27:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:27:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:27:36.940 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:36 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:27:36.941 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:36 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1694390524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:37.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:27:37.525 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.942+0000 7fa0b896f640 1 -- 172.21.15.16:0/245863557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b4102570 msgr2=0x7fa0b41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.942+0000 7fa0b896f640 1 --2- 172.21.15.16:0/245863557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b4102570 0x7fa0b41029b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009900 tx=0x7fa0a802a480 comp rx=0 tx=0).stop 2024-09-19T09:27:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.942+0000 7fa0b896f640 1 -- 172.21.15.16:0/245863557 shutdown_connections 2024-09-19T09:27:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.942+0000 7fa0b896f640 1 --2- 172.21.15.16:0/245863557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102ef0 0x7fa0b410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.942+0000 7fa0b896f640 1 --2- 172.21.15.16:0/245863557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b4102570 0x7fa0b41029b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.942+0000 7fa0b896f640 1 --2- 172.21.15.16:0/245863557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0b4100e20 0x7fa0b4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.942+0000 7fa0b896f640 1 -- 172.21.15.16:0/245863557 >> 172.21.15.16:0/245863557 conn(0x7fa0b40fc7d0 msgr2=0x7fa0b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.942+0000 7fa0b896f640 1 -- 172.21.15.16:0/245863557 shutdown_connections 2024-09-19T09:27:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.942+0000 7fa0b896f640 1 -- 172.21.15.16:0/245863557 wait complete. 2024-09-19T09:27:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.943+0000 7fa0b896f640 1 Processor -- start 2024-09-19T09:27:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.943+0000 7fa0b896f640 1 -- start start 2024-09-19T09:27:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.943+0000 7fa0b896f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b4100e20 0x7fa0b4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.943+0000 7fa0b896f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0b4102570 0x7fa0b4199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.943+0000 7fa0b896f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102ef0 0x7fa0b41a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.943+0000 7fa0b896f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4110fd0 con 0x7fa0b4100e20 2024-09-19T09:27:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.943+0000 7fa0b896f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4110e50 con 0x7fa0b4102570 2024-09-19T09:27:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.943+0000 7fa0b896f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4111150 con 0x7fa0b4102ef0 2024-09-19T09:27:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.943+0000 7fa0b259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b4100e20 0x7fa0b4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.943+0000 7fa0b259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b4100e20 0x7fa0b4199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55116/0 (socket says 172.21.15.16:55116) 2024-09-19T09:27:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.943+0000 7fa0b259c640 1 -- 172.21.15.16:0/912627533 learned_addr learned my addr 172.21.15.16:0/912627533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.943+0000 7fa0b1d9b640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0b4102570 0x7fa0b4199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.944+0000 7fa0b259c640 1 -- 172.21.15.16:0/912627533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102ef0 msgr2=0x7fa0b41a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.944+0000 7fa0b2d9d640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102ef0 0x7fa0b41a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.944+0000 7fa0b259c640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102ef0 0x7fa0b41a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.944+0000 7fa0b259c640 1 -- 172.21.15.16:0/912627533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0b4102570 msgr2=0x7fa0b4199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.944+0000 7fa0b259c640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0b4102570 0x7fa0b4199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.944+0000 7fa0b259c640 1 -- 172.21.15.16:0/912627533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b41a4900 con 0x7fa0b4100e20 2024-09-19T09:27:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.944+0000 7fa0b2d9d640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102ef0 0x7fa0b41a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.944+0000 7fa0b259c640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b4100e20 0x7fa0b4199340 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400c890 tx=0x7fa0a400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.944+0000 7fa0a37fe640 1 -- 172.21.15.16:0/912627533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4004220 con 0x7fa0b4100e20 2024-09-19T09:27:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.944+0000 7fa0a37fe640 1 -- 172.21.15.16:0/912627533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a40043c0 con 0x7fa0b4100e20 2024-09-19T09:27:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.944+0000 7fa0b896f640 1 -- 172.21.15.16:0/912627533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b41a4ad0 con 0x7fa0b4100e20 2024-09-19T09:27:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.944+0000 7fa0a37fe640 1 -- 172.21.15.16:0/912627533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a40049e0 con 0x7fa0b4100e20 2024-09-19T09:27:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.944+0000 7fa0b896f640 1 -- 172.21.15.16:0/912627533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b41a4f90 con 0x7fa0b4100e20 2024-09-19T09:27:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.945+0000 7fa0b896f640 1 -- 172.21.15.16:0/912627533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b4101200 con 0x7fa0b4100e20 2024-09-19T09:27:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.946+0000 7fa0a37fe640 1 -- 172.21.15.16:0/912627533 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa0a40079b0 con 0x7fa0b4100e20 2024-09-19T09:27:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.947+0000 7fa0a37fe640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa09c0a3240 0x7fa09c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.947+0000 7fa0b1d9b640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa09c0a3240 0x7fa09c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.947+0000 7fa0a37fe640 1 -- 172.21.15.16:0/912627533 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa0a40cfa60 con 0x7fa0b4100e20 2024-09-19T09:27:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.947+0000 7fa0b1d9b640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa09c0a3240 0x7fa09c0a5700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8000c00 tx=0x7fa0a8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:37.951+0000 7fa0a37fe640 1 -- 172.21.15.16:0/912627533 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa0a4014030 con 0x7fa0b4100e20 2024-09-19T09:27:38.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.131+0000 7fa0b896f640 1 -- 172.21.15.16:0/912627533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0b41a5220 con 0x7fa0b4100e20 2024-09-19T09:27:38.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.132+0000 7fa0a37fe640 1 -- 172.21.15.16:0/912627533 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22665 (secure 0 0 0) 0x7fa0a409c740 con 0x7fa0b4100e20 2024-09-19T09:27:38.133 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:38.134 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","created":"2024-09-19T09:17:48.242238+0000","modified":"2024-09-19T09:25:42.982671+0000","last_up_change":"2024-09-19T09:24:40.190289+0000","last_in_change":"2024-09-19T09:24:17.190380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:20:57.748686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:25:25.694451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:25:28.262997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbf3b28b-2282-4057-9e65-19ceb78ff19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6803","nonce":1578957623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6805","nonce":1578957623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6809","nonce":1578957623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6807","nonce":1578957623}]},"public_addr":"172.21.15.16:6803/1578957623","cluster_addr":"172.21.15.16:6805/1578957623","heartbeat_back_addr":"172.21.15.16:6809/1578957623","heartbeat_front_addr":"172.21.15.16:6807/1578957623","state":["exists","up"]},{"osd":1,"uuid":"c2517aeb-1d79-41ad-a180-369eecb9732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6801","nonce":3804152349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6803","nonce":3804152349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6807","nonce":3804152349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6805","nonce":3804152349}]},"public_addr":"172.21.15.97:6801/3804152349","cluster_addr":"172.21.15.97:6803/3804152349","heartbeat_back_addr":"172.21.15.97:6807/3804152349","heartbeat_front_addr":"172.21.15.97:6805/3804152349","state":["exists","up"]},{"osd":2,"uuid":"3c367518-5085-471a-98cb-ec5bf5facdf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6801","nonce":2692613951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6803","nonce":2692613951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6807","nonce":2692613951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6805","nonce":2692613951}]},"public_addr":"172.21.15.142:6801/2692613951","cluster_addr":"172.21.15.142:6803/2692613951","heartbeat_back_addr":"172.21.15.142:6807/2692613951","heartbeat_front_addr":"172.21.15.142:6805/2692613951","state":["exists","up"]},{"osd":3,"uuid":"e7118925-f0fa-46b3-8552-3e3bb76a6489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6811","nonce":3169290004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6813","nonce":3169290004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6817","nonce":3169290004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6815","nonce":3169290004}]},"public_addr":"172.21.15.16:6811/3169290004","cluster_addr":"172.21.15.16:6813/3169290004","heartbeat_back_addr":"172.21.15.16:6817/3169290004","heartbeat_front_addr":"172.21.15.16:6815/3169290004","state":["exists","up"]},{"osd":4,"uuid":"dc1964f7-af9c-4853-843b-9818f87c7768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6809","nonce":3491353825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6811","nonce":3491353825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6815","nonce":3491353825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6813","nonce":3491353825}]},"public_addr":"172.21.15.97:6809/3491353825","cluster_addr":"172.21.15.97:6811/3491353825","heartbeat_back_addr":"172.21.15.97:6815/3491353825","heartbeat_front_addr":"172.21.15.97:6813/3491353825","state":["exists","up"]},{"osd":5,"uuid":"4d7fbafc-d9ba-4036-a89d-aa295b8a23a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6809","nonce":577099986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6811","nonce":577099986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6815","nonce":577099986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6813","nonce":577099986}]},"public_addr":"172.21.15.142:6809/577099986","cluster_addr":"172.21.15.142:6811/577099986","heartbeat_back_addr":"172.21.15.142:6815/577099986","heartbeat_front_addr":"172.21.15.142:6813/577099986","state":["exists","up"]},{"osd":6,"uuid":"e3f346fc-c027-44f8-bd37-bccf6ec09e3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6819","nonce":4053539536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6821","nonce":4053539536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6825","nonce":4053539536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6823","nonce":4053539536}]},"public_addr":"172.21.15.16:6819/4053539536","cluster_addr":"172.21.15.16:6821/4053539536","heartbeat_back_addr":"172.21.15.16:6825/4053539536","heartbeat_front_addr":"172.21.15.16:6823/4053539536","state":["exists","up"]},{"osd":7,"uuid":"d390b547-8949-4d4b-8481-c49d6ae7e424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6817","nonce":1157122670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6819","nonce":1157122670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6823","nonce":1157122670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6821","nonce":1157122670}]},"public_addr":"172.21.15.97:6817/1157122670","cluster_addr":"172.21.15.97:6819/1157122670","heartbeat_back_addr":"172.21.15.97:6823/1157122670","heartbeat_front_addr":"172.21.15.97:6821/1157122670","state":["exists","up"]},{"osd":8,"uuid":"78a476c4-5deb-4f9e-a266-db4493850732","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6817","nonce":643795004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6819","nonce":643795004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6823","nonce":643795004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6821","nonce":643795004}]},"public_addr":"172.21.15.142:6817/643795004","cluster_addr":"172.21.15.142:6819/643795004","heartbeat_back_addr":"172.21.15.142:6823/643795004","heartbeat_front_addr":"172.21.15.142:6821/643795004","state":["exists","up"]},{"osd":9,"uuid":"0415d187-4a77-42f7-b8bb-2d0b52e9cb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6827","nonce":2651132360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6829","nonce":2651132360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2651132360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6831","nonce":2651132360}]},"public_addr":"172.21.15.16:6827/2651132360","cluster_addr":"172.21.15.16:6829/2651132360","heartbeat_back_addr":"172.21.15.16:6833/2651132360","heartbeat_front_addr":"172.21.15.16:6831/2651132360","state":["exists","up"]},{"osd":10,"uuid":"6fa78f96-cdfe-42a0-8d27-0705a0c7df8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6825","nonce":1848371486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6827","nonce":1848371486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6831","nonce":1848371486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6829","nonce":1848371486}]},"public_addr":"172.21.15.97:6825/1848371486","cluster_addr":"172.21.15.97:6827/1848371486","heartbeat_back_addr":"172.21.15.97:6831/1848371486","heartbeat_front_addr":"172.21.15.97:6829/1848371486","state":["exists","up"]},{"osd":11,"uuid":"d27f20e3-84ce-470c-9889-cd86906b158d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6825","nonce":155083878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6827","nonce":155083878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6831","nonce":155083878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6829","nonce":155083878}]},"public_addr":"172.21.15.142:6825/155083878","cluster_addr":"172.21.15.142:6827/155083878","heartbeat_back_addr":"172.21.15.142:6831/155083878","heartbeat_front_addr":"172.21.15.142:6829/155083878","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:05.088764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:29.928529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:55.236930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:20.379471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:44.984151+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:10.410074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:35.233553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:59.928700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:24.503069+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:49.853073+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:14.006841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:38.546467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3710267992":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3740275561":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/2755421444":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/357390846":"2024-09-20T09:18:17.070034+0000","172.21.15.16:6800/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3104087402":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3546947161":"2024-09-20T09:18:17.070034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:27:38.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.134+0000 7fa0b896f640 1 -- 172.21.15.16:0/912627533 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa09c0a3240 msgr2=0x7fa09c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.134+0000 7fa0b896f640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa09c0a3240 0x7fa09c0a5700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8000c00 tx=0x7fa0a8005a20 comp rx=0 tx=0).stop 2024-09-19T09:27:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.134+0000 7fa0b896f640 1 -- 172.21.15.16:0/912627533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b4100e20 msgr2=0x7fa0b4199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.134+0000 7fa0b896f640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b4100e20 0x7fa0b4199340 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400c890 tx=0x7fa0a400cd50 comp rx=0 tx=0).stop 2024-09-19T09:27:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.134+0000 7fa0b896f640 1 -- 172.21.15.16:0/912627533 shutdown_connections 2024-09-19T09:27:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.134+0000 7fa0b896f640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa09c0a3240 0x7fa09c0a5700 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.134+0000 7fa0b896f640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b4102ef0 0x7fa0b41a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.134+0000 7fa0b896f640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0b4102570 0x7fa0b4199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.134+0000 7fa0b896f640 1 --2- 172.21.15.16:0/912627533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0b4100e20 0x7fa0b4199340 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.134+0000 7fa0b896f640 1 -- 172.21.15.16:0/912627533 >> 172.21.15.16:0/912627533 conn(0x7fa0b40fc7d0 msgr2=0x7fa0b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.134+0000 7fa0b896f640 1 -- 172.21.15.16:0/912627533 shutdown_connections 2024-09-19T09:27:38.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:38.134+0000 7fa0b896f640 1 -- 172.21.15.16:0/912627533 wait complete. 2024-09-19T09:27:38.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-19T09:27:38.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:38 smithi142 ceph-mon[35032]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-19T09:27:38.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:38 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/912627533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:27:38.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:38 smithi097 ceph-mon[34864]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-19T09:27:38.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:38 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/912627533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:27:38.979 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:39.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:38 smithi016 ceph-mon[30979]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-19T09:27:39.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:38 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/912627533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:27:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.299+0000 7fd478366640 1 -- 172.21.15.16:0/2578733397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd470100a40 msgr2=0x7fd470100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.299+0000 7fd478366640 1 --2- 172.21.15.16:0/2578733397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd470100a40 0x7fd470100e40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd460009ca0 tx=0x7fd46002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.299+0000 7fd478366640 1 -- 172.21.15.16:0/2578733397 shutdown_connections 2024-09-19T09:27:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.299+0000 7fd478366640 1 --2- 172.21.15.16:0/2578733397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4701025f0 0x7fd470108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.299+0000 7fd478366640 1 --2- 172.21.15.16:0/2578733397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd470101c30 0x7fd4701020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.299+0000 7fd478366640 1 --2- 172.21.15.16:0/2578733397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd470100a40 0x7fd470100e40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.299+0000 7fd478366640 1 -- 172.21.15.16:0/2578733397 >> 172.21.15.16:0/2578733397 conn(0x7fd4700fc1e0 msgr2=0x7fd4700fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.299+0000 7fd478366640 1 -- 172.21.15.16:0/2578733397 shutdown_connections 2024-09-19T09:27:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.299+0000 7fd478366640 1 -- 172.21.15.16:0/2578733397 wait complete. 2024-09-19T09:27:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.300+0000 7fd478366640 1 Processor -- start 2024-09-19T09:27:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.300+0000 7fd478366640 1 -- start start 2024-09-19T09:27:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.300+0000 7fd478366640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd470100a40 0x7fd470197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd478366640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd470101c30 0x7fd470198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd478366640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4701025f0 0x7fd47019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd478366640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd47010c6c0 con 0x7fd470101c30 2024-09-19T09:27:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd478366640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd47010c540 con 0x7fd470100a40 2024-09-19T09:27:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd478366640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd47010c840 con 0x7fd4701025f0 2024-09-19T09:27:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd475901640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd470101c30 0x7fd470198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd475901640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd470101c30 0x7fd470198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55128/0 (socket says 172.21.15.16:55128) 2024-09-19T09:27:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd475901640 1 -- 172.21.15.16:0/2739488728 learned_addr learned my addr 172.21.15.16:0/2739488728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd476903640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4701025f0 0x7fd47019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd476102640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd470100a40 0x7fd470197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd475901640 1 -- 172.21.15.16:0/2739488728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4701025f0 msgr2=0x7fd47019f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd475901640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4701025f0 0x7fd47019f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd475901640 1 -- 172.21.15.16:0/2739488728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd470100a40 msgr2=0x7fd470197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd475901640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd470100a40 0x7fd470197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd475901640 1 -- 172.21.15.16:0/2739488728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47019fa10 con 0x7fd470101c30 2024-09-19T09:27:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd475901640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd470101c30 0x7fd470198330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd46400c890 tx=0x7fd46400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd46f7fe640 1 -- 172.21.15.16:0/2739488728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464004220 con 0x7fd470101c30 2024-09-19T09:27:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd478366640 1 -- 172.21.15.16:0/2739488728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47019fd00 con 0x7fd470101c30 2024-09-19T09:27:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd46f7fe640 1 -- 172.21.15.16:0/2739488728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4640043c0 con 0x7fd470101c30 2024-09-19T09:27:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd46f7fe640 1 -- 172.21.15.16:0/2739488728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464004a00 con 0x7fd470101c30 2024-09-19T09:27:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.301+0000 7fd478366640 1 -- 172.21.15.16:0/2739488728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4701a01f0 con 0x7fd470101c30 2024-09-19T09:27:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.304+0000 7fd46f7fe640 1 -- 172.21.15.16:0/2739488728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd4640079b0 con 0x7fd470101c30 2024-09-19T09:27:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.304+0000 7fd47b1e0740 1 Processor -- start 2024-09-19T09:27:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.304+0000 7fd47b1e0740 1 -- start start 2024-09-19T09:27:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd47b1e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 0x55f1f4f521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd47b1e0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd47b1e0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f57780 0x55f1f4f59c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd47b1e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f1f4eb12f0 con 0x55f1f4f51db0 2024-09-19T09:27:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd47b1e0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f1f4e59270 con 0x55f1f4f57780 2024-09-19T09:27:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd47b1e0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f1f4ead2b0 con 0x55f1f4f526f0 2024-09-19T09:27:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd46f7fe640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4580a32f0 0x7fd4580a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd476903640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 0x55f1f4f521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd476903640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 0x55f1f4f521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55134/0 (socket says 172.21.15.16:55134) 2024-09-19T09:27:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd476903640 1 -- 172.21.15.16:0/3216480571 learned_addr learned my addr 172.21.15.16:0/3216480571 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd475901640 1 --2- 172.21.15.16:0/3216480571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f57780 0x55f1f4f59c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd476102640 1 --2- 172.21.15.16:0/3216480571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd476903640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4580a32f0 0x7fd4580a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd46f7fe640 1 -- 172.21.15.16:0/2739488728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd4640cf650 con 0x7fd470101c30 2024-09-19T09:27:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd476903640 1 -- 172.21.15.16:0/3216480571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 msgr2=0x55f1f4f571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd476903640 1 --2- 172.21.15.16:0/3216480571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd476903640 1 -- 172.21.15.16:0/3216480571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f57780 msgr2=0x55f1f4f59c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd476903640 1 --2- 172.21.15.16:0/3216480571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f57780 0x55f1f4f59c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd476903640 1 -- 172.21.15.16:0/3216480571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f4e58340 con 0x55f1f4f51db0 2024-09-19T09:27:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.305+0000 7fd475901640 1 --2- 172.21.15.16:0/3216480571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f57780 0x55f1f4f59c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.306+0000 7fd476903640 1 --2- 172.21.15.16:0/3216480571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 0x55f1f4f521b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd468011ad0 tx=0x7fd468011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2b91c0862ef6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.306+0000 7fd476903640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4580a32f0 0x7fd4580a57b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd46800f850 tx=0x7fd46801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.306+0000 7fd46cbfe640 1 -- 172.21.15.16:0/3216480571 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468011050 con 0x55f1f4f51db0 2024-09-19T09:27:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.306+0000 7fd46cbfe640 1 -- 172.21.15.16:0/3216480571 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd46801ddc0 con 0x55f1f4f51db0 2024-09-19T09:27:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.306+0000 7fd46cbfe640 1 -- 172.21.15.16:0/3216480571 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468010b40 con 0x55f1f4f51db0 2024-09-19T09:27:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.306+0000 7fd47b1e0740 1 -- 172.21.15.16:0/3216480571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 msgr2=0x55f1f4f521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.306+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/3216480571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 0x55f1f4f521b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd468011ad0 tx=0x7fd468011f90 comp rx=0 tx=0).stop 2024-09-19T09:27:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.306+0000 7fd47b1e0740 1 -- 172.21.15.16:0/3216480571 shutdown_connections 2024-09-19T09:27:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.306+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/3216480571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.306+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/3216480571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f57780 0x55f1f4f59c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/3216480571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 0x55f1f4f521b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd47b1e0740 1 -- 172.21.15.16:0/3216480571 >> 172.21.15.16:0/3216480571 conn(0x55f1f4f49010 msgr2=0x55f1f4f49410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd47b1e0740 1 -- 172.21.15.16:0/3216480571 shutdown_connections 2024-09-19T09:27:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd47b1e0740 1 -- 172.21.15.16:0/3216480571 wait complete. 2024-09-19T09:27:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd47b1e0740 1 -- start start 2024-09-19T09:27:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd47b1e0740 1 Processor -- start 2024-09-19T09:27:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd47b1e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 0x55f1f4f5bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd47b1e0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f5c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd47b1e0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f57780 0x55f1f4f634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd47b1e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f1f4f5a6f0 con 0x55f1f4f51db0 2024-09-19T09:27:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd47b1e0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f1f4e3a290 con 0x55f1f4f57780 2024-09-19T09:27:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd47b1e0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f1f4f5a870 con 0x55f1f4f526f0 2024-09-19T09:27:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd47b1e0740 10 client.0 _ll_register_callbacks cb 0x55f1f4e6b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd476102640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 0x55f1f4f5bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd476102640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 0x55f1f4f5bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55140/0 (socket says 172.21.15.16:55140) 2024-09-19T09:27:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.307+0000 7fd476102640 1 -- 172.21.15.16:0/3593403306 learned_addr learned my addr 172.21.15.16:0/3593403306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd475901640 1 --2- 172.21.15.16:0/3593403306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f5c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd476903640 1 --2- 172.21.15.16:0/3593403306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f57780 0x55f1f4f634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd476102640 1 -- 172.21.15.16:0/3593403306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 msgr2=0x55f1f4f5c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd476102640 1 --2- 172.21.15.16:0/3593403306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f5c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd476102640 1 -- 172.21.15.16:0/3593403306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f57780 msgr2=0x55f1f4f634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd476102640 1 --2- 172.21.15.16:0/3593403306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f57780 0x55f1f4f634d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd476102640 1 -- 172.21.15.16:0/3593403306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f4bc4210 con 0x55f1f4f51db0 2024-09-19T09:27:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd476102640 1 --2- 172.21.15.16:0/3593403306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 0x55f1f4f5bf50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd460009ca0 tx=0x7fd460009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9fb062bb6dc2d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd455ffb640 1 -- 172.21.15.16:0/3593403306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460003970 con 0x55f1f4f51db0 2024-09-19T09:27:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd455ffb640 1 -- 172.21.15.16:0/3593403306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd460003b10 con 0x55f1f4f51db0 2024-09-19T09:27:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd47b1e0740 1 -- 172.21.15.16:0/3593403306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd46801ddc0 con 0x55f1f4f51db0 2024-09-19T09:27:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd455ffb640 1 -- 172.21.15.16:0/3593403306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd460003d10 con 0x55f1f4f51db0 2024-09-19T09:27:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd47b1e0740 10 client.15333 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd47b1e0740 1 -- 172.21.15.16:0/3593403306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1f4c90620 con 0x55f1f4f51db0 2024-09-19T09:27:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.308+0000 7fd455ffb640 1 -- 172.21.15.16:0/3593403306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd46801ddc0 con 0x55f1f4f51db0 2024-09-19T09:27:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.309+0000 7fd455ffb640 1 -- 172.21.15.16:0/3593403306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd46004d020 con 0x55f1f4f51db0 2024-09-19T09:27:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.309+0000 7fd47b1e0740 10 client.15333 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.309+0000 7fd47b1e0740 10 client.15333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.309+0000 7fd47b1e0740 20 client.15333 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.309+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/3593403306 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55f1f4f67bc0 0x55f1f4f6a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.309+0000 7fd47b1e0740 4 client.15333 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.309+0000 7fd47b1e0740 1 -- 172.21.15.16:0/3593403306 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1f4ebea20 con 0x55f1f4f67bc0 2024-09-19T09:27:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.309+0000 7fd475901640 1 --2- 172.21.15.16:0/3593403306 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55f1f4f67bc0 0x55f1f4f6a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.309+0000 7fd475901640 1 --2- 172.21.15.16:0/3593403306 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55f1f4f67bc0 0x55f1f4f6a010 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b48ba491a95d52 server_cookie=e70fb700d533e6ca in_seq=0 out_seq=0 2024-09-19T09:27:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.309+0000 7fd455ffb640 10 client.15333 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd455ffb640 1 -- 172.21.15.16:0/3593403306 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1f4ebea20 con 0x55f1f4f67bc0 2024-09-19T09:27:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd455ffb640 10 client.15333 handle_command_reply: tid=0 2024-09-19T09:27:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd47b1e0740 1 client.15333 shutdown 2024-09-19T09:27:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd47b1e0740 1 -- 172.21.15.16:0/3593403306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 msgr2=0x55f1f4f5bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/3593403306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 0x55f1f4f5bf50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd460009ca0 tx=0x7fd460009630 comp rx=0 tx=0).stop 2024-09-19T09:27:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd47b1e0740 1 -- 172.21.15.16:0/3593403306 shutdown_connections 2024-09-19T09:27:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/3593403306 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55f1f4f67bc0 0x55f1f4f6a010 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/3593403306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f5c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd455ffb640 0 client.15333 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/3593403306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f57780 0x55f1f4f634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/3593403306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f51db0 0x55f1f4f5bf50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd47b1e0740 1 -- 172.21.15.16:0/3593403306 >> 172.21.15.16:0/3593403306 conn(0x55f1f4f49010 msgr2=0x55f1f4f4a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd47b1e0740 1 -- 172.21.15.16:0/3593403306 shutdown_connections 2024-09-19T09:27:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd47b1e0740 1 -- 172.21.15.16:0/3593403306 wait complete. 2024-09-19T09:27:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.310+0000 7fd47b1e0740 20 client.15333 trim_cache size 0 max 16384 2024-09-19T09:27:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd47b1e0740 1 Processor -- start 2024-09-19T09:27:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd47b1e0740 1 -- start start 2024-09-19T09:27:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd47b1e0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f51db0 0x55f1f4f5c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd47b1e0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f633f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd47b1e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 0x55f1f4f49f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd47b1e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701a6e90 con 0x55f1f4f57780 2024-09-19T09:27:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd47b1e0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701a89d0 con 0x55f1f4f51db0 2024-09-19T09:27:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd47b1e0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701a7560 con 0x55f1f4f526f0 2024-09-19T09:27:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd476903640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 0x55f1f4f49f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd476903640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 0x55f1f4f49f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55154/0 (socket says 172.21.15.16:55154) 2024-09-19T09:27:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd476903640 1 -- 172.21.15.16:0/2637868133 learned_addr learned my addr 172.21.15.16:0/2637868133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd476102640 1 --2- 172.21.15.16:0/2637868133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f51db0 0x55f1f4f5c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd475901640 1 --2- 172.21.15.16:0/2637868133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd476903640 1 -- 172.21.15.16:0/2637868133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 msgr2=0x55f1f4f633f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd476903640 1 --2- 172.21.15.16:0/2637868133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f633f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd476903640 1 -- 172.21.15.16:0/2637868133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f51db0 msgr2=0x55f1f4f5c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd476903640 1 --2- 172.21.15.16:0/2637868133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f51db0 0x55f1f4f5c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd476903640 1 -- 172.21.15.16:0/2637868133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f4f56ba0 con 0x55f1f4f57780 2024-09-19T09:27:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd475901640 1 --2- 172.21.15.16:0/2637868133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f633f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.325+0000 7fd476903640 1 --2- 172.21.15.16:0/2637868133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 0x55f1f4f49f20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd468009b50 tx=0x7fd46800ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152e0f5c312d1d9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.326+0000 7fd457fff640 1 -- 172.21.15.16:0/2637868133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468011050 con 0x55f1f4f57780 2024-09-19T09:27:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.326+0000 7fd457fff640 1 -- 172.21.15.16:0/2637868133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd46801ddc0 con 0x55f1f4f57780 2024-09-19T09:27:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.326+0000 7fd457fff640 1 -- 172.21.15.16:0/2637868133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4680053e0 con 0x55f1f4f57780 2024-09-19T09:27:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.326+0000 7fd47b1e0740 1 -- 172.21.15.16:0/2637868133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 msgr2=0x55f1f4f49f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.326+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/2637868133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 0x55f1f4f49f20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd468009b50 tx=0x7fd46800ffd0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.326+0000 7fd47b1e0740 1 -- 172.21.15.16:0/2637868133 shutdown_connections 2024-09-19T09:27:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.326+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/2637868133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f633f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.326+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/2637868133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 0x55f1f4f49f20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.326+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/2637868133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f51db0 0x55f1f4f5c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.326+0000 7fd47b1e0740 1 -- 172.21.15.16:0/2637868133 >> 172.21.15.16:0/2637868133 conn(0x55f1f4f49010 msgr2=0x55f1f4f5bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.326+0000 7fd47b1e0740 1 -- 172.21.15.16:0/2637868133 shutdown_connections 2024-09-19T09:27:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.326+0000 7fd47b1e0740 1 -- 172.21.15.16:0/2637868133 wait complete. 2024-09-19T09:27:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.326+0000 7fd47b1e0740 1 -- start start 2024-09-19T09:27:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd47b1e0740 1 Processor -- start 2024-09-19T09:27:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd47b1e0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f51db0 0x55f1f4f60ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd47b1e0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f61020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd47b1e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 0x55f1f4f62030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd47b1e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701a8300 con 0x55f1f4f57780 2024-09-19T09:27:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd47b1e0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701a7c30 con 0x55f1f4f51db0 2024-09-19T09:27:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd47b1e0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701a4e70 con 0x55f1f4f526f0 2024-09-19T09:27:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd47b1e0740 10 client.0 _ll_register_callbacks cb 0x7fd460009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd476903640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 0x55f1f4f62030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd476903640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 0x55f1f4f62030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55166/0 (socket says 172.21.15.16:55166) 2024-09-19T09:27:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd476903640 1 -- 172.21.15.16:0/660748043 learned_addr learned my addr 172.21.15.16:0/660748043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd476102640 1 --2- 172.21.15.16:0/660748043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f51db0 0x55f1f4f60ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd475901640 1 --2- 172.21.15.16:0/660748043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f61020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd476903640 1 -- 172.21.15.16:0/660748043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 msgr2=0x55f1f4f61020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd476903640 1 --2- 172.21.15.16:0/660748043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f61020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd476903640 1 -- 172.21.15.16:0/660748043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f51db0 msgr2=0x55f1f4f60ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd476903640 1 --2- 172.21.15.16:0/660748043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f51db0 0x55f1f4f60ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd476903640 1 -- 172.21.15.16:0/660748043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f4f62640 con 0x55f1f4f57780 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd476903640 1 --2- 172.21.15.16:0/660748043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 0x55f1f4f62030 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd468011ad0 tx=0x7fd468009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e2685c06db9fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd455ffb640 1 -- 172.21.15.16:0/660748043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46803c020 con 0x55f1f4f57780 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd455ffb640 1 -- 172.21.15.16:0/660748043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd46801d8e0 con 0x55f1f4f57780 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.327+0000 7fd47b1e0740 1 -- 172.21.15.16:0/660748043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd46801ddc0 con 0x55f1f4f57780 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.328+0000 7fd455ffb640 1 -- 172.21.15.16:0/660748043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd46801dae0 con 0x55f1f4f57780 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.328+0000 7fd47b1e0740 10 client.15339 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.328+0000 7fd47b1e0740 1 -- 172.21.15.16:0/660748043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1f4fd28f0 con 0x55f1f4f57780 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.328+0000 7fd455ffb640 1 -- 172.21.15.16:0/660748043 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd46801ddc0 con 0x55f1f4f57780 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.328+0000 7fd455ffb640 1 -- 172.21.15.16:0/660748043 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd468041070 con 0x55f1f4f57780 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.328+0000 7fd47b1e0740 10 client.15339 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.328+0000 7fd47b1e0740 10 client.15339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.328+0000 7fd47b1e0740 20 client.15339 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.328+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/660748043 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55f1f4fd2fe0 0x55f1f4fd53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.328+0000 7fd47b1e0740 4 client.15339 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:27:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.328+0000 7fd47b1e0740 1 -- 172.21.15.16:0/660748043 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1f4eb8d70 con 0x55f1f4fd2fe0 2024-09-19T09:27:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.328+0000 7fd476102640 1 --2- 172.21.15.16:0/660748043 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55f1f4fd2fe0 0x55f1f4fd53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.329+0000 7fd455ffb640 10 client.15339 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.329+0000 7fd476102640 1 --2- 172.21.15.16:0/660748043 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55f1f4fd2fe0 0x55f1f4fd53d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50eb030ba6c834 server_cookie=6f2b85d4e81268d9 in_seq=0 out_seq=0 2024-09-19T09:27:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd455ffb640 1 -- 172.21.15.16:0/660748043 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1f4eb8d70 con 0x55f1f4fd2fe0 2024-09-19T09:27:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd455ffb640 10 client.15339 handle_command_reply: tid=0 2024-09-19T09:27:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd47b1e0740 1 client.15339 shutdown 2024-09-19T09:27:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd47b1e0740 1 -- 172.21.15.16:0/660748043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 msgr2=0x55f1f4f62030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/660748043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 0x55f1f4f62030 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd468011ad0 tx=0x7fd468009cd0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd47b1e0740 1 -- 172.21.15.16:0/660748043 shutdown_connections 2024-09-19T09:27:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/660748043 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55f1f4fd2fe0 0x55f1f4fd53d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/660748043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1f4f526f0 0x55f1f4f61020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd455ffb640 0 client.15339 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/660748043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1f4f57780 0x55f1f4f62030 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd47b1e0740 1 --2- 172.21.15.16:0/660748043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1f4f51db0 0x55f1f4f60ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd47b1e0740 1 -- 172.21.15.16:0/660748043 >> 172.21.15.16:0/660748043 conn(0x55f1f4f49010 msgr2=0x55f1f4f479a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd47b1e0740 1 -- 172.21.15.16:0/660748043 shutdown_connections 2024-09-19T09:27:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd47b1e0740 1 -- 172.21.15.16:0/660748043 wait complete. 2024-09-19T09:27:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.340+0000 7fd47b1e0740 20 client.15339 trim_cache size 0 max 16384 2024-09-19T09:27:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-19T09:27:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-19T09:27:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-19T09:27:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-19T09:27:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.341+0000 7fd478366640 1 -- 172.21.15.16:0/2739488728 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4580a32f0 msgr2=0x7fd4580a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.341+0000 7fd478366640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4580a32f0 0x7fd4580a57b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd46800f850 tx=0x7fd46801e000 comp rx=0 tx=0).stop 2024-09-19T09:27:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.341+0000 7fd478366640 1 -- 172.21.15.16:0/2739488728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd470101c30 msgr2=0x7fd470198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.341+0000 7fd478366640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd470101c30 0x7fd470198330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd46400c890 tx=0x7fd46400cd50 comp rx=0 tx=0).stop 2024-09-19T09:27:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.341+0000 7fd478366640 1 -- 172.21.15.16:0/2739488728 shutdown_connections 2024-09-19T09:27:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.341+0000 7fd478366640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd4580a32f0 0x7fd4580a57b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.341+0000 7fd478366640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4701025f0 0x7fd47019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.341+0000 7fd478366640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd470101c30 0x7fd470198330 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.341+0000 7fd478366640 1 --2- 172.21.15.16:0/2739488728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd470100a40 0x7fd470197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.341+0000 7fd478366640 1 -- 172.21.15.16:0/2739488728 >> 172.21.15.16:0/2739488728 conn(0x7fd4700fc1e0 msgr2=0x7fd4700698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.341+0000 7fd478366640 1 -- 172.21.15.16:0/2739488728 shutdown_connections 2024-09-19T09:27:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:39.341+0000 7fd478366640 1 -- 172.21.15.16:0/2739488728 wait complete. 2024-09-19T09:27:40.055 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:39 smithi016 ceph-mon[30979]: scrub summary: idle 2024-09-19T09:27:40.088 INFO:teuthology.orchestra.run.smithi016.stdout:{"run_dir":"/var/run/ceph"} 2024-09-19T09:27:40.088 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-19T09:27:40.088 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1747 2024-09-19T09:27:40.089 DEBUG:tasks.quiescer:Yielding 2024-09-19T09:27:40.089 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T09:27:40.102 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T09:27:40.103 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T09:27:40.103 INFO:tasks.workunit:timeout=3h 2024-09-19T09:27:40.103 INFO:tasks.workunit:cleanup=True 2024-09-19T09:27:40.103 DEBUG:teuthology.orchestra.run.smithi016:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T09:27:40.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:27:40.122 INFO:teuthology.orchestra.run.smithi016.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T09:27:40.122 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T09:27:40.122 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-19T09:27:40.122 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T09:27:40.122 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T09:27:40.122 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-19 09:25:47.867944171 +0000 2024-09-19T09:27:40.122 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-19 09:25:47.867944171 +0000 2024-09-19T09:27:40.122 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-19 09:27:18.034707061 +0000 2024-09-19T09:27:40.122 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-19 09:25:47.867944171 +0000 2024-09-19T09:27:40.123 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T09:27:40.123 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T09:27:40.196 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T09:27:40.252 INFO:tasks.workunit.client.0.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T09:27:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:40.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:39 smithi142 ceph-mon[35032]: scrub summary: idle 2024-09-19T09:27:40.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:39 smithi097 ceph-mon[34864]: scrub summary: idle 2024-09-19T09:27:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.822+0000 7f3bb7e12640 1 -- 172.21.15.16:0/3214028571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb0102db0 msgr2=0x7f3bb0103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.822+0000 7f3bb7e12640 1 --2- 172.21.15.16:0/3214028571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb0102db0 0x7f3bb0103190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009930 tx=0x7f3bac02a420 comp rx=0 tx=0).stop 2024-09-19T09:27:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.822+0000 7f3bb7e12640 1 -- 172.21.15.16:0/3214028571 shutdown_connections 2024-09-19T09:27:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.822+0000 7f3bb7e12640 1 --2- 172.21.15.16:0/3214028571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb01082e0 0x7f3bb010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.822+0000 7f3bb7e12640 1 --2- 172.21.15.16:0/3214028571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bb0103760 0x7f3bb0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:40.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.822+0000 7f3bb7e12640 1 --2- 172.21.15.16:0/3214028571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb0102db0 0x7f3bb0103190 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.822+0000 7f3bb7e12640 1 -- 172.21.15.16:0/3214028571 >> 172.21.15.16:0/3214028571 conn(0x7f3bb00fc7d0 msgr2=0x7f3bb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.822+0000 7f3bb7e12640 1 -- 172.21.15.16:0/3214028571 shutdown_connections 2024-09-19T09:27:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.822+0000 7f3bb7e12640 1 -- 172.21.15.16:0/3214028571 wait complete. 2024-09-19T09:27:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.823+0000 7f3bb7e12640 1 Processor -- start 2024-09-19T09:27:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.823+0000 7f3bb7e12640 1 -- start start 2024-09-19T09:27:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.823+0000 7f3bb7e12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bb0102db0 0x7f3bb0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.823+0000 7f3bb7e12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb0103760 0x7f3bb0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.823+0000 7f3bb7e12640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb01082e0 0x7f3bb019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.823+0000 7f3bb7e12640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01120e0 con 0x7f3bb0103760 2024-09-19T09:27:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.823+0000 7f3bb7e12640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0111f60 con 0x7f3bb0102db0 2024-09-19T09:27:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.823+0000 7f3bb7e12640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0112260 con 0x7f3bb01082e0 2024-09-19T09:27:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.823+0000 7f3bb53ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb0103760 0x7f3bb0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.823+0000 7f3bb53ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb0103760 0x7f3bb0198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55182/0 (socket says 172.21.15.16:55182) 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.823+0000 7f3bb53ad640 1 -- 172.21.15.16:0/860851676 learned_addr learned my addr 172.21.15.16:0/860851676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.823+0000 7f3bb63af640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb01082e0 0x7f3bb019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.823+0000 7f3bb5bae640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bb0102db0 0x7f3bb0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.824+0000 7f3bb53ad640 1 -- 172.21.15.16:0/860851676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb01082e0 msgr2=0x7f3bb019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.824+0000 7f3bb53ad640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb01082e0 0x7f3bb019cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.824+0000 7f3bb53ad640 1 -- 172.21.15.16:0/860851676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bb0102db0 msgr2=0x7f3bb0198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.824+0000 7f3bb53ad640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bb0102db0 0x7f3bb0198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.824+0000 7f3bb53ad640 1 -- 172.21.15.16:0/860851676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb019d4c0 con 0x7f3bb0103760 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.824+0000 7f3bb63af640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb01082e0 0x7f3bb019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.824+0000 7f3bb53ad640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb0103760 0x7f3bb0198a50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c8c0 tx=0x7f3ba000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.824+0000 7f3ba6ffd640 1 -- 172.21.15.16:0/860851676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0007bb0 con 0x7f3bb0103760 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.824+0000 7f3ba6ffd640 1 -- 172.21.15.16:0/860851676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba0007d50 con 0x7f3bb0103760 2024-09-19T09:27:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.824+0000 7f3bb7e12640 1 -- 172.21.15.16:0/860851676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb019d7b0 con 0x7f3bb0103760 2024-09-19T09:27:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.824+0000 7f3ba6ffd640 1 -- 172.21.15.16:0/860851676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0018830 con 0x7f3bb0103760 2024-09-19T09:27:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.825+0000 7f3bb7e12640 1 -- 172.21.15.16:0/860851676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb01a5090 con 0x7f3bb0103760 2024-09-19T09:27:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.826+0000 7f3bb7e12640 1 -- 172.21.15.16:0/860851676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb006a7d0 con 0x7f3bb0103760 2024-09-19T09:27:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.828+0000 7f3ba6ffd640 1 -- 172.21.15.16:0/860851676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3ba00189f0 con 0x7f3bb0103760 2024-09-19T09:27:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.828+0000 7f3ba6ffd640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3b9c0a32c0 0x7f3b9c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.829+0000 7f3bb5bae640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3b9c0a32c0 0x7f3b9c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.829+0000 7f3ba6ffd640 1 -- 172.21.15.16:0/860851676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3ba00cfbc0 con 0x7f3bb0103760 2024-09-19T09:27:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.829+0000 7f3bb5bae640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3b9c0a32c0 0x7f3b9c0a5780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3bac006010 tx=0x7f3bac000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:40.832+0000 7f3ba6ffd640 1 -- 172.21.15.16:0/860851676 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3ba0014030 con 0x7f3bb0103760 2024-09-19T09:27:41.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.048+0000 7f3bb7e12640 1 -- 172.21.15.16:0/860851676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bb0103190 con 0x7f3bb0103760 2024-09-19T09:27:41.050 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:40 smithi016 ceph-mon[30979]: pgmap v340: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-19T09:27:41.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.049+0000 7f3ba6ffd640 1 -- 172.21.15.16:0/860851676 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f3bb0103190 con 0x7f3bb0103760 2024-09-19T09:27:41.052 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:27:41.052 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:27:41.054 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:27:41.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.051+0000 7f3bb7e12640 1 -- 172.21.15.16:0/860851676 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3b9c0a32c0 msgr2=0x7f3b9c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:41.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.051+0000 7f3bb7e12640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3b9c0a32c0 0x7f3b9c0a5780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3bac006010 tx=0x7f3bac000bc0 comp rx=0 tx=0).stop 2024-09-19T09:27:41.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.051+0000 7f3bb7e12640 1 -- 172.21.15.16:0/860851676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb0103760 msgr2=0x7f3bb0198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:41.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.051+0000 7f3bb7e12640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb0103760 0x7f3bb0198a50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c8c0 tx=0x7f3ba000cd80 comp rx=0 tx=0).stop 2024-09-19T09:27:41.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.052+0000 7f3bb7e12640 1 -- 172.21.15.16:0/860851676 shutdown_connections 2024-09-19T09:27:41.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.052+0000 7f3bb7e12640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3b9c0a32c0 0x7f3b9c0a5780 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:41.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.052+0000 7f3bb7e12640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb01082e0 0x7f3bb019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:41.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.052+0000 7f3bb7e12640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb0103760 0x7f3bb0198a50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:41.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.052+0000 7f3bb7e12640 1 --2- 172.21.15.16:0/860851676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bb0102db0 0x7f3bb0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:41.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.052+0000 7f3bb7e12640 1 -- 172.21.15.16:0/860851676 >> 172.21.15.16:0/860851676 conn(0x7f3bb00fc7d0 msgr2=0x7f3bb0106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:41.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.052+0000 7f3bb7e12640 1 -- 172.21.15.16:0/860851676 shutdown_connections 2024-09-19T09:27:41.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:41.052+0000 7f3bb7e12640 1 -- 172.21.15.16:0/860851676 wait complete. 2024-09-19T09:27:41.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:40 smithi142 ceph-mon[35032]: pgmap v340: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-19T09:27:41.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:40 smithi097 ceph-mon[34864]: pgmap v340: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-19T09:27:41.701 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T09:25:29.829112+0000', 'modified': '2024-09-19T09:27:18.729819+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24500, 'mds_1': 24484, 'mds_2': 24470, 'mds_3': 24428, 'mds_4': 24424}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14580': {'gid': 14580, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/3573603820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 3573603820}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 3573603820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14592': {'gid': 14592, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/4017062522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 4017062522}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 4017062522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.142:6833/3906366829', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 3906366829}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 3906366829}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.97:6833/108652274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6832', 'nonce': 108652274}, {'type': 'v1', 'addr': '172.21.15.97:6833', 'nonce': 108652274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/2657443138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 2657443138}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 2657443138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/4135704970', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 4135704970}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 4135704970}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6837/81091251', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 81091251}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 81091251}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.97:6837/2018284957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 2018284957}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 2018284957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.142:6839/3822134597', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 3822134597}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 3822134597}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.97:6839/3225392797', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3225392797}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3225392797}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24500, 'qdb_cluster': [24424, 24428, 24470, 24484, 24500]} 2024-09-19T09:27:41.701 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T09:27:41.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.l status 2024-09-19T09:27:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:42.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:41 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/860851676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:42.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:41 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/860851676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:42.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:41 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/860851676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:27:42.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.543+0000 7fa2d8011640 1 -- 172.21.15.16:0/352330146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d0101e10 msgr2=0x7fa2d01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:42.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.543+0000 7fa2d8011640 1 --2- 172.21.15.16:0/352330146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d0101e10 0x7fa2d01021f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009900 tx=0x7fa2cc02a410 comp rx=0 tx=0).stop 2024-09-19T09:27:42.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.543+0000 7fa2d8011640 1 -- 172.21.15.16:0/352330146 shutdown_connections 2024-09-19T09:27:42.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.543+0000 7fa2d8011640 1 --2- 172.21.15.16:0/352330146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d0106f40 0x7fa2d010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.543+0000 7fa2d8011640 1 --2- 172.21.15.16:0/352330146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d01027c0 0x7fa2d0106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.543+0000 7fa2d8011640 1 --2- 172.21.15.16:0/352330146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d0101e10 0x7fa2d01021f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.543+0000 7fa2d8011640 1 -- 172.21.15.16:0/352330146 >> 172.21.15.16:0/352330146 conn(0x7fa2d00fd8d0 msgr2=0x7fa2d00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.543+0000 7fa2d8011640 1 -- 172.21.15.16:0/352330146 shutdown_connections 2024-09-19T09:27:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.543+0000 7fa2d8011640 1 -- 172.21.15.16:0/352330146 wait complete. 2024-09-19T09:27:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.543+0000 7fa2d8011640 1 Processor -- start 2024-09-19T09:27:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d8011640 1 -- start start 2024-09-19T09:27:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d8011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d0101e10 0x7fa2d00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d8011640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d01027c0 0x7fa2d0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d8011640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d0106f40 0x7fa2d0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d8011640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d0111fa0 con 0x7fa2d0101e10 2024-09-19T09:27:42.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d8011640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d0111e20 con 0x7fa2d0106f40 2024-09-19T09:27:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d8011640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d0112120 con 0x7fa2d01027c0 2024-09-19T09:27:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d5dad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d0101e10 0x7fa2d00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d5dad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d0101e10 0x7fa2d00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55210/0 (socket says 172.21.15.16:55210) 2024-09-19T09:27:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d5dad640 1 -- 172.21.15.16:0/2621083540 learned_addr learned my addr 172.21.15.16:0/2621083540 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d65ae640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d0106f40 0x7fa2d0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d55ac640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d01027c0 0x7fa2d0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d5dad640 1 -- 172.21.15.16:0/2621083540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d01027c0 msgr2=0x7fa2d0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d5dad640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d01027c0 0x7fa2d0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d5dad640 1 -- 172.21.15.16:0/2621083540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d0106f40 msgr2=0x7fa2d0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d5dad640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d0106f40 0x7fa2d0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d5dad640 1 -- 172.21.15.16:0/2621083540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d01a6890 con 0x7fa2d0101e10 2024-09-19T09:27:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.544+0000 7fa2d55ac640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d01027c0 0x7fa2d0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:42.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.545+0000 7fa2d5dad640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d0101e10 0x7fa2d00744f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc006010 tx=0x7fa2cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:42.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.545+0000 7fa2c6ffd640 1 -- 172.21.15.16:0/2621083540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc03f030 con 0x7fa2d0101e10 2024-09-19T09:27:42.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.545+0000 7fa2c6ffd640 1 -- 172.21.15.16:0/2621083540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2cc003ce0 con 0x7fa2d0101e10 2024-09-19T09:27:42.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.545+0000 7fa2c6ffd640 1 -- 172.21.15.16:0/2621083540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc035070 con 0x7fa2d0101e10 2024-09-19T09:27:42.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.545+0000 7fa2d8011640 1 -- 172.21.15.16:0/2621083540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d01a6b20 con 0x7fa2d0101e10 2024-09-19T09:27:42.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.545+0000 7fa2d8011640 1 -- 172.21.15.16:0/2621083540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d01a7050 con 0x7fa2d0101e10 2024-09-19T09:27:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.547+0000 7fa2c6ffd640 1 -- 172.21.15.16:0/2621083540 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa2cc042030 con 0x7fa2d0101e10 2024-09-19T09:27:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.547+0000 7fa2da3ea740 1 Processor -- start 2024-09-19T09:27:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2da3ea740 1 -- start start 2024-09-19T09:27:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2da3ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5789d0 0x56540b578db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2da3ea740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5792f0 0x56540b583670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2da3ea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b583bb0 0x56540b585fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2c6ffd640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa2bc0a32c0 0x7fa2bc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2da3ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56540b4df090 con 0x56540b5789d0 2024-09-19T09:27:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2da3ea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56540b486a70 con 0x56540b583bb0 2024-09-19T09:27:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2da3ea740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56540b4db050 con 0x56540b5792f0 2024-09-19T09:27:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2d65ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5789d0 0x56540b578db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2d65ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5789d0 0x56540b578db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55226/0 (socket says 172.21.15.16:55226) 2024-09-19T09:27:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2d65ae640 1 -- 172.21.15.16:0/2412727788 learned_addr learned my addr 172.21.15.16:0/2412727788 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2c6ffd640 1 -- 172.21.15.16:0/2621083540 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa2cc0efec0 con 0x7fa2d0101e10 2024-09-19T09:27:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2d5dad640 1 --2- 172.21.15.16:0/2412727788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5792f0 0x56540b583670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2d65ae640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa2bc0a32c0 0x7fa2bc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2d55ac640 1 --2- 172.21.15.16:0/2412727788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b583bb0 0x56540b585fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.548+0000 7fa2d65ae640 1 -- 172.21.15.16:0/2412727788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5792f0 msgr2=0x56540b583670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.549+0000 7fa2d65ae640 1 --2- 172.21.15.16:0/2412727788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5792f0 0x56540b583670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.549+0000 7fa2d65ae640 1 -- 172.21.15.16:0/2412727788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b583bb0 msgr2=0x56540b585fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.549+0000 7fa2d65ae640 1 --2- 172.21.15.16:0/2412727788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b583bb0 0x56540b585fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.549+0000 7fa2d65ae640 1 -- 172.21.15.16:0/2412727788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56540b493130 con 0x56540b5789d0 2024-09-19T09:27:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.549+0000 7fa2d65ae640 1 --2- 172.21.15.16:0/2412727788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5789d0 0x56540b578db0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8013e60 tx=0x7fa2c80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8ba14ca7e9235f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.549+0000 7fa2d65ae640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa2bc0a32c0 0x7fa2bc0a5780 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8011cb0 tx=0x7fa2c801c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.549+0000 7fa2bbfff640 1 -- 172.21.15.16:0/2412727788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800ccb0 con 0x56540b5789d0 2024-09-19T09:27:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.549+0000 7fa2bbfff640 1 -- 172.21.15.16:0/2412727788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c800ce50 con 0x56540b5789d0 2024-09-19T09:27:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.549+0000 7fa2bbfff640 1 -- 172.21.15.16:0/2412727788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800f450 con 0x56540b5789d0 2024-09-19T09:27:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.550+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2412727788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5789d0 msgr2=0x56540b578db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.550+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2412727788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5789d0 0x56540b578db0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8013e60 tx=0x7fa2c80117b0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.550+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2412727788 shutdown_connections 2024-09-19T09:27:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.550+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2412727788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5792f0 0x56540b583670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.550+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2412727788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b583bb0 0x56540b585fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.550+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2412727788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5789d0 0x56540b578db0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.550+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2412727788 >> 172.21.15.16:0/2412727788 conn(0x56540b574f60 msgr2=0x56540b575340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.550+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2412727788 shutdown_connections 2024-09-19T09:27:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.550+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2412727788 wait complete. 2024-09-19T09:27:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.550+0000 7fa2da3ea740 1 -- start start 2024-09-19T09:27:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.550+0000 7fa2da3ea740 1 Processor -- start 2024-09-19T09:27:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2da3ea740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5789d0 0x56540b583490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2da3ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 0x56540b5807e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2da3ea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b583bb0 0x56540b580d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2da3ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56540b586780 con 0x56540b5792f0 2024-09-19T09:27:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2da3ea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56540b499f20 con 0x56540b583bb0 2024-09-19T09:27:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2da3ea740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56540b586900 con 0x56540b5789d0 2024-09-19T09:27:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2da3ea740 10 client.0 _ll_register_callbacks cb 0x56540b463030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2d55ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 0x56540b5807e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2d55ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 0x56540b5807e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55230/0 (socket says 172.21.15.16:55230) 2024-09-19T09:27:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2d55ac640 1 -- 172.21.15.16:0/2238054454 learned_addr learned my addr 172.21.15.16:0/2238054454 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2d5dad640 1 --2- 172.21.15.16:0/2238054454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5789d0 0x56540b583490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2d65ae640 1 --2- 172.21.15.16:0/2238054454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b583bb0 0x56540b580d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2d55ac640 1 -- 172.21.15.16:0/2238054454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5789d0 msgr2=0x56540b583490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2d55ac640 1 --2- 172.21.15.16:0/2238054454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5789d0 0x56540b583490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2d55ac640 1 -- 172.21.15.16:0/2238054454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b583bb0 msgr2=0x56540b580d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2d55ac640 1 --2- 172.21.15.16:0/2238054454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b583bb0 0x56540b580d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.551+0000 7fa2d55ac640 1 -- 172.21.15.16:0/2238054454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56540b2502b0 con 0x56540b5792f0 2024-09-19T09:27:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2d55ac640 1 --2- 172.21.15.16:0/2238054454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 0x56540b5807e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000bec0 tx=0x7fa2c00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ebbb309ef09898 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2b97fa640 1 -- 172.21.15.16:0/2238054454 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0012650 con 0x56540b5792f0 2024-09-19T09:27:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2b97fa640 1 -- 172.21.15.16:0/2238054454 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa2c00127f0 con 0x56540b5792f0 2024-09-19T09:27:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2238054454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c800ce50 con 0x56540b5792f0 2024-09-19T09:27:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2b97fa640 1 -- 172.21.15.16:0/2238054454 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2c00129f0 con 0x56540b5792f0 2024-09-19T09:27:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2da3ea740 10 client.15357 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2238054454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56540b592150 con 0x56540b5792f0 2024-09-19T09:27:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2b97fa640 1 -- 172.21.15.16:0/2238054454 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c800ce50 con 0x56540b5792f0 2024-09-19T09:27:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2b97fa640 1 -- 172.21.15.16:0/2238054454 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa2c001bce0 con 0x56540b5792f0 2024-09-19T09:27:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2da3ea740 10 client.15357 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2da3ea740 10 client.15357 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-19T09:27:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2da3ea740 20 client.15357 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2238054454 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x56540b594780 0x56540b595bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2da3ea740 4 client.15357 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.552+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2238054454 --> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56540b4ec600 con 0x56540b594780 2024-09-19T09:27:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.553+0000 7fa2d5dad640 1 --2- 172.21.15.16:0/2238054454 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x56540b594780 0x56540b595bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.553+0000 7fa2b97fa640 10 client.15357 ms_handle_connect on v2:172.21.15.142:6832/3906366829 2024-09-19T09:27:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.553+0000 7fa2d5dad640 1 --2- 172.21.15.16:0/2238054454 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x56540b594780 0x56540b595bf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=988e09e648223828 server_cookie=d6e4d49d36f57af2 in_seq=0 out_seq=0 2024-09-19T09:27:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2b97fa640 1 -- 172.21.15.16:0/2238054454 <== mds.4 v2:172.21.15.142:6832/3906366829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56540b4ec600 con 0x56540b594780 2024-09-19T09:27:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2b97fa640 10 client.15357 handle_command_reply: tid=0 2024-09-19T09:27:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2da3ea740 1 client.15357 shutdown 2024-09-19T09:27:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2238054454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 msgr2=0x56540b5807e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2238054454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 0x56540b5807e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000bec0 tx=0x7fa2c00117b0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2238054454 shutdown_connections 2024-09-19T09:27:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2238054454 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x56540b594780 0x56540b595bf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2238054454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b583bb0 0x56540b580d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2238054454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 0x56540b5807e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2b97fa640 0 client.15357 ms_handle_reset on v2:172.21.15.142:6832/3906366829 2024-09-19T09:27:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2238054454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5789d0 0x56540b583490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2238054454 >> 172.21.15.16:0/2238054454 conn(0x56540b574f60 msgr2=0x56540b57ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2238054454 shutdown_connections 2024-09-19T09:27:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2238054454 wait complete. 2024-09-19T09:27:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.554+0000 7fa2da3ea740 20 client.15357 trim_cache size 0 max 16384 2024-09-19T09:27:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.572+0000 7fa2da3ea740 1 Processor -- start 2024-09-19T09:27:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.572+0000 7fa2da3ea740 1 -- start start 2024-09-19T09:27:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2da3ea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b5789d0 0x56540b582640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2da3ea740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5792f0 0x56540b57da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2da3ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b583bb0 0x56540b57dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2da3ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56540b593f30 con 0x56540b583bb0 2024-09-19T09:27:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2da3ea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56540b594600 con 0x56540b5789d0 2024-09-19T09:27:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2da3ea740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56540b594cd0 con 0x56540b5792f0 2024-09-19T09:27:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2d65ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b583bb0 0x56540b57dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2d65ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b583bb0 0x56540b57dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55238/0 (socket says 172.21.15.16:55238) 2024-09-19T09:27:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2d65ae640 1 -- 172.21.15.16:0/2633297419 learned_addr learned my addr 172.21.15.16:0/2633297419 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2d5dad640 1 --2- 172.21.15.16:0/2633297419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b5789d0 0x56540b582640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2d55ac640 1 --2- 172.21.15.16:0/2633297419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5792f0 0x56540b57da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2d65ae640 1 -- 172.21.15.16:0/2633297419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5792f0 msgr2=0x56540b57da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:42.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2d65ae640 1 --2- 172.21.15.16:0/2633297419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5792f0 0x56540b57da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2d65ae640 1 -- 172.21.15.16:0/2633297419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b5789d0 msgr2=0x56540b582640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2d65ae640 1 --2- 172.21.15.16:0/2633297419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b5789d0 0x56540b582640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2d65ae640 1 -- 172.21.15.16:0/2633297419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56540b575ff0 con 0x56540b583bb0 2024-09-19T09:27:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2d65ae640 1 --2- 172.21.15.16:0/2633297419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b583bb0 0x56540b57dfc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8013e60 tx=0x7fa2c800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae39e52a7d03bdb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2bb7fe640 1 -- 172.21.15.16:0/2633297419 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c80056d0 con 0x56540b583bb0 2024-09-19T09:27:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2bb7fe640 1 -- 172.21.15.16:0/2633297419 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c8005870 con 0x56540b583bb0 2024-09-19T09:27:42.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.573+0000 7fa2bb7fe640 1 -- 172.21.15.16:0/2633297419 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8003970 con 0x56540b583bb0 2024-09-19T09:27:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.574+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2633297419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b583bb0 msgr2=0x56540b57dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.574+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2633297419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b583bb0 0x56540b57dfc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8013e60 tx=0x7fa2c800cd60 comp rx=0 tx=0).stop 2024-09-19T09:27:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.574+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2633297419 shutdown_connections 2024-09-19T09:27:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.574+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2633297419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b5792f0 0x56540b57da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.574+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2633297419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b583bb0 0x56540b57dfc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.574+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/2633297419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b5789d0 0x56540b582640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.574+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2633297419 >> 172.21.15.16:0/2633297419 conn(0x56540b574f60 msgr2=0x56540b581f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.574+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2633297419 shutdown_connections 2024-09-19T09:27:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.574+0000 7fa2da3ea740 1 -- 172.21.15.16:0/2633297419 wait complete. 2024-09-19T09:27:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.574+0000 7fa2da3ea740 1 -- start start 2024-09-19T09:27:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.575+0000 7fa2da3ea740 1 Processor -- start 2024-09-19T09:27:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.575+0000 7fa2da3ea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b5789d0 0x56540b59e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.575+0000 7fa2da3ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 0x56540b5997c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.575+0000 7fa2da3ea740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b583bb0 0x56540b599d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.575+0000 7fa2da3ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56540b595a70 con 0x56540b5792f0 2024-09-19T09:27:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.575+0000 7fa2da3ea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56540b5953a0 con 0x56540b5789d0 2024-09-19T09:27:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.575+0000 7fa2da3ea740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56540b596140 con 0x56540b583bb0 2024-09-19T09:27:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.575+0000 7fa2da3ea740 10 client.0 _ll_register_callbacks cb 0x7fa2c000b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.575+0000 7fa2d55ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 0x56540b5997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.575+0000 7fa2d55ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 0x56540b5997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55240/0 (socket says 172.21.15.16:55240) 2024-09-19T09:27:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.575+0000 7fa2d55ac640 1 -- 172.21.15.16:0/301442068 learned_addr learned my addr 172.21.15.16:0/301442068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.575+0000 7fa2d65ae640 1 --2- 172.21.15.16:0/301442068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b583bb0 0x56540b599d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.575+0000 7fa2d5dad640 1 --2- 172.21.15.16:0/301442068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b5789d0 0x56540b59e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2d55ac640 1 -- 172.21.15.16:0/301442068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b583bb0 msgr2=0x56540b599d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2d55ac640 1 --2- 172.21.15.16:0/301442068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b583bb0 0x56540b599d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2d55ac640 1 -- 172.21.15.16:0/301442068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b5789d0 msgr2=0x56540b59e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2d55ac640 1 --2- 172.21.15.16:0/301442068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b5789d0 0x56540b59e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2d55ac640 1 -- 172.21.15.16:0/301442068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56540b59a3a0 con 0x56540b5792f0 2024-09-19T09:27:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2d55ac640 1 --2- 172.21.15.16:0/301442068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 0x56540b5997c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0025480 tx=0x7fa2c0011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74f25f85d53e96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2b97fa640 1 -- 172.21.15.16:0/301442068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c002b040 con 0x56540b5792f0 2024-09-19T09:27:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2b97fa640 1 -- 172.21.15.16:0/301442068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa2c0012450 con 0x56540b5792f0 2024-09-19T09:27:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2da3ea740 1 -- 172.21.15.16:0/301442068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c8005870 con 0x56540b5792f0 2024-09-19T09:27:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2b97fa640 1 -- 172.21.15.16:0/301442068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2c0012e10 con 0x56540b5792f0 2024-09-19T09:27:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2da3ea740 10 client.15363 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2da3ea740 1 -- 172.21.15.16:0/301442068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56540b5fed50 con 0x56540b5792f0 2024-09-19T09:27:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2b97fa640 1 -- 172.21.15.16:0/301442068 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c8005870 con 0x56540b5792f0 2024-09-19T09:27:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.576+0000 7fa2b97fa640 1 -- 172.21.15.16:0/301442068 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa2c0025900 con 0x56540b5792f0 2024-09-19T09:27:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.577+0000 7fa2da3ea740 10 client.15363 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.577+0000 7fa2da3ea740 10 client.15363 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-19T09:27:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.577+0000 7fa2da3ea740 20 client.15363 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.577+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/301442068 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x56540b594780 0x56540b602660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.577+0000 7fa2da3ea740 4 client.15363 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:27:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.577+0000 7fa2da3ea740 1 -- 172.21.15.16:0/301442068 --> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] -- command(tid 0: {"prefix": "status"}) -- 0x56540b4e6b10 con 0x56540b594780 2024-09-19T09:27:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.577+0000 7fa2d5dad640 1 --2- 172.21.15.16:0/301442068 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x56540b594780 0x56540b602660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.577+0000 7fa2b97fa640 10 client.15363 ms_handle_connect on v2:172.21.15.142:6832/3906366829 2024-09-19T09:27:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.577+0000 7fa2d5dad640 1 --2- 172.21.15.16:0/301442068 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x56540b594780 0x56540b602660 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8d6210320ca5e95 server_cookie=f95ee889a671a7d2 in_seq=0 out_seq=0 2024-09-19T09:27:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2b97fa640 1 -- 172.21.15.16:0/301442068 <== mds.4 v2:172.21.15.142:6832/3906366829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56540b4e6b10 con 0x56540b594780 2024-09-19T09:27:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2b97fa640 10 client.15363 handle_command_reply: tid=0 2024-09-19T09:27:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2da3ea740 1 client.15363 shutdown 2024-09-19T09:27:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2da3ea740 1 -- 172.21.15.16:0/301442068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 msgr2=0x56540b5997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/301442068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 0x56540b5997c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0025480 tx=0x7fa2c0011df0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2da3ea740 1 -- 172.21.15.16:0/301442068 shutdown_connections 2024-09-19T09:27:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/301442068 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x56540b594780 0x56540b602660 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/301442068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56540b583bb0 0x56540b599d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/301442068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56540b5792f0 0x56540b5997c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2da3ea740 1 --2- 172.21.15.16:0/301442068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56540b5789d0 0x56540b59e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2b97fa640 0 client.15363 ms_handle_reset on v2:172.21.15.142:6832/3906366829 2024-09-19T09:27:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2da3ea740 1 -- 172.21.15.16:0/301442068 >> 172.21.15.16:0/301442068 conn(0x56540b574f60 msgr2=0x56540b581e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2da3ea740 1 -- 172.21.15.16:0/301442068 shutdown_connections 2024-09-19T09:27:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2da3ea740 1 -- 172.21.15.16:0/301442068 wait complete. 2024-09-19T09:27:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.578+0000 7fa2da3ea740 20 client.15363 trim_cache size 0 max 16384 2024-09-19T09:27:42.585 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:27:42.585 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:27:42.585 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-09-19T09:27:42.585 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24424, 2024-09-19T09:27:42.586 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:27:42.586 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:27:42.586 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:27:42.586 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 122.611386067, 2024-09-19T09:27:42.586 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-19T09:27:42.586 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:27:42.586 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T09:27:42.586 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 173.03023897700001 2024-09-19T09:27:42.586 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:27:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.579+0000 7fa2d8011640 1 -- 172.21.15.16:0/2621083540 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa2bc0a32c0 msgr2=0x7fa2bc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.579+0000 7fa2d8011640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa2bc0a32c0 0x7fa2bc0a5780 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8011cb0 tx=0x7fa2c801c000 comp rx=0 tx=0).stop 2024-09-19T09:27:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.579+0000 7fa2d8011640 1 -- 172.21.15.16:0/2621083540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d0101e10 msgr2=0x7fa2d00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.579+0000 7fa2d8011640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d0101e10 0x7fa2d00744f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc006010 tx=0x7fa2cc0042f0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.579+0000 7fa2d8011640 1 -- 172.21.15.16:0/2621083540 shutdown_connections 2024-09-19T09:27:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.579+0000 7fa2d8011640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa2bc0a32c0 0x7fa2bc0a5780 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.579+0000 7fa2d8011640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d01027c0 0x7fa2d0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.579+0000 7fa2d8011640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d0106f40 0x7fa2d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.579+0000 7fa2d8011640 1 --2- 172.21.15.16:0/2621083540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d0101e10 0x7fa2d00744f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.579+0000 7fa2d8011640 1 -- 172.21.15.16:0/2621083540 >> 172.21.15.16:0/2621083540 conn(0x7fa2d00fd8d0 msgr2=0x7fa2d00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.579+0000 7fa2d8011640 1 -- 172.21.15.16:0/2621083540 shutdown_connections 2024-09-19T09:27:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:42.579+0000 7fa2d8011640 1 -- 172.21.15.16:0/2621083540 wait complete. 2024-09-19T09:27:43.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:42 smithi016 ceph-mon[30979]: pgmap v341: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:27:43.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.k status 2024-09-19T09:27:43.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:42 smithi142 ceph-mon[35032]: pgmap v341: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:27:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:42 smithi097 ceph-mon[34864]: pgmap v341: 97 pgs: 97 active+clean; 650 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:27:43.575 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.075+0000 7fd6e643c640 1 -- 172.21.15.16:0/2391306727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e0101de0 msgr2=0x7fd6e01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.075+0000 7fd6e643c640 1 --2- 172.21.15.16:0/2391306727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e0101de0 0x7fd6e01021c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009900 tx=0x7fd6d002a3f0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.075+0000 7fd6e643c640 1 -- 172.21.15.16:0/2391306727 shutdown_connections 2024-09-19T09:27:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.075+0000 7fd6e643c640 1 --2- 172.21.15.16:0/2391306727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0102790 0x7fd6e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.075+0000 7fd6e643c640 1 --2- 172.21.15.16:0/2391306727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e0106f10 0x7fd6e010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.075+0000 7fd6e643c640 1 --2- 172.21.15.16:0/2391306727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e0101de0 0x7fd6e01021c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.075+0000 7fd6e643c640 1 -- 172.21.15.16:0/2391306727 >> 172.21.15.16:0/2391306727 conn(0x7fd6e00fd8c0 msgr2=0x7fd6e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.075+0000 7fd6e643c640 1 -- 172.21.15.16:0/2391306727 shutdown_connections 2024-09-19T09:27:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.075+0000 7fd6e643c640 1 -- 172.21.15.16:0/2391306727 wait complete. 2024-09-19T09:27:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.076+0000 7fd6e643c640 1 Processor -- start 2024-09-19T09:27:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.076+0000 7fd6e643c640 1 -- start start 2024-09-19T09:27:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.076+0000 7fd6e643c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e0101de0 0x7fd6e01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.076+0000 7fd6e643c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e0102790 0x7fd6e0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.076+0000 7fd6e643c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0106f10 0x7fd6e019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.076+0000 7fd6e643c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e0111f70 con 0x7fd6e0102790 2024-09-19T09:27:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.076+0000 7fd6e643c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e0111df0 con 0x7fd6e0101de0 2024-09-19T09:27:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.076+0000 7fd6e643c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01120f0 con 0x7fd6e0106f10 2024-09-19T09:27:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.076+0000 7fd6df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e0102790 0x7fd6e0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.076+0000 7fd6df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e0102790 0x7fd6e0198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55268/0 (socket says 172.21.15.16:55268) 2024-09-19T09:27:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.076+0000 7fd6df7fe640 1 -- 172.21.15.16:0/3052119636 learned_addr learned my addr 172.21.15.16:0/3052119636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.077+0000 7fd6e49d9640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0106f10 0x7fd6e019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.077+0000 7fd6dffff640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e0101de0 0x7fd6e01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.077+0000 7fd6df7fe640 1 -- 172.21.15.16:0/3052119636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0106f10 msgr2=0x7fd6e019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.077+0000 7fd6df7fe640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0106f10 0x7fd6e019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.077+0000 7fd6df7fe640 1 -- 172.21.15.16:0/3052119636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e0101de0 msgr2=0x7fd6e01982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.077+0000 7fd6df7fe640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e0101de0 0x7fd6e01982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.077+0000 7fd6df7fe640 1 -- 172.21.15.16:0/3052119636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e019d2a0 con 0x7fd6e0102790 2024-09-19T09:27:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.077+0000 7fd6df7fe640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e0102790 0x7fd6e0198830 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c920 tx=0x7fd6c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.077+0000 7fd6dd7fa640 1 -- 172.21.15.16:0/3052119636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8007c70 con 0x7fd6e0102790 2024-09-19T09:27:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.077+0000 7fd6e643c640 1 -- 172.21.15.16:0/3052119636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e019d590 con 0x7fd6e0102790 2024-09-19T09:27:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.077+0000 7fd6dd7fa640 1 -- 172.21.15.16:0/3052119636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6c8007e10 con 0x7fd6e0102790 2024-09-19T09:27:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.077+0000 7fd6dd7fa640 1 -- 172.21.15.16:0/3052119636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8018890 con 0x7fd6e0102790 2024-09-19T09:27:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.077+0000 7fd6e643c640 1 -- 172.21.15.16:0/3052119636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e01a4e70 con 0x7fd6e0102790 2024-09-19T09:27:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.079+0000 7fd6dd7fa640 1 -- 172.21.15.16:0/3052119636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd6c8018a70 con 0x7fd6e0102790 2024-09-19T09:27:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6e8815740 1 Processor -- start 2024-09-19T09:27:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6e8815740 1 -- start start 2024-09-19T09:27:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6e8815740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 0x55ccf7a22c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6e8815740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 0x55ccf7a31a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6e8815740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6e8815740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7989090 con 0x55ccf7a231d0 2024-09-19T09:27:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6e8815740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7930a70 con 0x55ccf7a228b0 2024-09-19T09:27:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6e8815740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7985050 con 0x55ccf7a31fc0 2024-09-19T09:27:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 0x55ccf7a31a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 0x55ccf7a31a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55278/0 (socket says 172.21.15.16:55278) 2024-09-19T09:27:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6e49d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 0x55ccf7a22c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6dffff640 1 -- 172.21.15.16:0/915299639 learned_addr learned my addr 172.21.15.16:0/915299639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6dd7fa640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd6cc0a3240 0x7fd6cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6df7fe640 1 --2- 172.21.15.16:0/915299639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6dffff640 1 -- 172.21.15.16:0/915299639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a31fc0 msgr2=0x55ccf7a343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6dffff640 1 --2- 172.21.15.16:0/915299639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6dffff640 1 -- 172.21.15.16:0/915299639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 msgr2=0x55ccf7a22c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6dffff640 1 --2- 172.21.15.16:0/915299639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 0x55ccf7a22c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.080+0000 7fd6dffff640 1 -- 172.21.15.16:0/915299639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccf793d130 con 0x55ccf7a231d0 2024-09-19T09:27:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.081+0000 7fd6dd7fa640 1 -- 172.21.15.16:0/3052119636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd6c80cfa60 con 0x7fd6e0102790 2024-09-19T09:27:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.081+0000 7fd6df7fe640 1 --2- 172.21.15.16:0/915299639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.081+0000 7fd6e49d9640 1 --2- 172.21.15.16:0/915299639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 0x55ccf7a22c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.081+0000 7fd6e49d9640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd6cc0a3240 0x7fd6cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.081+0000 7fd6dffff640 1 --2- 172.21.15.16:0/915299639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 0x55ccf7a31a80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0005ab0 tx=0x7fd6d0031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76447652b8586e8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.081+0000 7fd6d67fc640 1 -- 172.21.15.16:0/915299639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d003f070 con 0x55ccf7a231d0 2024-09-19T09:27:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.081+0000 7fd6d67fc640 1 -- 172.21.15.16:0/915299639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6d0031a20 con 0x55ccf7a231d0 2024-09-19T09:27:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.081+0000 7fd6d67fc640 1 -- 172.21.15.16:0/915299639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0035070 con 0x55ccf7a231d0 2024-09-19T09:27:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.082+0000 7fd6e8815740 1 -- 172.21.15.16:0/915299639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 msgr2=0x55ccf7a31a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.082+0000 7fd6e49d9640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd6cc0a3240 0x7fd6cc0a5700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80125f0 tx=0x7fd6d8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.082+0000 7fd6e8815740 1 --2- 172.21.15.16:0/915299639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 0x55ccf7a31a80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0005ab0 tx=0x7fd6d0031630 comp rx=0 tx=0).stop 2024-09-19T09:27:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.082+0000 7fd6e8815740 1 -- 172.21.15.16:0/915299639 shutdown_connections 2024-09-19T09:27:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.082+0000 7fd6e8815740 1 --2- 172.21.15.16:0/915299639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.082+0000 7fd6e8815740 1 --2- 172.21.15.16:0/915299639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 0x55ccf7a31a80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.082+0000 7fd6e8815740 1 --2- 172.21.15.16:0/915299639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 0x55ccf7a22c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.082+0000 7fd6e8815740 1 -- 172.21.15.16:0/915299639 >> 172.21.15.16:0/915299639 conn(0x55ccf7a1ef60 msgr2=0x55ccf7a1f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.082+0000 7fd6e8815740 1 -- 172.21.15.16:0/915299639 shutdown_connections 2024-09-19T09:27:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.082+0000 7fd6e8815740 1 -- 172.21.15.16:0/915299639 wait complete. 2024-09-19T09:27:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.082+0000 7fd6e8815740 1 -- start start 2024-09-19T09:27:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.082+0000 7fd6e8815740 1 Processor -- start 2024-09-19T09:27:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e8815740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 0x55ccf7a31840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e8815740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a231d0 0x55ccf7a2eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e8815740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a2f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e8815740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7a34b90 con 0x55ccf7a31fc0 2024-09-19T09:27:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e8815740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7943f20 con 0x55ccf7a228b0 2024-09-19T09:27:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e8815740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7a34d10 con 0x55ccf7a231d0 2024-09-19T09:27:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e8815740 10 client.0 _ll_register_callbacks cb 0x55ccf790d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e49d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a2f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e49d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a2f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55282/0 (socket says 172.21.15.16:55282) 2024-09-19T09:27:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e49d9640 1 -- 172.21.15.16:0/258518287 learned_addr learned my addr 172.21.15.16:0/258518287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6df7fe640 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a231d0 0x55ccf7a2eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6dffff640 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 0x55ccf7a31840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e49d9640 1 -- 172.21.15.16:0/258518287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a231d0 msgr2=0x55ccf7a2eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e49d9640 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a231d0 0x55ccf7a2eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e49d9640 1 -- 172.21.15.16:0/258518287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 msgr2=0x55ccf7a31840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e49d9640 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 0x55ccf7a31840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e49d9640 1 -- 172.21.15.16:0/258518287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccf76fa2b0 con 0x55ccf7a31fc0 2024-09-19T09:27:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6df7fe640 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a231d0 0x55ccf7a2eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e49d9640 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a2f0d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800fc00 tx=0x7fd6d8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78743c8a4ccf115 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6bf7fe640 1 -- 172.21.15.16:0/258518287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8010c50 con 0x55ccf7a31fc0 2024-09-19T09:27:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6bf7fe640 1 -- 172.21.15.16:0/258518287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd6d8010df0 con 0x55ccf7a31fc0 2024-09-19T09:27:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.083+0000 7fd6e8815740 1 -- 172.21.15.16:0/258518287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d0031a20 con 0x55ccf7a31fc0 2024-09-19T09:27:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.084+0000 7fd6bf7fe640 1 -- 172.21.15.16:0/258518287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6d8026470 con 0x55ccf7a31fc0 2024-09-19T09:27:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.084+0000 7fd6e8815740 10 client.15375 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.084+0000 7fd6e8815740 1 -- 172.21.15.16:0/258518287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccf7a40540 con 0x55ccf7a31fc0 2024-09-19T09:27:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.084+0000 7fd6bf7fe640 1 -- 172.21.15.16:0/258518287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6d0031a20 con 0x55ccf7a31fc0 2024-09-19T09:27:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.084+0000 7fd6bf7fe640 1 -- 172.21.15.16:0/258518287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd6d801c980 con 0x55ccf7a31fc0 2024-09-19T09:27:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.084+0000 7fd6e8815740 10 client.15375 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.084+0000 7fd6e8815740 10 client.15375 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-19T09:27:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.084+0000 7fd6e8815740 20 client.15375 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.084+0000 7fd6e8815740 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ccf7a42bc0 0x55ccf7a44030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.084+0000 7fd6e8815740 4 client.15375 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.084+0000 7fd6e8815740 1 -- 172.21.15.16:0/258518287 --> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccf7996600 con 0x55ccf7a42bc0 2024-09-19T09:27:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.084+0000 7fd6dffff640 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ccf7a42bc0 0x55ccf7a44030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.085+0000 7fd6dffff640 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ccf7a42bc0 0x55ccf7a44030 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0054ec9b8dd3833 server_cookie=6660c5df1d3c00b8 in_seq=0 out_seq=0 2024-09-19T09:27:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.085+0000 7fd6bf7fe640 10 client.15375 ms_handle_connect on v2:172.21.15.97:6832/108652274 2024-09-19T09:27:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.085+0000 7fd6bf7fe640 1 -- 172.21.15.16:0/258518287 <== mds.3 v2:172.21.15.97:6832/108652274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ccf7996600 con 0x55ccf7a42bc0 2024-09-19T09:27:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.085+0000 7fd6bf7fe640 10 client.15375 handle_command_reply: tid=0 2024-09-19T09:27:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.085+0000 7fd6e8815740 1 client.15375 shutdown 2024-09-19T09:27:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.086+0000 7fd6e8815740 1 -- 172.21.15.16:0/258518287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 msgr2=0x55ccf7a2f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.086+0000 7fd6e8815740 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a2f0d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800fc00 tx=0x7fd6d8004ab0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.086+0000 7fd6e8815740 1 -- 172.21.15.16:0/258518287 shutdown_connections 2024-09-19T09:27:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.086+0000 7fd6e8815740 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ccf7a42bc0 0x55ccf7a44030 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.086+0000 7fd6e8815740 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a231d0 0x55ccf7a2eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.086+0000 7fd6e8815740 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a2f0d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.086+0000 7fd6e8815740 1 --2- 172.21.15.16:0/258518287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 0x55ccf7a31840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.086+0000 7fd6e8815740 1 -- 172.21.15.16:0/258518287 >> 172.21.15.16:0/258518287 conn(0x55ccf7a1ef60 msgr2=0x55ccf7a24cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.086+0000 7fd6bf7fe640 0 client.15375 ms_handle_reset on v2:172.21.15.97:6832/108652274 2024-09-19T09:27:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.086+0000 7fd6e8815740 1 -- 172.21.15.16:0/258518287 shutdown_connections 2024-09-19T09:27:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.086+0000 7fd6e8815740 1 -- 172.21.15.16:0/258518287 wait complete. 2024-09-19T09:27:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.086+0000 7fd6e8815740 20 client.15375 trim_cache size 0 max 16384 2024-09-19T09:27:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e8815740 1 Processor -- start 2024-09-19T09:27:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e8815740 1 -- start start 2024-09-19T09:27:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e8815740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 0x55ccf7a42bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e8815740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a231d0 0x55ccf7a2e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e8815740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a2d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e8815740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7a42370 con 0x55ccf7a31fc0 2024-09-19T09:27:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e8815740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7a42a40 con 0x55ccf7a228b0 2024-09-19T09:27:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e8815740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7a43110 con 0x55ccf7a231d0 2024-09-19T09:27:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e49d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a2d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e49d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a2d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55286/0 (socket says 172.21.15.16:55286) 2024-09-19T09:27:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e49d9640 1 -- 172.21.15.16:0/1640498114 learned_addr learned my addr 172.21.15.16:0/1640498114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e49d9640 1 -- 172.21.15.16:0/1640498114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a231d0 msgr2=0x55ccf7a2e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e49d9640 1 --2- 172.21.15.16:0/1640498114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a231d0 0x55ccf7a2e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e49d9640 1 -- 172.21.15.16:0/1640498114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 msgr2=0x55ccf7a42bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e49d9640 1 --2- 172.21.15.16:0/1640498114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 0x55ccf7a42bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e49d9640 1 -- 172.21.15.16:0/1640498114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccf7a2f950 con 0x55ccf7a31fc0 2024-09-19T09:27:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.104+0000 7fd6e49d9640 1 --2- 172.21.15.16:0/1640498114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a2d460 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8000f80 tx=0x7fd6d80124c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f061d19b8e35fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.105+0000 7fd6d5ffb640 1 -- 172.21.15.16:0/1640498114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8035070 con 0x55ccf7a31fc0 2024-09-19T09:27:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.105+0000 7fd6d5ffb640 1 -- 172.21.15.16:0/1640498114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6d8021070 con 0x55ccf7a31fc0 2024-09-19T09:27:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.105+0000 7fd6d5ffb640 1 -- 172.21.15.16:0/1640498114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d801ca40 con 0x55ccf7a31fc0 2024-09-19T09:27:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.105+0000 7fd6e8815740 1 -- 172.21.15.16:0/1640498114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 msgr2=0x55ccf7a2d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.105+0000 7fd6e8815740 1 --2- 172.21.15.16:0/1640498114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a2d460 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8000f80 tx=0x7fd6d80124c0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.105+0000 7fd6e8815740 1 -- 172.21.15.16:0/1640498114 shutdown_connections 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.105+0000 7fd6e8815740 1 --2- 172.21.15.16:0/1640498114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a231d0 0x55ccf7a2e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.105+0000 7fd6e8815740 1 --2- 172.21.15.16:0/1640498114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a2d460 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.105+0000 7fd6e8815740 1 --2- 172.21.15.16:0/1640498114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a228b0 0x55ccf7a42bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.105+0000 7fd6e8815740 1 -- 172.21.15.16:0/1640498114 >> 172.21.15.16:0/1640498114 conn(0x55ccf7a1ef60 msgr2=0x55ccf7a2eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.105+0000 7fd6e8815740 1 -- 172.21.15.16:0/1640498114 shutdown_connections 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.105+0000 7fd6e8815740 1 -- 172.21.15.16:0/1640498114 wait complete. 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.106+0000 7fd6e8815740 1 -- start start 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.106+0000 7fd6e8815740 1 Processor -- start 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.106+0000 7fd6e8815740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a228b0 0x55ccf7a4c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.106+0000 7fd6e8815740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 0x55ccf7a4c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.106+0000 7fd6e8815740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a45720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.106+0000 7fd6e8815740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7a43eb0 con 0x55ccf7a231d0 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.106+0000 7fd6e8815740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7a437e0 con 0x55ccf7a31fc0 2024-09-19T09:27:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.106+0000 7fd6e8815740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7a44580 con 0x55ccf7a228b0 2024-09-19T09:27:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.106+0000 7fd6e8815740 10 client.0 _ll_register_callbacks cb 0x7fd6d8007a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.106+0000 7fd6df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 0x55ccf7a4c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.106+0000 7fd6df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 0x55ccf7a4c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55294/0 (socket says 172.21.15.16:55294) 2024-09-19T09:27:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.106+0000 7fd6df7fe640 1 -- 172.21.15.16:0/3602738414 learned_addr learned my addr 172.21.15.16:0/3602738414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6e49d9640 1 --2- 172.21.15.16:0/3602738414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a45720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6dffff640 1 --2- 172.21.15.16:0/3602738414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a228b0 0x55ccf7a4c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6df7fe640 1 -- 172.21.15.16:0/3602738414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a228b0 msgr2=0x55ccf7a4c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6df7fe640 1 --2- 172.21.15.16:0/3602738414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a228b0 0x55ccf7a4c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6df7fe640 1 -- 172.21.15.16:0/3602738414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a31fc0 msgr2=0x55ccf7a45720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6df7fe640 1 --2- 172.21.15.16:0/3602738414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a45720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6df7fe640 1 -- 172.21.15.16:0/3602738414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccf7a45db0 con 0x55ccf7a231d0 2024-09-19T09:27:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6df7fe640 1 --2- 172.21.15.16:0/3602738414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 0x55ccf7a4c900 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd6c809bd80 tx=0x7fd6c80a0430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700da738e0863ca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6bf7fe640 1 -- 172.21.15.16:0/3602738414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c809c080 con 0x55ccf7a231d0 2024-09-19T09:27:44.133 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:27:44.133 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:27:44.133 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-09-19T09:27:44.133 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24428, 2024-09-19T09:27:44.133 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:27:44.133 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:27:44.133 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:27:44.134 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 125.150673799, 2024-09-19T09:27:44.134 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-19T09:27:44.134 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:27:44.134 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T09:27:44.134 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 171.98884233300001 2024-09-19T09:27:44.134 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:27:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6bf7fe640 1 -- 172.21.15.16:0/3602738414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd6c8018d60 con 0x55ccf7a231d0 2024-09-19T09:27:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6e8815740 1 -- 172.21.15.16:0/3602738414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d8021070 con 0x55ccf7a231d0 2024-09-19T09:27:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6bf7fe640 1 -- 172.21.15.16:0/3602738414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6c80a4cf0 con 0x55ccf7a231d0 2024-09-19T09:27:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6e8815740 10 client.15381 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6e8815740 1 -- 172.21.15.16:0/3602738414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccf7a4a860 con 0x55ccf7a231d0 2024-09-19T09:27:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6bf7fe640 1 -- 172.21.15.16:0/3602738414 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6d8021070 con 0x55ccf7a231d0 2024-09-19T09:27:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.107+0000 7fd6bf7fe640 1 -- 172.21.15.16:0/3602738414 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd6c809b4d0 con 0x55ccf7a231d0 2024-09-19T09:27:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.108+0000 7fd6e8815740 10 client.15381 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.108+0000 7fd6e8815740 10 client.15381 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-19T09:27:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.108+0000 7fd6e8815740 20 client.15381 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.108+0000 7fd6e8815740 1 --2- 172.21.15.16:0/3602738414 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ccf7ab17d0 0x55ccf7ab04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.108+0000 7fd6e8815740 4 client.15381 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:27:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.108+0000 7fd6e8815740 1 -- 172.21.15.16:0/3602738414 --> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] -- command(tid 0: {"prefix": "status"}) -- 0x55ccf7990b10 con 0x55ccf7ab17d0 2024-09-19T09:27:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.108+0000 7fd6dffff640 1 --2- 172.21.15.16:0/3602738414 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ccf7ab17d0 0x55ccf7ab04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.108+0000 7fd6dffff640 1 --2- 172.21.15.16:0/3602738414 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ccf7ab17d0 0x55ccf7ab04e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ac27910482a7f694 server_cookie=9e893b248cf175dc in_seq=0 out_seq=0 2024-09-19T09:27:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.108+0000 7fd6bf7fe640 10 client.15381 ms_handle_connect on v2:172.21.15.97:6832/108652274 2024-09-19T09:27:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.108+0000 7fd6bf7fe640 1 -- 172.21.15.16:0/3602738414 <== mds.3 v2:172.21.15.97:6832/108652274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ccf7990b10 con 0x55ccf7ab17d0 2024-09-19T09:27:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.108+0000 7fd6bf7fe640 10 client.15381 handle_command_reply: tid=0 2024-09-19T09:27:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.108+0000 7fd6e8815740 1 client.15381 shutdown 2024-09-19T09:27:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e8815740 1 -- 172.21.15.16:0/3602738414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 msgr2=0x55ccf7a4c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e8815740 1 --2- 172.21.15.16:0/3602738414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 0x55ccf7a4c900 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd6c809bd80 tx=0x7fd6c80a0430 comp rx=0 tx=0).stop 2024-09-19T09:27:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e8815740 1 -- 172.21.15.16:0/3602738414 shutdown_connections 2024-09-19T09:27:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e8815740 1 --2- 172.21.15.16:0/3602738414 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ccf7ab17d0 0x55ccf7ab04e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e8815740 1 --2- 172.21.15.16:0/3602738414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccf7a31fc0 0x55ccf7a45720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e8815740 1 --2- 172.21.15.16:0/3602738414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccf7a231d0 0x55ccf7a4c900 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e8815740 1 --2- 172.21.15.16:0/3602738414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccf7a228b0 0x55ccf7a4c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e8815740 1 -- 172.21.15.16:0/3602738414 >> 172.21.15.16:0/3602738414 conn(0x55ccf7a1ef60 msgr2=0x55ccf7a29a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6bf7fe640 0 client.15381 ms_handle_reset on v2:172.21.15.97:6832/108652274 2024-09-19T09:27:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e8815740 1 -- 172.21.15.16:0/3602738414 shutdown_connections 2024-09-19T09:27:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e8815740 1 -- 172.21.15.16:0/3602738414 wait complete. 2024-09-19T09:27:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e8815740 20 client.15381 trim_cache size 0 max 16384 2024-09-19T09:27:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e643c640 1 -- 172.21.15.16:0/3052119636 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd6cc0a3240 msgr2=0x7fd6cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e643c640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd6cc0a3240 0x7fd6cc0a5700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80125f0 tx=0x7fd6d8004a40 comp rx=0 tx=0).stop 2024-09-19T09:27:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e643c640 1 -- 172.21.15.16:0/3052119636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e0102790 msgr2=0x7fd6e0198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e643c640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e0102790 0x7fd6e0198830 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c920 tx=0x7fd6c800cde0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e643c640 1 -- 172.21.15.16:0/3052119636 shutdown_connections 2024-09-19T09:27:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e643c640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd6cc0a3240 0x7fd6cc0a5700 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e643c640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6e0106f10 0x7fd6e019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e643c640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e0102790 0x7fd6e0198830 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e643c640 1 --2- 172.21.15.16:0/3052119636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e0101de0 0x7fd6e01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.109+0000 7fd6e643c640 1 -- 172.21.15.16:0/3052119636 >> 172.21.15.16:0/3052119636 conn(0x7fd6e00fd8c0 msgr2=0x7fd6e00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.110+0000 7fd6e643c640 1 -- 172.21.15.16:0/3052119636 shutdown_connections 2024-09-19T09:27:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:44.110+0000 7fd6e643c640 1 -- 172.21.15.16:0/3052119636 wait complete. 2024-09-19T09:27:44.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T09:27:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:45.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:44 smithi016 ceph-mon[30979]: pgmap v342: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-09-19T09:27:45.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:44 smithi142 ceph-mon[35032]: pgmap v342: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-09-19T09:27:45.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:44 smithi097 ceph-mon[34864]: pgmap v342: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-09-19T09:27:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.474+0000 7f3edf415640 1 -- 172.21.15.16:0/1755158384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed8103660 msgr2=0x7f3ed810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.474+0000 7f3edf415640 1 --2- 172.21.15.16:0/1755158384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed8103660 0x7f3ed810bb50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009900 tx=0x7f3ec802a480 comp rx=0 tx=0).stop 2024-09-19T09:27:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.474+0000 7f3edf415640 1 -- 172.21.15.16:0/1755158384 shutdown_connections 2024-09-19T09:27:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.474+0000 7f3edf415640 1 --2- 172.21.15.16:0/1755158384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed810c170 0x7f3ed810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.474+0000 7f3edf415640 1 --2- 172.21.15.16:0/1755158384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed8103660 0x7f3ed810bb50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.474+0000 7f3edf415640 1 --2- 172.21.15.16:0/1755158384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed8102d40 0x7f3ed8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.474+0000 7f3edf415640 1 -- 172.21.15.16:0/1755158384 >> 172.21.15.16:0/1755158384 conn(0x7f3ed80fc7a0 msgr2=0x7f3ed80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.474+0000 7f3edf415640 1 -- 172.21.15.16:0/1755158384 shutdown_connections 2024-09-19T09:27:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.474+0000 7f3edf415640 1 -- 172.21.15.16:0/1755158384 wait complete. 2024-09-19T09:27:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.475+0000 7f3edf415640 1 Processor -- start 2024-09-19T09:27:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.475+0000 7f3edf415640 1 -- start start 2024-09-19T09:27:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.475+0000 7f3edf415640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed8102d40 0x7f3ed81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.475+0000 7f3edf415640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed8103660 0x7f3ed8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.475+0000 7f3edf415640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed810c170 0x7f3ed819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.475+0000 7f3edf415640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed8111fb0 con 0x7f3ed8103660 2024-09-19T09:27:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.475+0000 7f3edf415640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed8111e30 con 0x7f3ed8102d40 2024-09-19T09:27:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.475+0000 7f3edf415640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed8112130 con 0x7f3ed810c170 2024-09-19T09:27:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.475+0000 7f3edc9b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed8103660 0x7f3ed8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.475+0000 7f3edc9b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed8103660 0x7f3ed8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41924/0 (socket says 172.21.15.16:41924) 2024-09-19T09:27:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.475+0000 7f3edc9b0640 1 -- 172.21.15.16:0/1577161831 learned_addr learned my addr 172.21.15.16:0/1577161831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.475+0000 7f3edd1b1640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed8102d40 0x7f3ed81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.476+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed810c170 0x7f3ed819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.476+0000 7f3edc9b0640 1 -- 172.21.15.16:0/1577161831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed810c170 msgr2=0x7f3ed819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.476+0000 7f3edc9b0640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed810c170 0x7f3ed819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.476+0000 7f3edc9b0640 1 -- 172.21.15.16:0/1577161831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed8102d40 msgr2=0x7f3ed81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.476+0000 7f3edc9b0640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed8102d40 0x7f3ed81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.476+0000 7f3edc9b0640 1 -- 172.21.15.16:0/1577161831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed819d3a0 con 0x7f3ed8103660 2024-09-19T09:27:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.476+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed810c170 0x7f3ed819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.476+0000 7f3edc9b0640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed8103660 0x7f3ed8198930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8005ab0 tx=0x7f3ec80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.476+0000 7f3ece7fc640 1 -- 172.21.15.16:0/1577161831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec803f030 con 0x7f3ed8103660 2024-09-19T09:27:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.476+0000 7f3ece7fc640 1 -- 172.21.15.16:0/1577161831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec8003ce0 con 0x7f3ed8103660 2024-09-19T09:27:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.476+0000 7f3ece7fc640 1 -- 172.21.15.16:0/1577161831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8035070 con 0x7f3ed8103660 2024-09-19T09:27:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.476+0000 7f3edf415640 1 -- 172.21.15.16:0/1577161831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed819d630 con 0x7f3ed8103660 2024-09-19T09:27:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.476+0000 7f3edf415640 1 -- 172.21.15.16:0/1577161831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed819dac0 con 0x7f3ed8103660 2024-09-19T09:27:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.478+0000 7f3ece7fc640 1 -- 172.21.15.16:0/1577161831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3ec8042030 con 0x7f3ed8103660 2024-09-19T09:27:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3ee17ee740 1 Processor -- start 2024-09-19T09:27:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3ee17ee740 1 -- start start 2024-09-19T09:27:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3ee17ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db68b0 0x562671db6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3ee17ee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db71d0 0x562671dc5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3ee17ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671dc5fc0 0x562671dc83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3ee17ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562671d1d090 con 0x562671db68b0 2024-09-19T09:27:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3ee17ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562671cc4a70 con 0x562671dc5fc0 2024-09-19T09:27:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3ee17ee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562671d19050 con 0x562671db71d0 2024-09-19T09:27:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3edd9b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db68b0 0x562671db6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3edd9b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db68b0 0x562671db6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41940/0 (socket says 172.21.15.16:41940) 2024-09-19T09:27:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3edd9b2640 1 -- 172.21.15.16:0/2621037209 learned_addr learned my addr 172.21.15.16:0/2621037209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3edc9b0640 1 --2- 172.21.15.16:0/2621037209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671dc5fc0 0x562671dc83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3ece7fc640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3ec40a32c0 0x7f3ec40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.479+0000 7f3edd1b1640 1 --2- 172.21.15.16:0/2621037209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db71d0 0x562671dc5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3edd9b2640 1 -- 172.21.15.16:0/2621037209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db71d0 msgr2=0x562671dc5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/2621037209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db71d0 0x562671dc5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3edd9b2640 1 -- 172.21.15.16:0/2621037209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671dc5fc0 msgr2=0x562671dc83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/2621037209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671dc5fc0 0x562671dc83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3edd9b2640 1 -- 172.21.15.16:0/2621037209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562671cd1130 con 0x562671db68b0 2024-09-19T09:27:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3edc9b0640 1 --2- 172.21.15.16:0/2621037209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671dc5fc0 0x562671dc83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3edd1b1640 1 --2- 172.21.15.16:0/2621037209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db71d0 0x562671dc5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3ece7fc640 1 -- 172.21.15.16:0/1577161831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3ec80f00d0 con 0x7f3ed8103660 2024-09-19T09:27:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3ec40a32c0 0x7f3ec40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/2621037209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db68b0 0x562671db6c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000c370 tx=0x7f3ed000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4618472c2f716537 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3ec37fe640 1 -- 172.21.15.16:0/2621037209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0016070 con 0x562671db68b0 2024-09-19T09:27:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3ec37fe640 1 -- 172.21.15.16:0/2621037209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed0011d00 con 0x562671db68b0 2024-09-19T09:27:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3ec37fe640 1 -- 172.21.15.16:0/2621037209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed000fcb0 con 0x562671db68b0 2024-09-19T09:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.480+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3ec40a32c0 0x7f3ec40a5780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3ed001d000 tx=0x7f3ed001e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.481+0000 7f3ee17ee740 1 -- 172.21.15.16:0/2621037209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db68b0 msgr2=0x562671db6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.481+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/2621037209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db68b0 0x562671db6c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000c370 tx=0x7f3ed000c830 comp rx=0 tx=0).stop 2024-09-19T09:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.481+0000 7f3ee17ee740 1 -- 172.21.15.16:0/2621037209 shutdown_connections 2024-09-19T09:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.481+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/2621037209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db71d0 0x562671dc5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.481+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/2621037209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671dc5fc0 0x562671dc83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.481+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/2621037209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db68b0 0x562671db6c90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.481+0000 7f3ee17ee740 1 -- 172.21.15.16:0/2621037209 >> 172.21.15.16:0/2621037209 conn(0x562671db2f60 msgr2=0x562671db3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.481+0000 7f3ee17ee740 1 -- 172.21.15.16:0/2621037209 shutdown_connections 2024-09-19T09:27:45.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.481+0000 7f3ee17ee740 1 -- 172.21.15.16:0/2621037209 wait complete. 2024-09-19T09:27:45.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.481+0000 7f3ee17ee740 1 -- start start 2024-09-19T09:27:45.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.481+0000 7f3ee17ee740 1 Processor -- start 2024-09-19T09:27:45.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3ee17ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db68b0 0x562671dc5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3ee17ee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db71d0 0x562671dc2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3ee17ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 0x562671dc30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3ee17ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562671dc8b90 con 0x562671dc5fc0 2024-09-19T09:27:45.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3ee17ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562671cd7f20 con 0x562671db68b0 2024-09-19T09:27:45.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3ee17ee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562671dc8d10 con 0x562671db71d0 2024-09-19T09:27:45.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3ee17ee740 10 client.0 _ll_register_callbacks cb 0x562671ca1030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:45.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3edd9b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 0x562671dc30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3edd9b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 0x562671dc30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41944/0 (socket says 172.21.15.16:41944) 2024-09-19T09:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3edd9b2640 1 -- 172.21.15.16:0/1078849375 learned_addr learned my addr 172.21.15.16:0/1078849375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3edd1b1640 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db68b0 0x562671dc5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3edc9b0640 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db71d0 0x562671dc2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3edd9b2640 1 -- 172.21.15.16:0/1078849375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db71d0 msgr2=0x562671dc2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db71d0 0x562671dc2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3edd9b2640 1 -- 172.21.15.16:0/1078849375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db68b0 msgr2=0x562671dc5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db68b0 0x562671dc5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3edd9b2640 1 -- 172.21.15.16:0/1078849375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562671a8e2b0 con 0x562671dc5fc0 2024-09-19T09:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.482+0000 7f3edd1b1640 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db68b0 0x562671dc5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 0x562671dc30c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000f870 tx=0x7f3ed00077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f201d86ce64c20e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ec0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ec0ff9640 1 -- 172.21.15.16:0/1078849375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0016070 con 0x562671dc5fc0 2024-09-19T09:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ee17ee740 1 -- 172.21.15.16:0/1078849375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ed0011d00 con 0x562671dc5fc0 2024-09-19T09:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ec0ff9640 1 -- 172.21.15.16:0/1078849375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3ed0011a30 con 0x562671dc5fc0 2024-09-19T09:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ec0ff9640 1 -- 172.21.15.16:0/1078849375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3ed0010600 con 0x562671dc5fc0 2024-09-19T09:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ee17ee740 10 client.15393 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ee17ee740 1 -- 172.21.15.16:0/1078849375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562671dd44c0 con 0x562671dc5fc0 2024-09-19T09:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ec0ff9640 1 -- 172.21.15.16:0/1078849375 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed0011d00 con 0x562671dc5fc0 2024-09-19T09:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ec0ff9640 1 -- 172.21.15.16:0/1078849375 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3ed002e020 con 0x562671dc5fc0 2024-09-19T09:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ee17ee740 10 client.15393 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ee17ee740 10 client.15393 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ee17ee740 20 client.15393 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x562671dd6840 0x562671dd7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ee17ee740 4 client.15393 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.483+0000 7f3ee17ee740 1 -- 172.21.15.16:0/1078849375 --> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562671d2a600 con 0x562671dd6840 2024-09-19T09:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.484+0000 7f3edd1b1640 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x562671dd6840 0x562671dd7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.484+0000 7f3edd1b1640 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x562671dd6840 0x562671dd7cb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dcb41d29f7005f04 server_cookie=3eac155c2cad45d3 in_seq=0 out_seq=0 2024-09-19T09:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.484+0000 7f3ec0ff9640 10 client.15393 ms_handle_connect on v2:172.21.15.97:6836/2018284957 2024-09-19T09:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ec0ff9640 1 -- 172.21.15.16:0/1078849375 <== mds.2 v2:172.21.15.97:6836/2018284957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562671d2a600 con 0x562671dd6840 2024-09-19T09:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ec0ff9640 10 client.15393 handle_command_reply: tid=0 2024-09-19T09:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ee17ee740 1 client.15393 shutdown 2024-09-19T09:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ee17ee740 1 -- 172.21.15.16:0/1078849375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 msgr2=0x562671dc30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 0x562671dc30c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000f870 tx=0x7f3ed00077f0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ee17ee740 1 -- 172.21.15.16:0/1078849375 shutdown_connections 2024-09-19T09:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x562671dd6840 0x562671dd7cb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db71d0 0x562671dc2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ec0ff9640 0 client.15393 ms_handle_reset on v2:172.21.15.97:6836/2018284957 2024-09-19T09:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 0x562671dc30c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/1078849375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db68b0 0x562671dc5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ee17ee740 1 -- 172.21.15.16:0/1078849375 >> 172.21.15.16:0/1078849375 conn(0x562671db2f60 msgr2=0x562671db8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ee17ee740 1 -- 172.21.15.16:0/1078849375 shutdown_connections 2024-09-19T09:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ee17ee740 1 -- 172.21.15.16:0/1078849375 wait complete. 2024-09-19T09:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.485+0000 7f3ee17ee740 20 client.15393 trim_cache size 0 max 16384 2024-09-19T09:27:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.503+0000 7f3ee17ee740 1 Processor -- start 2024-09-19T09:27:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.503+0000 7f3ee17ee740 1 -- start start 2024-09-19T09:27:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3ee17ee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db68b0 0x562671dd7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3ee17ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db71d0 0x562671dbfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3ee17ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 0x562671dc0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3ee17ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562671dd5ff0 con 0x562671dc5fc0 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3ee17ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562671dd66c0 con 0x562671db71d0 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3ee17ee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562671dd6d90 con 0x562671db68b0 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3edd9b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 0x562671dc0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3edd9b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 0x562671dc0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41954/0 (socket says 172.21.15.16:41954) 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3edd9b2640 1 -- 172.21.15.16:0/183770014 learned_addr learned my addr 172.21.15.16:0/183770014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3edd1b1640 1 --2- 172.21.15.16:0/183770014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db68b0 0x562671dd7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3edc9b0640 1 --2- 172.21.15.16:0/183770014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db71d0 0x562671dbfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3edd9b2640 1 -- 172.21.15.16:0/183770014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db68b0 msgr2=0x562671dd7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/183770014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db68b0 0x562671dd7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3edd9b2640 1 -- 172.21.15.16:0/183770014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db71d0 msgr2=0x562671dbfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/183770014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db71d0 0x562671dbfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3edd9b2640 1 -- 172.21.15.16:0/183770014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562671dd7240 con 0x562671dc5fc0 2024-09-19T09:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3edc9b0640 1 --2- 172.21.15.16:0/183770014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db71d0 0x562671dbfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/183770014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 0x562671dc0340 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0030530 tx=0x7f3ed0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3818b748dd2a8ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3ec2ffd640 1 -- 172.21.15.16:0/183770014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0016030 con 0x562671dc5fc0 2024-09-19T09:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3ec2ffd640 1 -- 172.21.15.16:0/183770014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed0011d00 con 0x562671dc5fc0 2024-09-19T09:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.504+0000 7f3ec2ffd640 1 -- 172.21.15.16:0/183770014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0025470 con 0x562671dc5fc0 2024-09-19T09:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.505+0000 7f3ee17ee740 1 -- 172.21.15.16:0/183770014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 msgr2=0x562671dc0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.505+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/183770014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 0x562671dc0340 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0030530 tx=0x7f3ed0004b50 comp rx=0 tx=0).stop 2024-09-19T09:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.505+0000 7f3ee17ee740 1 -- 172.21.15.16:0/183770014 shutdown_connections 2024-09-19T09:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.505+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/183770014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671dc5fc0 0x562671dc0340 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.505+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/183770014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db71d0 0x562671dbfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.505+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/183770014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671db68b0 0x562671dd7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.505+0000 7f3ee17ee740 1 -- 172.21.15.16:0/183770014 >> 172.21.15.16:0/183770014 conn(0x562671db2f60 msgr2=0x562671dbe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.505+0000 7f3ee17ee740 1 -- 172.21.15.16:0/183770014 shutdown_connections 2024-09-19T09:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.505+0000 7f3ee17ee740 1 -- 172.21.15.16:0/183770014 wait complete. 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3ee17ee740 1 -- start start 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3ee17ee740 1 Processor -- start 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3ee17ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db68b0 0x562671de0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3ee17ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db71d0 0x562671de0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3ee17ee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671dc5fc0 0x562671dbf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3ee17ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562671dd7b30 con 0x562671db71d0 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3ee17ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562671dd7460 con 0x562671db68b0 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3ee17ee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562671dd8200 con 0x562671dc5fc0 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3ee17ee740 10 client.0 _ll_register_callbacks cb 0x7f3ed0007ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3edc9b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db71d0 0x562671de0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3edc9b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db71d0 0x562671de0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41966/0 (socket says 172.21.15.16:41966) 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3edc9b0640 1 -- 172.21.15.16:0/311832721 learned_addr learned my addr 172.21.15.16:0/311832721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3edd1b1640 1 --2- 172.21.15.16:0/311832721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db68b0 0x562671de0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.506+0000 7f3edd9b2640 1 --2- 172.21.15.16:0/311832721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671dc5fc0 0x562671dbf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3edc9b0640 1 -- 172.21.15.16:0/311832721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671dc5fc0 msgr2=0x562671dbf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3edc9b0640 1 --2- 172.21.15.16:0/311832721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671dc5fc0 0x562671dbf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3edc9b0640 1 -- 172.21.15.16:0/311832721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db68b0 msgr2=0x562671de0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3edc9b0640 1 --2- 172.21.15.16:0/311832721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db68b0 0x562671de0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3edc9b0640 1 -- 172.21.15.16:0/311832721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562671dc01c0 con 0x562671db71d0 2024-09-19T09:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3edc9b0640 1 --2- 172.21.15.16:0/311832721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db71d0 0x562671de0860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8006010 tx=0x7f3ec803b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb6a882f90b85dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3ec0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3ec0ff9640 1 -- 172.21.15.16:0/311832721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8049070 con 0x562671db71d0 2024-09-19T09:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3ec0ff9640 1 -- 172.21.15.16:0/311832721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3ec8003800 con 0x562671db71d0 2024-09-19T09:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3ee17ee740 1 -- 172.21.15.16:0/311832721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ed0011d00 con 0x562671db71d0 2024-09-19T09:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3ec0ff9640 1 -- 172.21.15.16:0/311832721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3ec80bcdf0 con 0x562671db71d0 2024-09-19T09:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3ee17ee740 10 client.15399 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3ee17ee740 1 -- 172.21.15.16:0/311832721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562671ddda20 con 0x562671db71d0 2024-09-19T09:27:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3ec0ff9640 1 -- 172.21.15.16:0/311832721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed0011d00 con 0x562671db71d0 2024-09-19T09:27:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.507+0000 7f3ec0ff9640 1 -- 172.21.15.16:0/311832721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3ec803bc90 con 0x562671db71d0 2024-09-19T09:27:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.508+0000 7f3ee17ee740 10 client.15399 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.508+0000 7f3ee17ee740 10 client.15399 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T09:27:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.508+0000 7f3ee17ee740 20 client.15399 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.508+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/311832721 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x562671dd6840 0x562671e444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.508+0000 7f3ee17ee740 4 client.15399 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:27:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.508+0000 7f3ee17ee740 1 -- 172.21.15.16:0/311832721 --> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] -- command(tid 0: {"prefix": "status"}) -- 0x562671d24b10 con 0x562671dd6840 2024-09-19T09:27:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.508+0000 7f3edd1b1640 1 --2- 172.21.15.16:0/311832721 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x562671dd6840 0x562671e444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.508+0000 7f3edd1b1640 1 --2- 172.21.15.16:0/311832721 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x562671dd6840 0x562671e444d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90fe0ac2e8f345a3 server_cookie=24f1f30d7d50d399 in_seq=0 out_seq=0 2024-09-19T09:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.508+0000 7f3ec0ff9640 10 client.15399 ms_handle_connect on v2:172.21.15.97:6836/2018284957 2024-09-19T09:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.508+0000 7f3ec0ff9640 1 -- 172.21.15.16:0/311832721 <== mds.2 v2:172.21.15.97:6836/2018284957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562671d24b10 con 0x562671dd6840 2024-09-19T09:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.508+0000 7f3ec0ff9640 10 client.15399 handle_command_reply: tid=0 2024-09-19T09:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.508+0000 7f3ee17ee740 1 client.15399 shutdown 2024-09-19T09:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3ee17ee740 1 -- 172.21.15.16:0/311832721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db71d0 msgr2=0x562671de0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/311832721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db71d0 0x562671de0860 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8006010 tx=0x7f3ec803b720 comp rx=0 tx=0).stop 2024-09-19T09:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3ee17ee740 1 -- 172.21.15.16:0/311832721 shutdown_connections 2024-09-19T09:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/311832721 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x562671dd6840 0x562671e444d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/311832721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562671dc5fc0 0x562671dbf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/311832721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562671db71d0 0x562671de0860 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3ee17ee740 1 --2- 172.21.15.16:0/311832721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562671db68b0 0x562671de0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3ee17ee740 1 -- 172.21.15.16:0/311832721 >> 172.21.15.16:0/311832721 conn(0x562671db2f60 msgr2=0x562671db83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3ec0ff9640 0 client.15399 ms_handle_reset on v2:172.21.15.97:6836/2018284957 2024-09-19T09:27:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3ee17ee740 1 -- 172.21.15.16:0/311832721 shutdown_connections 2024-09-19T09:27:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3ee17ee740 1 -- 172.21.15.16:0/311832721 wait complete. 2024-09-19T09:27:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3ee17ee740 20 client.15399 trim_cache size 0 max 16384 2024-09-19T09:27:45.516 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:27:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:27:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-19T09:27:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24470, 2024-09-19T09:27:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:27:45.516 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:27:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:27:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 127.18294914800001, 2024-09-19T09:27:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-19T09:27:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:27:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T09:27:45.517 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 157.03231900200001 2024-09-19T09:27:45.517 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:27:45.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3edf415640 1 -- 172.21.15.16:0/1577161831 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3ec40a32c0 msgr2=0x7f3ec40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:45.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3edf415640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3ec40a32c0 0x7f3ec40a5780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3ed001d000 tx=0x7f3ed001e000 comp rx=0 tx=0).stop 2024-09-19T09:27:45.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3edf415640 1 -- 172.21.15.16:0/1577161831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed8103660 msgr2=0x7f3ed8198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:45.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3edf415640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed8103660 0x7f3ed8198930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8005ab0 tx=0x7f3ec80042f0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3edf415640 1 -- 172.21.15.16:0/1577161831 shutdown_connections 2024-09-19T09:27:45.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3edf415640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3ec40a32c0 0x7f3ec40a5780 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3edf415640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed810c170 0x7f3ed819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3edf415640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed8103660 0x7f3ed8198930 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3edf415640 1 --2- 172.21.15.16:0/1577161831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ed8102d40 0x7f3ed81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:45.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.509+0000 7f3edf415640 1 -- 172.21.15.16:0/1577161831 >> 172.21.15.16:0/1577161831 conn(0x7f3ed80fc7a0 msgr2=0x7f3ed80fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:45.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.510+0000 7f3edf415640 1 -- 172.21.15.16:0/1577161831 shutdown_connections 2024-09-19T09:27:45.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:45.510+0000 7f3edf415640 1 -- 172.21.15.16:0/1577161831 wait complete. 2024-09-19T09:27:46.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T09:27:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:46.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.974+0000 7f84cff09640 1 -- 172.21.15.16:0/1849038192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102ef0 msgr2=0x7f84c810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.974+0000 7f84cff09640 1 --2- 172.21.15.16:0/1849038192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102ef0 0x7f84c810da40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009ca0 tx=0x7f84c002a950 comp rx=0 tx=0).stop 2024-09-19T09:27:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.974+0000 7f84cff09640 1 -- 172.21.15.16:0/1849038192 shutdown_connections 2024-09-19T09:27:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.974+0000 7f84cff09640 1 --2- 172.21.15.16:0/1849038192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84c8102570 0x7f84c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.974+0000 7f84cff09640 1 --2- 172.21.15.16:0/1849038192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102ef0 0x7f84c810da40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.974+0000 7f84cff09640 1 --2- 172.21.15.16:0/1849038192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c8100e20 0x7f84c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.974+0000 7f84cff09640 1 -- 172.21.15.16:0/1849038192 >> 172.21.15.16:0/1849038192 conn(0x7f84c80fc7d0 msgr2=0x7f84c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.974+0000 7f84cff09640 1 -- 172.21.15.16:0/1849038192 shutdown_connections 2024-09-19T09:27:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.974+0000 7f84cff09640 1 -- 172.21.15.16:0/1849038192 wait complete. 2024-09-19T09:27:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.974+0000 7f84cff09640 1 Processor -- start 2024-09-19T09:27:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cff09640 1 -- start start 2024-09-19T09:27:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cff09640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84c8100e20 0x7f84c80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cff09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102570 0x7f84c80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cff09640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c8102ef0 0x7f84c80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cff09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84c8110f80 con 0x7f84c8102570 2024-09-19T09:27:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cff09640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84c8110e00 con 0x7f84c8102ef0 2024-09-19T09:27:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cff09640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84c8111100 con 0x7f84c8100e20 2024-09-19T09:27:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cd4a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102570 0x7f84c80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cdca5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84c8100e20 0x7f84c80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cd4a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102570 0x7f84c80fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41980/0 (socket says 172.21.15.16:41980) 2024-09-19T09:27:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cd4a4640 1 -- 172.21.15.16:0/560554411 learned_addr learned my addr 172.21.15.16:0/560554411 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84ce4a6640 1 --2- 172.21.15.16:0/560554411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c8102ef0 0x7f84c80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cd4a4640 1 -- 172.21.15.16:0/560554411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84c8100e20 msgr2=0x7f84c80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cd4a4640 1 --2- 172.21.15.16:0/560554411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84c8100e20 0x7f84c80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cd4a4640 1 -- 172.21.15.16:0/560554411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c8102ef0 msgr2=0x7f84c80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:46.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cd4a4640 1 --2- 172.21.15.16:0/560554411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c8102ef0 0x7f84c80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.975+0000 7f84cd4a4640 1 -- 172.21.15.16:0/560554411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c81a67f0 con 0x7f84c8102570 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.976+0000 7f84cd4a4640 1 --2- 172.21.15.16:0/560554411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102570 0x7f84c80fd1c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c920 tx=0x7f84b800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.976+0000 7f84beffd640 1 -- 172.21.15.16:0/560554411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8007bb0 con 0x7f84c8102570 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.976+0000 7f84beffd640 1 -- 172.21.15.16:0/560554411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b8007d50 con 0x7f84c8102570 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.976+0000 7f84cff09640 1 -- 172.21.15.16:0/560554411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c81a6ae0 con 0x7f84c8102570 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.976+0000 7f84beffd640 1 -- 172.21.15.16:0/560554411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8018780 con 0x7f84c8102570 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.976+0000 7f84cff09640 1 -- 172.21.15.16:0/560554411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c81a7020 con 0x7f84c8102570 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.978+0000 7f84beffd640 1 -- 172.21.15.16:0/560554411 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f84b800f450 con 0x7f84c8102570 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.978+0000 7f84d22e2740 1 Processor -- start 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84d22e2740 1 -- start start 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84d22e2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206d9d0 0x56224206ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84d22e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 0x562242078670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84d22e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562242078bb0 0x56224207afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84d22e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562241fd4090 con 0x56224206e2f0 2024-09-19T09:27:46.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84d22e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562241f7ba70 con 0x562242078bb0 2024-09-19T09:27:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84d22e2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562241fd0050 con 0x56224206d9d0 2024-09-19T09:27:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84cdca5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 0x562242078670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84cdca5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 0x562242078670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41992/0 (socket says 172.21.15.16:41992) 2024-09-19T09:27:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84cdca5640 1 -- 172.21.15.16:0/3811062028 learned_addr learned my addr 172.21.15.16:0/3811062028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84beffd640 1 --2- 172.21.15.16:0/560554411 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f84b40a3240 0x7f84b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84cd4a4640 1 --2- 172.21.15.16:0/3811062028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562242078bb0 0x56224207afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84ce4a6640 1 --2- 172.21.15.16:0/3811062028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206d9d0 0x56224206ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84ce4a6640 1 --2- 172.21.15.16:0/560554411 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f84b40a3240 0x7f84b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84cdca5640 1 -- 172.21.15.16:0/3811062028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206d9d0 msgr2=0x56224206ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84cdca5640 1 --2- 172.21.15.16:0/3811062028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206d9d0 0x56224206ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84cdca5640 1 -- 172.21.15.16:0/3811062028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562242078bb0 msgr2=0x56224207afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84cdca5640 1 --2- 172.21.15.16:0/3811062028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562242078bb0 0x56224207afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84cdca5640 1 -- 172.21.15.16:0/3811062028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562241f88130 con 0x56224206e2f0 2024-09-19T09:27:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.979+0000 7f84cd4a4640 1 --2- 172.21.15.16:0/3811062028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562242078bb0 0x56224207afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.980+0000 7f84beffd640 1 -- 172.21.15.16:0/560554411 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f84b80cf950 con 0x7f84c8102570 2024-09-19T09:27:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.980+0000 7f84cdca5640 1 --2- 172.21.15.16:0/3811062028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 0x562242078670 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f84c400ad00 tx=0x7f84c400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782694ecf95e16ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.980+0000 7f84b3fff640 1 -- 172.21.15.16:0/3811062028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c401d070 con 0x56224206e2f0 2024-09-19T09:27:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.980+0000 7f84ce4a6640 1 --2- 172.21.15.16:0/560554411 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f84b40a3240 0x7f84b40a5700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f84c0005fe0 tx=0x7f84c0005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.980+0000 7f84b3fff640 1 -- 172.21.15.16:0/3811062028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c4018b20 con 0x56224206e2f0 2024-09-19T09:27:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.980+0000 7f84b3fff640 1 -- 172.21.15.16:0/3811062028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4010bc0 con 0x56224206e2f0 2024-09-19T09:27:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.981+0000 7f84d22e2740 1 -- 172.21.15.16:0/3811062028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 msgr2=0x562242078670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.981+0000 7f84d22e2740 1 --2- 172.21.15.16:0/3811062028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 0x562242078670 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f84c400ad00 tx=0x7f84c400f790 comp rx=0 tx=0).stop 2024-09-19T09:27:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.981+0000 7f84d22e2740 1 -- 172.21.15.16:0/3811062028 shutdown_connections 2024-09-19T09:27:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.981+0000 7f84d22e2740 1 --2- 172.21.15.16:0/3811062028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562242078bb0 0x56224207afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.981+0000 7f84d22e2740 1 --2- 172.21.15.16:0/3811062028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 0x562242078670 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.981+0000 7f84d22e2740 1 --2- 172.21.15.16:0/3811062028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206d9d0 0x56224206ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.981+0000 7f84d22e2740 1 -- 172.21.15.16:0/3811062028 >> 172.21.15.16:0/3811062028 conn(0x562242069f60 msgr2=0x56224206a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.981+0000 7f84d22e2740 1 -- 172.21.15.16:0/3811062028 shutdown_connections 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.981+0000 7f84d22e2740 1 -- 172.21.15.16:0/3811062028 wait complete. 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.981+0000 7f84d22e2740 1 -- start start 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.981+0000 7f84d22e2740 1 Processor -- start 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84d22e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206d9d0 0x562242078420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84d22e2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206e2f0 0x562242075770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84d22e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562242078bb0 0x562242075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84d22e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56224207b780 con 0x56224206d9d0 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84d22e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562241f8ef20 con 0x562242078bb0 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84d22e2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56224207b900 con 0x56224206e2f0 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84d22e2740 10 client.0 _ll_register_callbacks cb 0x562241f58030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84cdca5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206d9d0 0x562242078420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84cdca5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206d9d0 0x562242078420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42000/0 (socket says 172.21.15.16:42000) 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84cdca5640 1 -- 172.21.15.16:0/2803283877 learned_addr learned my addr 172.21.15.16:0/2803283877 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84ce4a6640 1 --2- 172.21.15.16:0/2803283877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562242078bb0 0x562242075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84cd4a4640 1 --2- 172.21.15.16:0/2803283877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206e2f0 0x562242075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84cdca5640 1 -- 172.21.15.16:0/2803283877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206e2f0 msgr2=0x562242075770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84cdca5640 1 --2- 172.21.15.16:0/2803283877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206e2f0 0x562242075770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84cdca5640 1 -- 172.21.15.16:0/2803283877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562242078bb0 msgr2=0x562242075cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84cdca5640 1 --2- 172.21.15.16:0/2803283877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562242078bb0 0x562242075cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84cdca5640 1 -- 172.21.15.16:0/2803283877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562241d452b0 con 0x56224206d9d0 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84cdca5640 1 --2- 172.21.15.16:0/2803283877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206d9d0 0x562242078420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f84c400a550 tx=0x7f84c40186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52529b979180db55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.982+0000 7f84b17fa640 1 -- 172.21.15.16:0/2803283877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c401d040 con 0x56224206d9d0 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.983+0000 7f84b17fa640 1 -- 172.21.15.16:0/2803283877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f84c40108e0 con 0x56224206d9d0 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.983+0000 7f84d22e2740 1 -- 172.21.15.16:0/2803283877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84c4018b20 con 0x56224206d9d0 2024-09-19T09:27:46.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.983+0000 7f84b17fa640 1 -- 172.21.15.16:0/2803283877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84c4010ae0 con 0x56224206d9d0 2024-09-19T09:27:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.983+0000 7f84d22e2740 10 client.15411 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.983+0000 7f84d22e2740 1 -- 172.21.15.16:0/2803283877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562242087070 con 0x56224206d9d0 2024-09-19T09:27:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.983+0000 7f84b17fa640 1 -- 172.21.15.16:0/2803283877 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c4018b20 con 0x56224206d9d0 2024-09-19T09:27:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.983+0000 7f84b17fa640 1 -- 172.21.15.16:0/2803283877 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84c4020030 con 0x56224206d9d0 2024-09-19T09:27:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.983+0000 7f84d22e2740 10 client.15411 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.983+0000 7f84d22e2740 10 client.15411 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:27:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.983+0000 7f84d22e2740 20 client.15411 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.983+0000 7f84d22e2740 1 --2- 172.21.15.16:0/2803283877 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x5622420892f0 0x56224208a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.983+0000 7f84d22e2740 4 client.15411 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.983+0000 7f84d22e2740 1 -- 172.21.15.16:0/2803283877 --> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562241fe1600 con 0x5622420892f0 2024-09-19T09:27:46.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.984+0000 7f84cd4a4640 1 --2- 172.21.15.16:0/2803283877 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x5622420892f0 0x56224208a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.984+0000 7f84cd4a4640 1 --2- 172.21.15.16:0/2803283877 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x5622420892f0 0x56224208a760 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3fe0b0e8717ae187 server_cookie=708c400dcd5369bd in_seq=0 out_seq=0 2024-09-19T09:27:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.984+0000 7f84b17fa640 10 client.15411 ms_handle_connect on v2:172.21.15.142:6838/3822134597 2024-09-19T09:27:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84b17fa640 1 -- 172.21.15.16:0/2803283877 <== mds.1 v2:172.21.15.142:6838/3822134597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562241fe1600 con 0x5622420892f0 2024-09-19T09:27:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84b17fa640 10 client.15411 handle_command_reply: tid=0 2024-09-19T09:27:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84d22e2740 1 client.15411 shutdown 2024-09-19T09:27:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84d22e2740 1 -- 172.21.15.16:0/2803283877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206d9d0 msgr2=0x562242078420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84d22e2740 1 --2- 172.21.15.16:0/2803283877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206d9d0 0x562242078420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f84c400a550 tx=0x7f84c40186f0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84d22e2740 1 -- 172.21.15.16:0/2803283877 shutdown_connections 2024-09-19T09:27:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84d22e2740 1 --2- 172.21.15.16:0/2803283877 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x5622420892f0 0x56224208a760 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84d22e2740 1 --2- 172.21.15.16:0/2803283877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206e2f0 0x562242075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84d22e2740 1 --2- 172.21.15.16:0/2803283877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562242078bb0 0x562242075cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84b17fa640 0 client.15411 ms_handle_reset on v2:172.21.15.142:6838/3822134597 2024-09-19T09:27:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84d22e2740 1 --2- 172.21.15.16:0/2803283877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206d9d0 0x562242078420 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84d22e2740 1 -- 172.21.15.16:0/2803283877 >> 172.21.15.16:0/2803283877 conn(0x562242069f60 msgr2=0x56224206fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84d22e2740 1 -- 172.21.15.16:0/2803283877 shutdown_connections 2024-09-19T09:27:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84d22e2740 1 -- 172.21.15.16:0/2803283877 wait complete. 2024-09-19T09:27:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:46.985+0000 7f84d22e2740 20 client.15411 trim_cache size 0 max 16384 2024-09-19T09:27:47.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.003+0000 7f84d22e2740 1 Processor -- start 2024-09-19T09:27:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.003+0000 7f84d22e2740 1 -- start start 2024-09-19T09:27:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84d22e2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206d9d0 0x56224208a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84d22e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56224206e2f0 0x5622420729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84d22e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562242078bb0 0x562242072f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84d22e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562242088aa0 con 0x562242078bb0 2024-09-19T09:27:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84d22e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562242089170 con 0x56224206e2f0 2024-09-19T09:27:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84d22e2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562242089840 con 0x56224206d9d0 2024-09-19T09:27:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84ce4a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562242078bb0 0x562242072f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84ce4a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562242078bb0 0x562242072f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42006/0 (socket says 172.21.15.16:42006) 2024-09-19T09:27:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84cd4a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56224206e2f0 0x5622420729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84ce4a6640 1 -- 172.21.15.16:0/3369889931 learned_addr learned my addr 172.21.15.16:0/3369889931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84cdca5640 1 --2- 172.21.15.16:0/3369889931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206d9d0 0x56224208a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:47.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84ce4a6640 1 -- 172.21.15.16:0/3369889931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206d9d0 msgr2=0x56224208a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:47.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84ce4a6640 1 --2- 172.21.15.16:0/3369889931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206d9d0 0x56224208a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84ce4a6640 1 -- 172.21.15.16:0/3369889931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56224206e2f0 msgr2=0x5622420729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:47.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84ce4a6640 1 --2- 172.21.15.16:0/3369889931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56224206e2f0 0x5622420729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84ce4a6640 1 -- 172.21.15.16:0/3369889931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56224207b300 con 0x562242078bb0 2024-09-19T09:27:47.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84cd4a4640 1 --2- 172.21.15.16:0/3369889931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56224206e2f0 0x5622420729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:47.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.004+0000 7f84ce4a6640 1 --2- 172.21.15.16:0/3369889931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562242078bb0 0x562242072f10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009ca0 tx=0x7f84c0009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d349bc634ebac87b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:47.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.005+0000 7f84b37fe640 1 -- 172.21.15.16:0/3369889931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0034040 con 0x562242078bb0 2024-09-19T09:27:47.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.005+0000 7f84b37fe640 1 -- 172.21.15.16:0/3369889931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c0004720 con 0x562242078bb0 2024-09-19T09:27:47.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.005+0000 7f84b37fe640 1 -- 172.21.15.16:0/3369889931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c004bb60 con 0x562242078bb0 2024-09-19T09:27:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.005+0000 7f84d22e2740 1 -- 172.21.15.16:0/3369889931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562242078bb0 msgr2=0x562242072f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.005+0000 7f84d22e2740 1 --2- 172.21.15.16:0/3369889931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562242078bb0 0x562242072f10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009ca0 tx=0x7f84c0009360 comp rx=0 tx=0).stop 2024-09-19T09:27:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.005+0000 7f84d22e2740 1 -- 172.21.15.16:0/3369889931 shutdown_connections 2024-09-19T09:27:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.005+0000 7f84d22e2740 1 --2- 172.21.15.16:0/3369889931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562242078bb0 0x562242072f10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.005+0000 7f84d22e2740 1 --2- 172.21.15.16:0/3369889931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56224206e2f0 0x5622420729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.005+0000 7f84d22e2740 1 --2- 172.21.15.16:0/3369889931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56224206d9d0 0x56224208a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.005+0000 7f84d22e2740 1 -- 172.21.15.16:0/3369889931 >> 172.21.15.16:0/3369889931 conn(0x562242069f60 msgr2=0x562242069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.005+0000 7f84d22e2740 1 -- 172.21.15.16:0/3369889931 shutdown_connections 2024-09-19T09:27:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.006+0000 7f84d22e2740 1 -- 172.21.15.16:0/3369889931 wait complete. 2024-09-19T09:27:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.006+0000 7f84d22e2740 1 -- start start 2024-09-19T09:27:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.006+0000 7f84d22e2740 1 Processor -- start 2024-09-19T09:27:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.006+0000 7f84d22e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56224206d9d0 0x562242093300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.006+0000 7f84d22e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 0x56224208e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.006+0000 7f84d22e2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562242078bb0 0x56224208ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.006+0000 7f84d22e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56224208a5e0 con 0x56224206e2f0 2024-09-19T09:27:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.006+0000 7f84d22e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562242089f10 con 0x56224206d9d0 2024-09-19T09:27:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.006+0000 7f84d22e2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56224208acb0 con 0x562242078bb0 2024-09-19T09:27:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.006+0000 7f84d22e2740 10 client.0 _ll_register_callbacks cb 0x7f84c401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84cd4a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 0x56224208e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84cd4a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 0x56224208e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42022/0 (socket says 172.21.15.16:42022) 2024-09-19T09:27:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84cd4a4640 1 -- 172.21.15.16:0/2212360661 learned_addr learned my addr 172.21.15.16:0/2212360661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84cdca5640 1 --2- 172.21.15.16:0/2212360661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56224206d9d0 0x562242093300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84ce4a6640 1 --2- 172.21.15.16:0/2212360661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562242078bb0 0x56224208ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84cd4a4640 1 -- 172.21.15.16:0/2212360661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562242078bb0 msgr2=0x56224208ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84cd4a4640 1 --2- 172.21.15.16:0/2212360661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562242078bb0 0x56224208ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84cd4a4640 1 -- 172.21.15.16:0/2212360661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56224206d9d0 msgr2=0x562242093300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84cd4a4640 1 --2- 172.21.15.16:0/2212360661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56224206d9d0 0x562242093300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84cd4a4640 1 -- 172.21.15.16:0/2212360661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622420f2a00 con 0x56224206e2f0 2024-09-19T09:27:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84cd4a4640 1 --2- 172.21.15.16:0/2212360661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 0x56224208e630 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f84b809bec0 tx=0x7f84b80bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8251fe9b20ff3b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84b17fa640 1 -- 172.21.15.16:0/2212360661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80d2e60 con 0x56224206e2f0 2024-09-19T09:27:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84b17fa640 1 -- 172.21.15.16:0/2212360661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f84b80cd520 con 0x56224206e2f0 2024-09-19T09:27:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84d22e2740 1 -- 172.21.15.16:0/2212360661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84c0004720 con 0x56224206e2f0 2024-09-19T09:27:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84b17fa640 1 -- 172.21.15.16:0/2212360661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84b809bf30 con 0x56224206e2f0 2024-09-19T09:27:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84d22e2740 10 client.15417 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.007+0000 7f84d22e2740 1 -- 172.21.15.16:0/2212360661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622420f3ba0 con 0x56224206e2f0 2024-09-19T09:27:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.008+0000 7f84b17fa640 1 -- 172.21.15.16:0/2212360661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c0004720 con 0x56224206e2f0 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.008+0000 7f84b17fa640 1 -- 172.21.15.16:0/2212360661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84c4019ce0 con 0x56224206e2f0 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.008+0000 7f84d22e2740 10 client.15417 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.008+0000 7f84d22e2740 10 client.15417 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.008+0000 7f84d22e2740 20 client.15417 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.008+0000 7f84d22e2740 1 --2- 172.21.15.16:0/2212360661 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x5622420f87d0 0x5622420f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.008+0000 7f84d22e2740 4 client.15417 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.008+0000 7f84d22e2740 1 -- 172.21.15.16:0/2212360661 --> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] -- command(tid 0: {"prefix": "status"}) -- 0x562241fdbb10 con 0x5622420f87d0 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.008+0000 7f84cdca5640 1 --2- 172.21.15.16:0/2212360661 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x5622420f87d0 0x5622420f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.008+0000 7f84b17fa640 10 client.15417 ms_handle_connect on v2:172.21.15.142:6838/3822134597 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.008+0000 7f84cdca5640 1 --2- 172.21.15.16:0/2212360661 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x5622420f87d0 0x5622420f74e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6aa9c187331bbbad server_cookie=b2c39482c18abb6d in_seq=0 out_seq=0 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84b17fa640 1 -- 172.21.15.16:0/2212360661 <== mds.1 v2:172.21.15.142:6838/3822134597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562241fdbb10 con 0x5622420f87d0 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84b17fa640 10 client.15417 handle_command_reply: tid=0 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84d22e2740 1 client.15417 shutdown 2024-09-19T09:27:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84d22e2740 1 -- 172.21.15.16:0/2212360661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 msgr2=0x56224208e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84d22e2740 1 --2- 172.21.15.16:0/2212360661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 0x56224208e630 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f84b809bec0 tx=0x7f84b80bc040 comp rx=0 tx=0).stop 2024-09-19T09:27:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84d22e2740 1 -- 172.21.15.16:0/2212360661 shutdown_connections 2024-09-19T09:27:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84d22e2740 1 --2- 172.21.15.16:0/2212360661 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x5622420f87d0 0x5622420f74e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84d22e2740 1 --2- 172.21.15.16:0/2212360661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562242078bb0 0x56224208ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84b17fa640 0 client.15417 ms_handle_reset on v2:172.21.15.142:6838/3822134597 2024-09-19T09:27:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84d22e2740 1 --2- 172.21.15.16:0/2212360661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224206e2f0 0x56224208e630 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84d22e2740 1 --2- 172.21.15.16:0/2212360661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56224206d9d0 0x562242093300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84d22e2740 1 -- 172.21.15.16:0/2212360661 >> 172.21.15.16:0/2212360661 conn(0x562242069f60 msgr2=0x562242083b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84d22e2740 1 -- 172.21.15.16:0/2212360661 shutdown_connections 2024-09-19T09:27:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84d22e2740 1 -- 172.21.15.16:0/2212360661 wait complete. 2024-09-19T09:27:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.009+0000 7f84d22e2740 20 client.15417 trim_cache size 0 max 16384 2024-09-19T09:27:47.015 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:27:47.016 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:27:47.016 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-19T09:27:47.016 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24484, 2024-09-19T09:27:47.016 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:27:47.016 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:27:47.016 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:27:47.016 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 129.69125837199999, 2024-09-19T09:27:47.016 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-19T09:27:47.016 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:27:47.016 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T09:27:47.016 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 156.03883951200001 2024-09-19T09:27:47.016 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:27:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.010+0000 7f84cff09640 1 -- 172.21.15.16:0/560554411 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f84b40a3240 msgr2=0x7f84b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.010+0000 7f84cff09640 1 --2- 172.21.15.16:0/560554411 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f84b40a3240 0x7f84b40a5700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f84c0005fe0 tx=0x7f84c0005c50 comp rx=0 tx=0).stop 2024-09-19T09:27:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.010+0000 7f84cff09640 1 -- 172.21.15.16:0/560554411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102570 msgr2=0x7f84c80fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.010+0000 7f84cff09640 1 --2- 172.21.15.16:0/560554411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102570 0x7f84c80fd1c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c920 tx=0x7f84b800cd20 comp rx=0 tx=0).stop 2024-09-19T09:27:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.010+0000 7f84cff09640 1 -- 172.21.15.16:0/560554411 shutdown_connections 2024-09-19T09:27:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.010+0000 7f84cff09640 1 --2- 172.21.15.16:0/560554411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c8102ef0 0x7f84c80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.010+0000 7f84cff09640 1 --2- 172.21.15.16:0/560554411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102570 0x7f84c80fd1c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.010+0000 7f84cff09640 1 --2- 172.21.15.16:0/560554411 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f84b40a3240 0x7f84b40a5700 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.010+0000 7f84cff09640 1 --2- 172.21.15.16:0/560554411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84c8100e20 0x7f84c80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.010+0000 7f84cff09640 1 -- 172.21.15.16:0/560554411 >> 172.21.15.16:0/560554411 conn(0x7f84c80fc7d0 msgr2=0x7f84c806a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.010+0000 7f84cff09640 1 -- 172.21.15.16:0/560554411 shutdown_connections 2024-09-19T09:27:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:47.010+0000 7f84cff09640 1 -- 172.21.15.16:0/560554411 wait complete. 2024-09-19T09:27:47.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:46 smithi016 ceph-mon[30979]: pgmap v343: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-19T09:27:47.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:46 smithi142 ceph-mon[35032]: pgmap v343: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-19T09:27:47.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:46 smithi097 ceph-mon[34864]: pgmap v343: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-19T09:27:47.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T09:27:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:27:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.449+0000 7faa9bf1e640 1 -- 172.21.15.16:0/1622145934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa94102570 msgr2=0x7faa941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.449+0000 7faa9bf1e640 1 --2- 172.21.15.16:0/1622145934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa94102570 0x7faa941029b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faa84009900 tx=0x7faa8402a480 comp rx=0 tx=0).stop 2024-09-19T09:27:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.449+0000 7faa9bf1e640 1 -- 172.21.15.16:0/1622145934 shutdown_connections 2024-09-19T09:27:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.449+0000 7faa9bf1e640 1 --2- 172.21.15.16:0/1622145934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa94102ef0 0x7faa9410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.449+0000 7faa9bf1e640 1 --2- 172.21.15.16:0/1622145934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa94102570 0x7faa941029b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.449+0000 7faa9bf1e640 1 --2- 172.21.15.16:0/1622145934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94100e20 0x7faa94101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.449+0000 7faa9bf1e640 1 -- 172.21.15.16:0/1622145934 >> 172.21.15.16:0/1622145934 conn(0x7faa940fc7d0 msgr2=0x7faa940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.449+0000 7faa9bf1e640 1 -- 172.21.15.16:0/1622145934 shutdown_connections 2024-09-19T09:27:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.449+0000 7faa9bf1e640 1 -- 172.21.15.16:0/1622145934 wait complete. 2024-09-19T09:27:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.450+0000 7faa9bf1e640 1 Processor -- start 2024-09-19T09:27:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.450+0000 7faa9bf1e640 1 -- start start 2024-09-19T09:27:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.450+0000 7faa9bf1e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa94100e20 0x7faa94198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.450+0000 7faa9bf1e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94102570 0x7faa94198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.450+0000 7faa9bf1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa94102ef0 0x7faa9419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.450+0000 7faa9bf1e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa94110fa0 con 0x7faa94102ef0 2024-09-19T09:27:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.450+0000 7faa9bf1e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faa94110e20 con 0x7faa94102570 2024-09-19T09:27:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.450+0000 7faa9bf1e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faa94111120 con 0x7faa94100e20 2024-09-19T09:27:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa9a4bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa94102ef0 0x7faa9419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa994b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94102570 0x7faa94198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa9a4bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa94102ef0 0x7faa9419ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42044/0 (socket says 172.21.15.16:42044) 2024-09-19T09:27:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa9a4bb640 1 -- 172.21.15.16:0/3379712295 learned_addr learned my addr 172.21.15.16:0/3379712295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa99cba640 1 --2- 172.21.15.16:0/3379712295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa94100e20 0x7faa94198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa9a4bb640 1 -- 172.21.15.16:0/3379712295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa94100e20 msgr2=0x7faa94198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa9a4bb640 1 --2- 172.21.15.16:0/3379712295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa94100e20 0x7faa94198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa9a4bb640 1 -- 172.21.15.16:0/3379712295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94102570 msgr2=0x7faa94198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa9a4bb640 1 --2- 172.21.15.16:0/3379712295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94102570 0x7faa94198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa9a4bb640 1 -- 172.21.15.16:0/3379712295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9419d4f0 con 0x7faa94102ef0 2024-09-19T09:27:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa9a4bb640 1 --2- 172.21.15.16:0/3379712295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa94102ef0 0x7faa9419ce10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00b970 tx=0x7faa8c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa8affd640 1 -- 172.21.15.16:0/3379712295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c019070 con 0x7faa94102ef0 2024-09-19T09:27:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa8affd640 1 -- 172.21.15.16:0/3379712295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa8c00cb40 con 0x7faa94102ef0 2024-09-19T09:27:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa9bf1e640 1 -- 172.21.15.16:0/3379712295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9419d7e0 con 0x7faa94102ef0 2024-09-19T09:27:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa9bf1e640 1 -- 172.21.15.16:0/3379712295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa941a50c0 con 0x7faa94102ef0 2024-09-19T09:27:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.451+0000 7faa8affd640 1 -- 172.21.15.16:0/3379712295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c007650 con 0x7faa94102ef0 2024-09-19T09:27:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.454+0000 7faa8affd640 1 -- 172.21.15.16:0/3379712295 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7faa8c01c070 con 0x7faa94102ef0 2024-09-19T09:27:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.454+0000 7faa9e2f7740 1 Processor -- start 2024-09-19T09:27:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.454+0000 7faa9e2f7740 1 -- start start 2024-09-19T09:27:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.454+0000 7faa9e2f7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf4128d0 0x5642bf412cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.454+0000 7faa9e2f7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf421a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.454+0000 7faa9e2f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf421fc0 0x5642bf4243b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.454+0000 7faa9e2f7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5642bf379090 con 0x5642bf4131f0 2024-09-19T09:27:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.454+0000 7faa9e2f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5642bf320a70 con 0x5642bf421fc0 2024-09-19T09:27:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.454+0000 7faa9e2f7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5642bf375050 con 0x5642bf4128d0 2024-09-19T09:27:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa99cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf421a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa99cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf421a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42056/0 (socket says 172.21.15.16:42056) 2024-09-19T09:27:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa99cba640 1 -- 172.21.15.16:0/2495117880 learned_addr learned my addr 172.21.15.16:0/2495117880 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa994b9640 1 --2- 172.21.15.16:0/2495117880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf421fc0 0x5642bf4243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa8affd640 1 --2- 172.21.15.16:0/3379712295 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7faa800a32c0 0x7faa800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa994b9640 1 --2- 172.21.15.16:0/2495117880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf4128d0 0x5642bf412cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa99cba640 1 -- 172.21.15.16:0/2495117880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf4128d0 msgr2=0x5642bf412cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa99cba640 1 --2- 172.21.15.16:0/2495117880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf4128d0 0x5642bf412cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa99cba640 1 -- 172.21.15.16:0/2495117880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf421fc0 msgr2=0x5642bf4243b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa99cba640 1 --2- 172.21.15.16:0/2495117880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf421fc0 0x5642bf4243b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa99cba640 1 -- 172.21.15.16:0/2495117880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642bf32d130 con 0x5642bf4131f0 2024-09-19T09:27:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa9a4bb640 1 --2- 172.21.15.16:0/3379712295 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7faa800a32c0 0x7faa800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa994b9640 1 --2- 172.21.15.16:0/2495117880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf4128d0 0x5642bf412cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa8affd640 1 -- 172.21.15.16:0/3379712295 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faa8c0ceda0 con 0x7faa94102ef0 2024-09-19T09:27:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa99cba640 1 --2- 172.21.15.16:0/2495117880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf421a80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7faa9000cc60 tx=0x7faa9000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f7ca97264c2cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa7ffff640 1 -- 172.21.15.16:0/2495117880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90014070 con 0x5642bf4131f0 2024-09-19T09:27:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.455+0000 7faa7ffff640 1 -- 172.21.15.16:0/2495117880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa900047a0 con 0x5642bf4131f0 2024-09-19T09:27:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.456+0000 7faa7ffff640 1 -- 172.21.15.16:0/2495117880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90010790 con 0x5642bf4131f0 2024-09-19T09:27:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.456+0000 7faa9a4bb640 1 --2- 172.21.15.16:0/3379712295 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7faa800a32c0 0x7faa800a5780 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faa8c09b400 tx=0x7faa8c090950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.456+0000 7faa9e2f7740 1 -- 172.21.15.16:0/2495117880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 msgr2=0x5642bf421a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.456+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/2495117880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf421a80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7faa9000cc60 tx=0x7faa9000e720 comp rx=0 tx=0).stop 2024-09-19T09:27:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.456+0000 7faa9e2f7740 1 -- 172.21.15.16:0/2495117880 shutdown_connections 2024-09-19T09:27:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.456+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/2495117880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf421fc0 0x5642bf4243b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.456+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/2495117880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf421a80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.456+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/2495117880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf4128d0 0x5642bf412cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.456+0000 7faa9e2f7740 1 -- 172.21.15.16:0/2495117880 >> 172.21.15.16:0/2495117880 conn(0x5642bf40ef60 msgr2=0x5642bf40f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.457+0000 7faa9e2f7740 1 -- 172.21.15.16:0/2495117880 shutdown_connections 2024-09-19T09:27:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.457+0000 7faa9e2f7740 1 -- 172.21.15.16:0/2495117880 wait complete. 2024-09-19T09:27:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.457+0000 7faa9e2f7740 1 -- start start 2024-09-19T09:27:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.457+0000 7faa9e2f7740 1 Processor -- start 2024-09-19T09:27:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.457+0000 7faa9e2f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf4217c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa9e2f7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf41eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa9e2f7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf421fc0 0x5642bf41f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa9e2f7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5642bf424bf0 con 0x5642bf4131f0 2024-09-19T09:27:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa9e2f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5642bf333f20 con 0x5642bf4128d0 2024-09-19T09:27:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa9e2f7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5642bf424d70 con 0x5642bf421fc0 2024-09-19T09:27:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa99cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf41eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa9e2f7740 10 client.0 _ll_register_callbacks cb 0x5642bf2fd030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa99cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf41eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42060/0 (socket says 172.21.15.16:42060) 2024-09-19T09:27:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa99cba640 1 -- 172.21.15.16:0/211218596 learned_addr learned my addr 172.21.15.16:0/211218596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa994b9640 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf4217c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa994b9640 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf421fc0 0x5642bf41f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa99cba640 1 -- 172.21.15.16:0/211218596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf421fc0 msgr2=0x5642bf41f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa99cba640 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf421fc0 0x5642bf41f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa99cba640 1 -- 172.21.15.16:0/211218596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 msgr2=0x5642bf4217c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa99cba640 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf4217c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa99cba640 1 -- 172.21.15.16:0/211218596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642bf0ea2b0 con 0x5642bf4131f0 2024-09-19T09:27:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa994b9640 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf4217c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:27:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa99cba640 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf41eb10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7faa90010bb0 tx=0x7faa90010be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8277aad2f30369b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.458+0000 7faa7d7fa640 1 -- 172.21.15.16:0/211218596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90027060 con 0x5642bf4131f0 2024-09-19T09:27:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.459+0000 7faa7d7fa640 1 -- 172.21.15.16:0/211218596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faa90010e00 con 0x5642bf4131f0 2024-09-19T09:27:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.459+0000 7faa9e2f7740 1 -- 172.21.15.16:0/211218596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa900047a0 con 0x5642bf4131f0 2024-09-19T09:27:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.459+0000 7faa7d7fa640 1 -- 172.21.15.16:0/211218596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa90014070 con 0x5642bf4131f0 2024-09-19T09:27:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.459+0000 7faa9e2f7740 10 client.15429 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.459+0000 7faa9e2f7740 1 -- 172.21.15.16:0/211218596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5642bf430520 con 0x5642bf4131f0 2024-09-19T09:27:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.459+0000 7faa7d7fa640 1 -- 172.21.15.16:0/211218596 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa900047a0 con 0x5642bf4131f0 2024-09-19T09:27:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.459+0000 7faa7d7fa640 1 -- 172.21.15.16:0/211218596 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faa9002c070 con 0x5642bf4131f0 2024-09-19T09:27:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.459+0000 7faa9e2f7740 10 client.15429 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.459+0000 7faa9e2f7740 10 client.15429 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:27:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.459+0000 7faa9e2f7740 20 client.15429 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.459+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5642bf4328f0 0x5642bf433d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.459+0000 7faa9e2f7740 4 client.15429 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.459+0000 7faa9e2f7740 1 -- 172.21.15.16:0/211218596 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642bf386600 con 0x5642bf4328f0 2024-09-19T09:27:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.460+0000 7faa994b9640 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5642bf4328f0 0x5642bf433d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.460+0000 7faa994b9640 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5642bf4328f0 0x5642bf433d60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b78cb5200f5e84b7 server_cookie=cadde85fb2116aaf in_seq=0 out_seq=0 2024-09-19T09:27:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.460+0000 7faa7d7fa640 10 client.15429 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa7d7fa640 1 -- 172.21.15.16:0/211218596 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642bf386600 con 0x5642bf4328f0 2024-09-19T09:27:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa7d7fa640 10 client.15429 handle_command_reply: tid=0 2024-09-19T09:27:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa9e2f7740 1 client.15429 shutdown 2024-09-19T09:27:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa9e2f7740 1 -- 172.21.15.16:0/211218596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 msgr2=0x5642bf41eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf41eb10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7faa90010bb0 tx=0x7faa90010be0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa9e2f7740 1 -- 172.21.15.16:0/211218596 shutdown_connections 2024-09-19T09:27:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5642bf4328f0 0x5642bf433d60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf421fc0 0x5642bf41f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf41eb10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa7d7fa640 0 client.15429 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/211218596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf4217c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa9e2f7740 1 -- 172.21.15.16:0/211218596 >> 172.21.15.16:0/211218596 conn(0x5642bf40ef60 msgr2=0x5642bf414cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa9e2f7740 1 -- 172.21.15.16:0/211218596 shutdown_connections 2024-09-19T09:27:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa9e2f7740 1 -- 172.21.15.16:0/211218596 wait complete. 2024-09-19T09:27:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.461+0000 7faa9e2f7740 20 client.15429 trim_cache size 0 max 16384 2024-09-19T09:27:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa9e2f7740 1 Processor -- start 2024-09-19T09:27:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa9e2f7740 1 -- start start 2024-09-19T09:27:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa9e2f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf432220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa9e2f7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf432fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa9e2f7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf421fc0 0x5642bf431920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa9e2f7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5642bf4320a0 con 0x5642bf4131f0 2024-09-19T09:27:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa9e2f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5642bf432770 con 0x5642bf4128d0 2024-09-19T09:27:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa9e2f7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5642bf432e40 con 0x5642bf421fc0 2024-09-19T09:27:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa99cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf432fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa99cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf432fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42076/0 (socket says 172.21.15.16:42076) 2024-09-19T09:27:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa99cba640 1 -- 172.21.15.16:0/3186252966 learned_addr learned my addr 172.21.15.16:0/3186252966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa994b9640 1 --2- 172.21.15.16:0/3186252966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf432220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa99cba640 1 -- 172.21.15.16:0/3186252966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf421fc0 msgr2=0x5642bf431920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa99cba640 1 --2- 172.21.15.16:0/3186252966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf421fc0 0x5642bf431920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa99cba640 1 -- 172.21.15.16:0/3186252966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 msgr2=0x5642bf432220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa99cba640 1 --2- 172.21.15.16:0/3186252966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf432220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.479+0000 7faa99cba640 1 -- 172.21.15.16:0/3186252966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642bf41f630 con 0x5642bf4131f0 2024-09-19T09:27:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.480+0000 7faa994b9640 1 --2- 172.21.15.16:0/3186252966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf432220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.480+0000 7faa99cba640 1 --2- 172.21.15.16:0/3186252966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf432fc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7faa900228e0 tx=0x7faa9000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8cdd25c1e4c90f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.480+0000 7faa7f7fe640 1 -- 172.21.15.16:0/3186252966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90032020 con 0x5642bf4131f0 2024-09-19T09:27:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.480+0000 7faa7f7fe640 1 -- 172.21.15.16:0/3186252966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa900223f0 con 0x5642bf4131f0 2024-09-19T09:27:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.480+0000 7faa7f7fe640 1 -- 172.21.15.16:0/3186252966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90027040 con 0x5642bf4131f0 2024-09-19T09:27:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.480+0000 7faa9e2f7740 1 -- 172.21.15.16:0/3186252966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 msgr2=0x5642bf432fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.480+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/3186252966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf432fc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7faa900228e0 tx=0x7faa9000e720 comp rx=0 tx=0).stop 2024-09-19T09:27:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.480+0000 7faa9e2f7740 1 -- 172.21.15.16:0/3186252966 shutdown_connections 2024-09-19T09:27:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.480+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/3186252966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf421fc0 0x5642bf431920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.480+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/3186252966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf4131f0 0x5642bf432fc0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.480+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/3186252966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf432220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.480+0000 7faa9e2f7740 1 -- 172.21.15.16:0/3186252966 >> 172.21.15.16:0/3186252966 conn(0x5642bf40ef60 msgr2=0x5642bf40f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.481+0000 7faa9e2f7740 1 -- 172.21.15.16:0/3186252966 shutdown_connections 2024-09-19T09:27:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.481+0000 7faa9e2f7740 1 -- 172.21.15.16:0/3186252966 wait complete. 2024-09-19T09:27:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.481+0000 7faa9e2f7740 1 -- start start 2024-09-19T09:27:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.481+0000 7faa9e2f7740 1 Processor -- start 2024-09-19T09:27:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.481+0000 7faa9e2f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf41ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.481+0000 7faa9e2f7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf4131f0 0x5642bf41b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.481+0000 7faa9e2f7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf421fc0 0x5642bf41b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.481+0000 7faa9e2f7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5642bf433be0 con 0x5642bf421fc0 2024-09-19T09:27:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.481+0000 7faa9e2f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5642bf433510 con 0x5642bf4128d0 2024-09-19T09:27:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.481+0000 7faa9e2f7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5642bf4342b0 con 0x5642bf4131f0 2024-09-19T09:27:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa9e2f7740 10 client.0 _ll_register_callbacks cb 0x7faa84004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa994b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf421fc0 0x5642bf41b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa994b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf421fc0 0x5642bf41b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42082/0 (socket says 172.21.15.16:42082) 2024-09-19T09:27:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa994b9640 1 -- 172.21.15.16:0/560668038 learned_addr learned my addr 172.21.15.16:0/560668038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa994b9640 1 --2- 172.21.15.16:0/560668038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf41ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa99cba640 1 --2- 172.21.15.16:0/560668038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf4131f0 0x5642bf41b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa994b9640 1 -- 172.21.15.16:0/560668038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf4131f0 msgr2=0x5642bf41b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa994b9640 1 --2- 172.21.15.16:0/560668038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf4131f0 0x5642bf41b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa994b9640 1 -- 172.21.15.16:0/560668038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 msgr2=0x5642bf41ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa994b9640 1 --2- 172.21.15.16:0/560668038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf41ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa994b9640 1 -- 172.21.15.16:0/560668038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642bf4100f0 con 0x5642bf421fc0 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa994b9640 1 --2- 172.21.15.16:0/560668038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf421fc0 0x5642bf41b800 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faa84005b90 tx=0x7faa8402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f046ffa1cd0153 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa7d7fa640 1 -- 172.21.15.16:0/560668038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84035040 con 0x5642bf421fc0 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa7d7fa640 1 -- 172.21.15.16:0/560668038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7faa84003c70 con 0x5642bf421fc0 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.482+0000 7faa9e2f7740 1 -- 172.21.15.16:0/560668038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa900223f0 con 0x5642bf421fc0 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.483+0000 7faa7d7fa640 1 -- 172.21.15.16:0/560668038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa84060050 con 0x5642bf421fc0 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.483+0000 7faa9e2f7740 10 client.15435 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.483+0000 7faa9e2f7740 1 -- 172.21.15.16:0/560668038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5642bf43c140 con 0x5642bf421fc0 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.483+0000 7faa7d7fa640 1 -- 172.21.15.16:0/560668038 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa900223f0 con 0x5642bf421fc0 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.483+0000 7faa7d7fa640 1 -- 172.21.15.16:0/560668038 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faa84057070 con 0x5642bf421fc0 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.483+0000 7faa9e2f7740 10 client.15435 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.483+0000 7faa9e2f7740 10 client.15435 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T09:27:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.483+0000 7faa9e2f7740 20 client.15435 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.483+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/560668038 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5642bf4a0e70 0x5642bf49fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.483+0000 7faa9e2f7740 4 client.15435 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T09:27:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.483+0000 7faa9e2f7740 1 -- 172.21.15.16:0/560668038 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "status"}) -- 0x5642bf380b10 con 0x5642bf4a0e70 2024-09-19T09:27:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.483+0000 7faa99cba640 1 --2- 172.21.15.16:0/560668038 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5642bf4a0e70 0x5642bf49fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa99cba640 1 --2- 172.21.15.16:0/560668038 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5642bf4a0e70 0x5642bf49fb80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae9568faebf12e2 server_cookie=f75e8ca9e3537704 in_seq=0 out_seq=0 2024-09-19T09:27:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa7d7fa640 10 client.15435 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa7d7fa640 1 -- 172.21.15.16:0/560668038 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5642bf380b10 con 0x5642bf4a0e70 2024-09-19T09:27:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa7d7fa640 10 client.15435 handle_command_reply: tid=0 2024-09-19T09:27:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa9e2f7740 1 client.15435 shutdown 2024-09-19T09:27:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa9e2f7740 1 -- 172.21.15.16:0/560668038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf421fc0 msgr2=0x5642bf41b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/560668038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf421fc0 0x5642bf41b800 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faa84005b90 tx=0x7faa8402a480 comp rx=0 tx=0).stop 2024-09-19T09:27:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa9e2f7740 1 -- 172.21.15.16:0/560668038 shutdown_connections 2024-09-19T09:27:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/560668038 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5642bf4a0e70 0x5642bf49fb80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/560668038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642bf4131f0 0x5642bf41b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/560668038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642bf421fc0 0x5642bf41b800 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa9e2f7740 1 --2- 172.21.15.16:0/560668038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5642bf4128d0 0x5642bf41ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa7d7fa640 0 client.15435 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa9e2f7740 1 -- 172.21.15.16:0/560668038 >> 172.21.15.16:0/560668038 conn(0x5642bf40ef60 msgr2=0x5642bf431920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa9e2f7740 1 -- 172.21.15.16:0/560668038 shutdown_connections 2024-09-19T09:27:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.484+0000 7faa9e2f7740 1 -- 172.21.15.16:0/560668038 wait complete. 2024-09-19T09:27:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.485+0000 7faa9e2f7740 20 client.15435 trim_cache size 0 max 16384 2024-09-19T09:27:48.494 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:27:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ee1ab676-7667-11ef-bceb-c7b262605968", 2024-09-19T09:27:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-19T09:27:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24500, 2024-09-19T09:27:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-19T09:27:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-19T09:27:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-19T09:27:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 138.198184209, 2024-09-19T09:27:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-19T09:27:48.494 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-09-19T09:27:48.495 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T09:27:48.495 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 149.154316978 2024-09-19T09:27:48.495 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:27:48.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.485+0000 7faa9bf1e640 1 -- 172.21.15.16:0/3379712295 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7faa800a32c0 msgr2=0x7faa800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.485+0000 7faa9bf1e640 1 --2- 172.21.15.16:0/3379712295 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7faa800a32c0 0x7faa800a5780 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faa8c09b400 tx=0x7faa8c090950 comp rx=0 tx=0).stop 2024-09-19T09:27:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.485+0000 7faa9bf1e640 1 -- 172.21.15.16:0/3379712295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa94102ef0 msgr2=0x7faa9419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.485+0000 7faa9bf1e640 1 --2- 172.21.15.16:0/3379712295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa94102ef0 0x7faa9419ce10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00b970 tx=0x7faa8c00be30 comp rx=0 tx=0).stop 2024-09-19T09:27:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.485+0000 7faa9bf1e640 1 -- 172.21.15.16:0/3379712295 shutdown_connections 2024-09-19T09:27:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.485+0000 7faa9bf1e640 1 --2- 172.21.15.16:0/3379712295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa94102ef0 0x7faa9419ce10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.485+0000 7faa9bf1e640 1 --2- 172.21.15.16:0/3379712295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94102570 0x7faa94198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.485+0000 7faa9bf1e640 1 --2- 172.21.15.16:0/3379712295 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7faa800a32c0 0x7faa800a5780 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.485+0000 7faa9bf1e640 1 --2- 172.21.15.16:0/3379712295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa94100e20 0x7faa94198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.485+0000 7faa9bf1e640 1 -- 172.21.15.16:0/3379712295 >> 172.21.15.16:0/3379712295 conn(0x7faa940fc7d0 msgr2=0x7faa9406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.485+0000 7faa9bf1e640 1 -- 172.21.15.16:0/3379712295 shutdown_connections 2024-09-19T09:27:48.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:27:48.485+0000 7faa9bf1e640 1 -- 172.21.15.16:0/3379712295 wait complete. 2024-09-19T09:27:49.193 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-19T09:27:49.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:49 smithi016 ceph-mon[30979]: pgmap v344: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-09-19T09:27:49.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:49 smithi142 ceph-mon[35032]: pgmap v344: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-09-19T09:27:49.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:49 smithi097 ceph-mon[34864]: pgmap v344: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2024-09-19T09:27:51.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:51 smithi142 ceph-mon[35032]: pgmap v345: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-09-19T09:27:51.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:51 smithi097 ceph-mon[34864]: pgmap v345: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-09-19T09:27:51.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:51 smithi016 ceph-mon[30979]: pgmap v345: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-09-19T09:27:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.441+0000 7fb87bf4b640 1 -- 172.21.15.16:0/2661172123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8741003b0 msgr2=0x7fb8741027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.441+0000 7fb87bf4b640 1 --2- 172.21.15.16:0/2661172123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8741003b0 0x7fb8741027c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb870009900 tx=0x7fb87002a410 comp rx=0 tx=0).stop 2024-09-19T09:27:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.442+0000 7fb87bf4b640 1 -- 172.21.15.16:0/2661172123 shutdown_connections 2024-09-19T09:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.442+0000 7fb87bf4b640 1 --2- 172.21.15.16:0/2661172123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8741056b0 0x7fb874105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.442+0000 7fb87bf4b640 1 --2- 172.21.15.16:0/2661172123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb874102d00 0x7fb874105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.442+0000 7fb87bf4b640 1 --2- 172.21.15.16:0/2661172123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8741003b0 0x7fb8741027c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.442+0000 7fb87bf4b640 1 -- 172.21.15.16:0/2661172123 >> 172.21.15.16:0/2661172123 conn(0x7fb8740fc1e0 msgr2=0x7fb8740fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.442+0000 7fb87bf4b640 1 -- 172.21.15.16:0/2661172123 shutdown_connections 2024-09-19T09:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.442+0000 7fb87bf4b640 1 -- 172.21.15.16:0/2661172123 wait complete. 2024-09-19T09:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.442+0000 7fb87bf4b640 1 Processor -- start 2024-09-19T09:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.442+0000 7fb87bf4b640 1 -- start start 2024-09-19T09:27:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87bf4b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8741003b0 0x7fb87419e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87bf4b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb874102d00 0x7fb87419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87bf4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8741056b0 0x7fb8741a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87bf4b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb874113e40 con 0x7fb8741056b0 2024-09-19T09:27:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87bf4b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb874113cc0 con 0x7fb8741003b0 2024-09-19T09:27:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87bf4b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb874113fc0 con 0x7fb874102d00 2024-09-19T09:27:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87a4e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8741056b0 0x7fb8741a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87a4e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8741056b0 0x7fb8741a5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42090/0 (socket says 172.21.15.16:42090) 2024-09-19T09:27:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87a4e8640 1 -- 172.21.15.16:0/223761473 learned_addr learned my addr 172.21.15.16:0/223761473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb879ce7640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8741003b0 0x7fb87419e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb8794e6640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb874102d00 0x7fb87419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87a4e8640 1 -- 172.21.15.16:0/223761473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb874102d00 msgr2=0x7fb87419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87a4e8640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb874102d00 0x7fb87419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87a4e8640 1 -- 172.21.15.16:0/223761473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8741003b0 msgr2=0x7fb87419e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87a4e8640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8741003b0 0x7fb87419e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87a4e8640 1 -- 172.21.15.16:0/223761473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8741a5fd0 con 0x7fb8741056b0 2024-09-19T09:27:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb87a4e8640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8741056b0 0x7fb8741a5960 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00eaa0 tx=0x7fb86c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb86affd640 1 -- 172.21.15.16:0/223761473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00cdb0 con 0x7fb8741056b0 2024-09-19T09:27:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.443+0000 7fb86affd640 1 -- 172.21.15.16:0/223761473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb86c005020 con 0x7fb8741056b0 2024-09-19T09:27:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.444+0000 7fb87bf4b640 1 -- 172.21.15.16:0/223761473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8741a62c0 con 0x7fb8741056b0 2024-09-19T09:27:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.444+0000 7fb86affd640 1 -- 172.21.15.16:0/223761473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0116b0 con 0x7fb8741056b0 2024-09-19T09:27:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.444+0000 7fb87bf4b640 1 -- 172.21.15.16:0/223761473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8741a6810 con 0x7fb8741056b0 2024-09-19T09:27:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.446+0000 7fb86affd640 1 -- 172.21.15.16:0/223761473 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb86c005560 con 0x7fb8741056b0 2024-09-19T09:27:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.446+0000 7fb87edc1740 1 Processor -- start 2024-09-19T09:27:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.446+0000 7fb87edc1740 1 -- start start 2024-09-19T09:27:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb87edc1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003d9a0 0x561bb003dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb87edc1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003e2e0 0x561bb0042da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb87edc1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 0x561bb0045830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb87edc1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561baffa12f0 con 0x561bb0043370 2024-09-19T09:27:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb87edc1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561baff49270 con 0x561bb003d9a0 2024-09-19T09:27:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb87edc1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561baff9d2b0 con 0x561bb003e2e0 2024-09-19T09:27:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb8794e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 0x561bb0045830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb8794e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 0x561bb0045830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42104/0 (socket says 172.21.15.16:42104) 2024-09-19T09:27:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb8794e6640 1 -- 172.21.15.16:0/2132392340 learned_addr learned my addr 172.21.15.16:0/2132392340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb8794e6640 1 --2- 172.21.15.16:0/2132392340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003d9a0 0x561bb003dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb879ce7640 1 --2- 172.21.15.16:0/2132392340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003e2e0 0x561bb0042da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb86affd640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb8600a3370 0x7fb8600a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb8794e6640 1 -- 172.21.15.16:0/2132392340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003e2e0 msgr2=0x561bb0042da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb8794e6640 1 --2- 172.21.15.16:0/2132392340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003e2e0 0x561bb0042da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb8794e6640 1 -- 172.21.15.16:0/2132392340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003d9a0 msgr2=0x561bb003dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb8794e6640 1 --2- 172.21.15.16:0/2132392340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003d9a0 0x561bb003dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb8794e6640 1 -- 172.21.15.16:0/2132392340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561baff48340 con 0x561bb0043370 2024-09-19T09:27:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb87a4e8640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb8600a3370 0x7fb8600a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb86affd640 1 -- 172.21.15.16:0/223761473 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb86c0cfb30 con 0x7fb8741056b0 2024-09-19T09:27:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.447+0000 7fb8794e6640 1 --2- 172.21.15.16:0/2132392340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 0x561bb0045830 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb864012780 tx=0x7fb8640096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56918310fc22f54f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.448+0000 7fb85ffff640 1 -- 172.21.15.16:0/2132392340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864012d00 con 0x561bb0043370 2024-09-19T09:27:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.448+0000 7fb85ffff640 1 -- 172.21.15.16:0/2132392340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8640107d0 con 0x561bb0043370 2024-09-19T09:27:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.448+0000 7fb85ffff640 1 -- 172.21.15.16:0/2132392340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8640038e0 con 0x561bb0043370 2024-09-19T09:27:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.448+0000 7fb87edc1740 1 -- 172.21.15.16:0/2132392340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 msgr2=0x561bb0045830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.448+0000 7fb87edc1740 1 --2- 172.21.15.16:0/2132392340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 0x561bb0045830 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb864012780 tx=0x7fb8640096e0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.448+0000 7fb87a4e8640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb8600a3370 0x7fb8600a5830 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0985b0 tx=0x7fb86c0bc040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.448+0000 7fb87edc1740 1 -- 172.21.15.16:0/2132392340 shutdown_connections 2024-09-19T09:27:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.448+0000 7fb87edc1740 1 --2- 172.21.15.16:0/2132392340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003e2e0 0x561bb0042da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.448+0000 7fb87edc1740 1 --2- 172.21.15.16:0/2132392340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 0x561bb0045830 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.448+0000 7fb87edc1740 1 --2- 172.21.15.16:0/2132392340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003d9a0 0x561bb003dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.448+0000 7fb87edc1740 1 -- 172.21.15.16:0/2132392340 >> 172.21.15.16:0/2132392340 conn(0x561bb0039010 msgr2=0x561bb0039410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.449+0000 7fb87edc1740 1 -- 172.21.15.16:0/2132392340 shutdown_connections 2024-09-19T09:27:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.449+0000 7fb87edc1740 1 -- 172.21.15.16:0/2132392340 wait complete. 2024-09-19T09:27:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.449+0000 7fb87edc1740 1 -- start start 2024-09-19T09:27:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.449+0000 7fb87edc1740 1 Processor -- start 2024-09-19T09:27:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.449+0000 7fb87edc1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003d9a0 0x561bb0047c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb87edc1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003e2e0 0x561bb0048190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb87edc1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 0x561bb004f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb87edc1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561bb0046340 con 0x561bb0043370 2024-09-19T09:27:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb87edc1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561baff2a290 con 0x561bb003e2e0 2024-09-19T09:27:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb87edc1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561bb00464c0 con 0x561bb003d9a0 2024-09-19T09:27:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb87edc1740 10 client.0 _ll_register_callbacks cb 0x561baff5b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb8794e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 0x561bb004f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb8794e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 0x561bb004f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42116/0 (socket says 172.21.15.16:42116) 2024-09-19T09:27:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb8794e6640 1 -- 172.21.15.16:0/3293050930 learned_addr learned my addr 172.21.15.16:0/3293050930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb879ce7640 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003e2e0 0x561bb0048190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb8794e6640 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003d9a0 0x561bb0047c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb8794e6640 1 -- 172.21.15.16:0/3293050930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003d9a0 msgr2=0x561bb0047c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb8794e6640 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003d9a0 0x561bb0047c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb8794e6640 1 -- 172.21.15.16:0/3293050930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003e2e0 msgr2=0x561bb0048190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb8794e6640 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003e2e0 0x561bb0048190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb8794e6640 1 -- 172.21.15.16:0/3293050930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bafcb4210 con 0x561bb0043370 2024-09-19T09:27:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb879ce7640 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003e2e0 0x561bb0048190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:27:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb8794e6640 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 0x561bb004f1d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x561bb0047180 tx=0x7fb864014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b30c4cfb0d3bbb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb85d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb85d7fa640 1 -- 172.21.15.16:0/3293050930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86401f4e0 con 0x561bb0043370 2024-09-19T09:27:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb85d7fa640 1 -- 172.21.15.16:0/3293050930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb86401f680 con 0x561bb0043370 2024-09-19T09:27:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.450+0000 7fb87edc1740 1 -- 172.21.15.16:0/3293050930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8640107d0 con 0x561bb0043370 2024-09-19T09:27:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.451+0000 7fb85d7fa640 1 -- 172.21.15.16:0/3293050930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb864033050 con 0x561bb0043370 2024-09-19T09:27:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.451+0000 7fb87edc1740 10 client.15447 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.451+0000 7fb87edc1740 1 -- 172.21.15.16:0/3293050930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561bafd80620 con 0x561bb0043370 2024-09-19T09:27:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.451+0000 7fb85d7fa640 1 -- 172.21.15.16:0/3293050930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8640107d0 con 0x561bb0043370 2024-09-19T09:27:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.451+0000 7fb85d7fa640 1 -- 172.21.15.16:0/3293050930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb86400e9b0 con 0x561bb0043370 2024-09-19T09:27:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.451+0000 7fb87edc1740 10 client.15447 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.451+0000 7fb87edc1740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.451+0000 7fb87edc1740 20 client.15447 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.451+0000 7fb87edc1740 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x561bb0053590 0x561bb0055980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.451+0000 7fb87edc1740 4 client.15447 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:27:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.451+0000 7fb87edc1740 1 -- 172.21.15.16:0/3293050930 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561baffaea20 con 0x561bb0053590 2024-09-19T09:27:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.451+0000 7fb879ce7640 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x561bb0053590 0x561bb0055980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.452+0000 7fb85d7fa640 10 client.15447 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.452+0000 7fb879ce7640 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x561bb0053590 0x561bb0055980 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54bac3b410230b8a server_cookie=c3f6b52d6409bf81 in_seq=0 out_seq=0 2024-09-19T09:27:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.452+0000 7fb85d7fa640 1 -- 172.21.15.16:0/3293050930 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561baffaea20 con 0x561bb0053590 2024-09-19T09:27:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.452+0000 7fb85d7fa640 10 client.15447 handle_command_reply: tid=0 2024-09-19T09:27:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.453+0000 7fb87edc1740 1 client.15447 shutdown 2024-09-19T09:27:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.453+0000 7fb87edc1740 1 -- 172.21.15.16:0/3293050930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 msgr2=0x561bb004f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.453+0000 7fb87edc1740 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 0x561bb004f1d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x561bb0047180 tx=0x7fb864014040 comp rx=0 tx=0).stop 2024-09-19T09:27:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.453+0000 7fb87edc1740 1 -- 172.21.15.16:0/3293050930 shutdown_connections 2024-09-19T09:27:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.453+0000 7fb87edc1740 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x561bb0053590 0x561bb0055980 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.453+0000 7fb87edc1740 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb0043370 0x561bb004f1d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.453+0000 7fb87edc1740 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003e2e0 0x561bb0048190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.453+0000 7fb87edc1740 1 --2- 172.21.15.16:0/3293050930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003d9a0 0x561bb0047c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.453+0000 7fb85d7fa640 0 client.15447 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.453+0000 7fb87edc1740 1 -- 172.21.15.16:0/3293050930 >> 172.21.15.16:0/3293050930 conn(0x561bb0039010 msgr2=0x561bb003a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.453+0000 7fb87edc1740 1 -- 172.21.15.16:0/3293050930 shutdown_connections 2024-09-19T09:27:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.453+0000 7fb87edc1740 1 -- 172.21.15.16:0/3293050930 wait complete. 2024-09-19T09:27:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.453+0000 7fb87edc1740 20 client.15447 trim_cache size 0 max 16384 2024-09-19T09:27:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb87edc1740 1 Processor -- start 2024-09-19T09:27:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb87edc1740 1 -- start start 2024-09-19T09:27:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb87edc1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 0x561bb003a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb87edc1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003e2e0 0x561bb004f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb87edc1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb0043370 0x561bb005a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb87edc1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741af660 con 0x561bb003d9a0 2024-09-19T09:27:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb87edc1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741b11a0 con 0x561bb003e2e0 2024-09-19T09:27:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb87edc1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741afd30 con 0x561bb0043370 2024-09-19T09:27:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb8794e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 0x561bb003a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb8794e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 0x561bb003a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42122/0 (socket says 172.21.15.16:42122) 2024-09-19T09:27:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb8794e6640 1 -- 172.21.15.16:0/1379041179 learned_addr learned my addr 172.21.15.16:0/1379041179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb879ce7640 1 --2- 172.21.15.16:0/1379041179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003e2e0 0x561bb004f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb8794e6640 1 -- 172.21.15.16:0/1379041179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb0043370 msgr2=0x561bb005a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb8794e6640 1 --2- 172.21.15.16:0/1379041179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb0043370 0x561bb005a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb8794e6640 1 -- 172.21.15.16:0/1379041179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003e2e0 msgr2=0x561bb004f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb8794e6640 1 --2- 172.21.15.16:0/1379041179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003e2e0 0x561bb004f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb8794e6640 1 -- 172.21.15.16:0/1379041179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bb00332f0 con 0x561bb003d9a0 2024-09-19T09:27:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb879ce7640 1 --2- 172.21.15.16:0/1379041179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003e2e0 0x561bb004f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:27:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb8794e6640 1 --2- 172.21.15.16:0/1379041179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 0x561bb003a670 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb86400ed80 tx=0x7fb86401d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693406a68872d3c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.468+0000 7fb85f7fe640 1 -- 172.21.15.16:0/1379041179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8640107a0 con 0x561bb003d9a0 2024-09-19T09:27:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.469+0000 7fb85f7fe640 1 -- 172.21.15.16:0/1379041179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb864009530 con 0x561bb003d9a0 2024-09-19T09:27:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.469+0000 7fb85f7fe640 1 -- 172.21.15.16:0/1379041179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86400e6a0 con 0x561bb003d9a0 2024-09-19T09:27:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.469+0000 7fb87edc1740 1 -- 172.21.15.16:0/1379041179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 msgr2=0x561bb003a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.469+0000 7fb87edc1740 1 --2- 172.21.15.16:0/1379041179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 0x561bb003a670 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb86400ed80 tx=0x7fb86401d420 comp rx=0 tx=0).stop 2024-09-19T09:27:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.469+0000 7fb87edc1740 1 -- 172.21.15.16:0/1379041179 shutdown_connections 2024-09-19T09:27:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.469+0000 7fb87edc1740 1 --2- 172.21.15.16:0/1379041179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb0043370 0x561bb005a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.469+0000 7fb87edc1740 1 --2- 172.21.15.16:0/1379041179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb003e2e0 0x561bb004f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.469+0000 7fb87edc1740 1 --2- 172.21.15.16:0/1379041179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 0x561bb003a670 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.469+0000 7fb87edc1740 1 -- 172.21.15.16:0/1379041179 >> 172.21.15.16:0/1379041179 conn(0x561bb0039010 msgr2=0x561bb0039f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.469+0000 7fb87edc1740 1 -- 172.21.15.16:0/1379041179 shutdown_connections 2024-09-19T09:27:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.469+0000 7fb87edc1740 1 -- 172.21.15.16:0/1379041179 wait complete. 2024-09-19T09:27:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.469+0000 7fb87edc1740 1 -- start start 2024-09-19T09:27:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb87edc1740 1 Processor -- start 2024-09-19T09:27:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb87edc1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 0x561bb004c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb87edc1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003e2e0 0x561bb004cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb87edc1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb0043370 0x561bb004df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb87edc1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741b0ad0 con 0x561bb003d9a0 2024-09-19T09:27:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb87edc1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741b0400 con 0x561bb0043370 2024-09-19T09:27:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb87edc1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb8741ad620 con 0x561bb003e2e0 2024-09-19T09:27:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb87edc1740 10 client.0 _ll_register_callbacks cb 0x7fb864009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:27:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb8794e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 0x561bb004c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb8794e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 0x561bb004c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42138/0 (socket says 172.21.15.16:42138) 2024-09-19T09:27:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb8794e6640 1 -- 172.21.15.16:0/2242904213 learned_addr learned my addr 172.21.15.16:0/2242904213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:27:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb8794e6640 1 --2- 172.21.15.16:0/2242904213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb0043370 0x561bb004df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb879ce7640 1 --2- 172.21.15.16:0/2242904213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003e2e0 0x561bb004cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb8794e6640 1 -- 172.21.15.16:0/2242904213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003e2e0 msgr2=0x561bb004cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb8794e6640 1 --2- 172.21.15.16:0/2242904213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003e2e0 0x561bb004cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb8794e6640 1 -- 172.21.15.16:0/2242904213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb0043370 msgr2=0x561bb004df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:27:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.470+0000 7fb8794e6640 1 --2- 172.21.15.16:0/2242904213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb0043370 0x561bb004df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb8794e6640 1 -- 172.21.15.16:0/2242904213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bb004e540 con 0x561bb003d9a0 2024-09-19T09:27:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb8794e6640 1 --2- 172.21.15.16:0/2242904213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 0x561bb004c9e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb864005c20 tx=0x7fb864005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e256c182521d2fe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:27:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb85d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:27:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb85d7fa640 1 -- 172.21.15.16:0/2242904213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86403c040 con 0x561bb003d9a0 2024-09-19T09:27:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb85d7fa640 1 -- 172.21.15.16:0/2242904213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb86400e420 con 0x561bb003d9a0 2024-09-19T09:27:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb87edc1740 1 -- 172.21.15.16:0/2242904213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb864009530 con 0x561bb003d9a0 2024-09-19T09:27:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb85d7fa640 1 -- 172.21.15.16:0/2242904213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8640054f0 con 0x561bb003d9a0 2024-09-19T09:27:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb87edc1740 10 client.15453 fetch_fsmap learned FSMap version 28 2024-09-19T09:27:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb87edc1740 1 -- 172.21.15.16:0/2242904213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561bb00be2e0 con 0x561bb003d9a0 2024-09-19T09:27:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb85d7fa640 1 -- 172.21.15.16:0/2242904213 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb864009530 con 0x561bb003d9a0 2024-09-19T09:27:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb85d7fa640 1 -- 172.21.15.16:0/2242904213 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb864042070 con 0x561bb003d9a0 2024-09-19T09:27:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb87edc1740 10 client.15453 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:27:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb87edc1740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:27:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb87edc1740 20 client.15453 populate_metadata read hostname 'smithi016' 2024-09-19T09:27:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.471+0000 7fb87edc1740 1 --2- 172.21.15.16:0/2242904213 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x561bb00bebd0 0x561bb00c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:27:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.472+0000 7fb87edc1740 4 client.15453 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:27:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.472+0000 7fb87edc1740 1 -- 172.21.15.16:0/2242904213 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561baffa8d70 con 0x561bb00bebd0 2024-09-19T09:27:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.472+0000 7fb879ce7640 1 --2- 172.21.15.16:0/2242904213 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x561bb00bebd0 0x561bb00c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:27:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.472+0000 7fb879ce7640 1 --2- 172.21.15.16:0/2242904213 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x561bb00bebd0 0x561bb00c0ff0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74edd57a00fee373 server_cookie=78ad6a9b9fb06920 in_seq=0 out_seq=0 2024-09-19T09:27:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.472+0000 7fb85d7fa640 10 client.15453 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.479+0000 7fb85d7fa640 1 -- 172.21.15.16:0/2242904213 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561baffa8d70 con 0x561bb00bebd0 2024-09-19T09:27:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.479+0000 7fb85d7fa640 10 client.15453 handle_command_reply: tid=0 2024-09-19T09:27:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.479+0000 7fb87edc1740 1 client.15453 shutdown 2024-09-19T09:27:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.479+0000 7fb87edc1740 1 -- 172.21.15.16:0/2242904213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 msgr2=0x561bb004c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.479+0000 7fb87edc1740 1 --2- 172.21.15.16:0/2242904213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 0x561bb004c9e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb864005c20 tx=0x7fb864005bb0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87edc1740 1 -- 172.21.15.16:0/2242904213 shutdown_connections 2024-09-19T09:27:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87edc1740 1 --2- 172.21.15.16:0/2242904213 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x561bb00bebd0 0x561bb00c0ff0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87edc1740 1 --2- 172.21.15.16:0/2242904213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bb003e2e0 0x561bb004cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb85d7fa640 0 client.15453 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:27:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87edc1740 1 --2- 172.21.15.16:0/2242904213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bb0043370 0x561bb004df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87edc1740 1 --2- 172.21.15.16:0/2242904213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561bb003d9a0 0x561bb004c9e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87edc1740 1 -- 172.21.15.16:0/2242904213 >> 172.21.15.16:0/2242904213 conn(0x561bb0039010 msgr2=0x561bb003a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87edc1740 1 -- 172.21.15.16:0/2242904213 shutdown_connections 2024-09-19T09:27:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87edc1740 1 -- 172.21.15.16:0/2242904213 wait complete. 2024-09-19T09:27:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87edc1740 20 client.15453 trim_cache size 0 max 16384 2024-09-19T09:27:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-19T09:27:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-19T09:27:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-19T09:27:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-19T09:27:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87bf4b640 1 -- 172.21.15.16:0/223761473 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb8600a3370 msgr2=0x7fb8600a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87bf4b640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb8600a3370 0x7fb8600a5830 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0985b0 tx=0x7fb86c0bc040 comp rx=0 tx=0).stop 2024-09-19T09:27:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87bf4b640 1 -- 172.21.15.16:0/223761473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8741056b0 msgr2=0x7fb8741a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:27:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87bf4b640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8741056b0 0x7fb8741a5960 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00eaa0 tx=0x7fb86c00ef60 comp rx=0 tx=0).stop 2024-09-19T09:27:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87bf4b640 1 -- 172.21.15.16:0/223761473 shutdown_connections 2024-09-19T09:27:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87bf4b640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb8600a3370 0x7fb8600a5830 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87bf4b640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb874102d00 0x7fb87419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87bf4b640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8741056b0 0x7fb8741a5960 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87bf4b640 1 --2- 172.21.15.16:0/223761473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8741003b0 0x7fb87419e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:27:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.480+0000 7fb87bf4b640 1 -- 172.21.15.16:0/223761473 >> 172.21.15.16:0/223761473 conn(0x7fb8740fc1e0 msgr2=0x7fb874103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:27:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.481+0000 7fb87bf4b640 1 -- 172.21.15.16:0/223761473 shutdown_connections 2024-09-19T09:27:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:27:52.481+0000 7fb87bf4b640 1 -- 172.21.15.16:0/223761473 wait complete. 2024-09-19T09:27:53.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:53 smithi142 ceph-mon[35032]: pgmap v346: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-19T09:27:53.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:53 smithi097 ceph-mon[34864]: pgmap v346: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-19T09:27:53.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:53 smithi016 ceph-mon[30979]: pgmap v346: 97 pgs: 97 active+clean; 658 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-19T09:27:55.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:55 smithi142 ceph-mon[35032]: pgmap v347: 97 pgs: 97 active+clean; 659 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 18 op/s 2024-09-19T09:27:55.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:55 smithi097 ceph-mon[34864]: pgmap v347: 97 pgs: 97 active+clean; 659 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 18 op/s 2024-09-19T09:27:55.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:55 smithi016 ceph-mon[30979]: pgmap v347: 97 pgs: 97 active+clean; 659 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 18 op/s 2024-09-19T09:27:57.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:57 smithi142 ceph-mon[35032]: pgmap v348: 97 pgs: 97 active+clean; 659 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T09:27:57.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:57 smithi097 ceph-mon[34864]: pgmap v348: 97 pgs: 97 active+clean; 659 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T09:27:57.505 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:57 smithi016 ceph-mon[30979]: pgmap v348: 97 pgs: 97 active+clean; 659 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T09:27:59.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:27:59 smithi142 ceph-mon[35032]: pgmap v349: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-19T09:27:59.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:27:59 smithi097 ceph-mon[34864]: pgmap v349: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-19T09:27:59.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:27:59 smithi016 ceph-mon[30979]: pgmap v349: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-19T09:28:00.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:28:01.086 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:01.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:01 smithi016 ceph-mon[30979]: pgmap v350: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T09:28:01.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:01 smithi142 ceph-mon[35032]: pgmap v350: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T09:28:01.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:01 smithi097 ceph-mon[34864]: pgmap v350: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T09:28:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.596+0000 7f2dec4fb640 1 -- 172.21.15.16:0/2502498314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4101410 msgr2=0x7f2de410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.596+0000 7f2dec4fb640 1 --2- 172.21.15.16:0/2502498314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4101410 0x7f2de410d940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009cd0 tx=0x7f2ddc02a8f0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.596+0000 7f2dec4fb640 1 -- 172.21.15.16:0/2502498314 shutdown_connections 2024-09-19T09:28:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.596+0000 7f2dec4fb640 1 --2- 172.21.15.16:0/2502498314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4101410 0x7f2de410d940 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.596+0000 7f2dec4fb640 1 --2- 172.21.15.16:0/2502498314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2de4100a70 0x7f2de4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.596+0000 7f2dec4fb640 1 --2- 172.21.15.16:0/2502498314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2de4106a70 0x7f2de4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.596+0000 7f2dec4fb640 1 -- 172.21.15.16:0/2502498314 >> 172.21.15.16:0/2502498314 conn(0x7f2de40fc790 msgr2=0x7f2de40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.596+0000 7f2dec4fb640 1 -- 172.21.15.16:0/2502498314 shutdown_connections 2024-09-19T09:28:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.596+0000 7f2dec4fb640 1 -- 172.21.15.16:0/2502498314 wait complete. 2024-09-19T09:28:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.597+0000 7f2dec4fb640 1 Processor -- start 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.597+0000 7f2dec4fb640 1 -- start start 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.597+0000 7f2dec4fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100a70 0x7f2de40fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.597+0000 7f2dec4fb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2de4101410 0x7f2de40fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.597+0000 7f2dec4fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2de4106a70 0x7f2de40fd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.597+0000 7f2dec4fb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4110e70 con 0x7f2de4100a70 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.597+0000 7f2dec4fb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4110cf0 con 0x7f2de4101410 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.597+0000 7f2dec4fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4110ff0 con 0x7f2de4106a70 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.597+0000 7f2dea297640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100a70 0x7f2de40fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.597+0000 7f2dea297640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100a70 0x7f2de40fcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42754/0 (socket says 172.21.15.16:42754) 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.597+0000 7f2dea297640 1 -- 172.21.15.16:0/3383750179 learned_addr learned my addr 172.21.15.16:0/3383750179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.598+0000 7f2de9a96640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2de4101410 0x7f2de40fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.598+0000 7f2dea297640 1 -- 172.21.15.16:0/3383750179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2de4106a70 msgr2=0x7f2de40fd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.598+0000 7f2dea297640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2de4106a70 0x7f2de40fd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.598+0000 7f2dea297640 1 -- 172.21.15.16:0/3383750179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2de4101410 msgr2=0x7f2de40fd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.598+0000 7f2dea297640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2de4101410 0x7f2de40fd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.598+0000 7f2dea297640 1 -- 172.21.15.16:0/3383750179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de41a68f0 con 0x7f2de4100a70 2024-09-19T09:28:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.598+0000 7f2de9a96640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2de4101410 0x7f2de40fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.598+0000 7f2dea297640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100a70 0x7f2de40fcd80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2de000c920 tx=0x7f2de000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.598+0000 7f2ddb7fe640 1 -- 172.21.15.16:0/3383750179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0007c70 con 0x7f2de4100a70 2024-09-19T09:28:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.598+0000 7f2ddb7fe640 1 -- 172.21.15.16:0/3383750179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de0007e10 con 0x7f2de4100a70 2024-09-19T09:28:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.598+0000 7f2dec4fb640 1 -- 172.21.15.16:0/3383750179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de41a6be0 con 0x7f2de4100a70 2024-09-19T09:28:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.598+0000 7f2ddb7fe640 1 -- 172.21.15.16:0/3383750179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0018890 con 0x7f2de4100a70 2024-09-19T09:28:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.598+0000 7f2dec4fb640 1 -- 172.21.15.16:0/3383750179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de41a7170 con 0x7f2de4100a70 2024-09-19T09:28:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.600+0000 7f2ddb7fe640 1 -- 172.21.15.16:0/3383750179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f2de0018a70 con 0x7f2de4100a70 2024-09-19T09:28:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.601+0000 7f2dee8d4740 1 Processor -- start 2024-09-19T09:28:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2dee8d4740 1 -- start start 2024-09-19T09:28:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2ddb7fe640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2dd00a32c0 0x7f2dd00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2dee8d4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 0x564986809880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2dee8d4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986809dc0 0x564986814140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2deaa98640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2dd00a32c0 0x7f2dd00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2dee8d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 0x564986816a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2dee8d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56498676f2e0 con 0x564986814680 2024-09-19T09:28:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2dee8d4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564986716ce0 con 0x5649868094a0 2024-09-19T09:28:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2dee8d4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56498676b2a0 con 0x564986809dc0 2024-09-19T09:28:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2deaa98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 0x564986816a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2deaa98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 0x564986816a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42766/0 (socket says 172.21.15.16:42766) 2024-09-19T09:28:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2deaa98640 1 -- 172.21.15.16:0/3087833140 learned_addr learned my addr 172.21.15.16:0/3087833140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2de9a96640 1 --2- 172.21.15.16:0/3087833140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986809dc0 0x564986814140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2deaa98640 1 -- 172.21.15.16:0/3087833140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986809dc0 msgr2=0x564986814140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2deaa98640 1 --2- 172.21.15.16:0/3087833140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986809dc0 0x564986814140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2deaa98640 1 -- 172.21.15.16:0/3087833140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 msgr2=0x564986809880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:28:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2deaa98640 1 --2- 172.21.15.16:0/3087833140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 0x564986809880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2deaa98640 1 -- 172.21.15.16:0/3087833140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649866fb440 con 0x564986814680 2024-09-19T09:28:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.602+0000 7f2ddb7fe640 1 -- 172.21.15.16:0/3383750179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2de00cfa50 con 0x7f2de4100a70 2024-09-19T09:28:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.603+0000 7f2deaa98640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2dd00a32c0 0x7f2dd00a5780 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009430 tx=0x7f2ddc009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.603+0000 7f2deaa98640 1 --2- 172.21.15.16:0/3087833140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 0x564986816a70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc030830 tx=0x7f2ddc046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb99b7edc3950e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.603+0000 7f2dd8ff9640 1 -- 172.21.15.16:0/3087833140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc050070 con 0x564986814680 2024-09-19T09:28:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.603+0000 7f2dd8ff9640 1 -- 172.21.15.16:0/3087833140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc036070 con 0x564986814680 2024-09-19T09:28:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.603+0000 7f2dd8ff9640 1 -- 172.21.15.16:0/3087833140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc031800 con 0x564986814680 2024-09-19T09:28:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 -- 172.21.15.16:0/3087833140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 msgr2=0x564986816a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/3087833140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 0x564986816a70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc030830 tx=0x7f2ddc046000 comp rx=0 tx=0).stop 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 -- 172.21.15.16:0/3087833140 shutdown_connections 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/3087833140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986809dc0 0x564986814140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/3087833140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 0x564986816a70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/3087833140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 0x564986809880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 -- 172.21.15.16:0/3087833140 >> 172.21.15.16:0/3087833140 conn(0x564986805a30 msgr2=0x564986805e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 -- 172.21.15.16:0/3087833140 shutdown_connections 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 -- 172.21.15.16:0/3087833140 wait complete. 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 -- start start 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 Processor -- start 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649868094a0 0x564986814040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 0x5649868112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564986814680 0x5649868118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.604+0000 7f2dee8d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564986817250 con 0x564986809dc0 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2dee8d4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56498672a050 con 0x564986814680 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2dee8d4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5649868173d0 con 0x5649868094a0 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2de9a96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 0x5649868112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2de9a96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 0x5649868112f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42778/0 (socket says 172.21.15.16:42778) 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2de9a96640 1 -- 172.21.15.16:0/2283350989 learned_addr learned my addr 172.21.15.16:0/2283350989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2dee8d4740 10 client.0 _ll_register_callbacks cb 0x56498676a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2dea297640 1 --2- 172.21.15.16:0/2283350989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649868094a0 0x564986814040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2deaa98640 1 --2- 172.21.15.16:0/2283350989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564986814680 0x5649868118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2de9a96640 1 -- 172.21.15.16:0/2283350989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649868094a0 msgr2=0x564986814040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2de9a96640 1 --2- 172.21.15.16:0/2283350989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649868094a0 0x564986814040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2de9a96640 1 -- 172.21.15.16:0/2283350989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564986814680 msgr2=0x5649868118e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2de9a96640 1 --2- 172.21.15.16:0/2283350989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564986814680 0x5649868118e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2de9a96640 1 -- 172.21.15.16:0/2283350989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649864e03c0 con 0x564986809dc0 2024-09-19T09:28:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2de9a96640 1 --2- 172.21.15.16:0/2283350989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 0x5649868112f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009dd0 tx=0x7f2dd40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a43e55fb637dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2dcdffb640 1 -- 172.21.15.16:0/2283350989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4004220 con 0x564986809dc0 2024-09-19T09:28:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2dcdffb640 1 -- 172.21.15.16:0/2283350989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2dd40043c0 con 0x564986809dc0 2024-09-19T09:28:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.605+0000 7f2dee8d4740 1 -- 172.21.15.16:0/2283350989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ddc036070 con 0x564986809dc0 2024-09-19T09:28:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.606+0000 7f2dcdffb640 1 -- 172.21.15.16:0/2283350989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2dd4012660 con 0x564986809dc0 2024-09-19T09:28:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.606+0000 7f2dee8d4740 10 client.15465 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.606+0000 7f2dee8d4740 1 -- 172.21.15.16:0/2283350989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564986822ce0 con 0x564986809dc0 2024-09-19T09:28:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.606+0000 7f2dcdffb640 1 -- 172.21.15.16:0/2283350989 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc036070 con 0x564986809dc0 2024-09-19T09:28:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.606+0000 7f2dcdffb640 1 -- 172.21.15.16:0/2283350989 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2dd400f040 con 0x564986809dc0 2024-09-19T09:28:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.606+0000 7f2dee8d4740 10 client.15465 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.606+0000 7f2dee8d4740 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.606+0000 7f2dee8d4740 20 client.15465 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.606+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/2283350989 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x564986824190 0x5649868265b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.606+0000 7f2dee8d4740 4 client.15465 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.606+0000 7f2dee8d4740 1 -- 172.21.15.16:0/2283350989 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56498677c850 con 0x564986824190 2024-09-19T09:28:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.607+0000 7f2dea297640 1 --2- 172.21.15.16:0/2283350989 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x564986824190 0x5649868265b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.607+0000 7f2dcdffb640 10 client.15465 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.607+0000 7f2dea297640 1 --2- 172.21.15.16:0/2283350989 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x564986824190 0x5649868265b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbfccad33301000 server_cookie=c1e8f200c12f8d09 in_seq=0 out_seq=0 2024-09-19T09:28:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.608+0000 7f2dcdffb640 1 -- 172.21.15.16:0/2283350989 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56498677c850 con 0x564986824190 2024-09-19T09:28:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.608+0000 7f2dcdffb640 10 client.15465 handle_command_reply: tid=0 2024-09-19T09:28:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.608+0000 7f2dee8d4740 1 client.15465 shutdown 2024-09-19T09:28:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.608+0000 7f2dee8d4740 1 -- 172.21.15.16:0/2283350989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 msgr2=0x5649868112f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.608+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/2283350989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 0x5649868112f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009dd0 tx=0x7f2dd40117b0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.608+0000 7f2dee8d4740 1 -- 172.21.15.16:0/2283350989 shutdown_connections 2024-09-19T09:28:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.608+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/2283350989 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x564986824190 0x5649868265b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.608+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/2283350989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564986814680 0x5649868118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.608+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/2283350989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 0x5649868112f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.608+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/2283350989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5649868094a0 0x564986814040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.608+0000 7f2dee8d4740 1 -- 172.21.15.16:0/2283350989 >> 172.21.15.16:0/2283350989 conn(0x564986805a30 msgr2=0x56498680b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.608+0000 7f2dcdffb640 0 client.15465 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.608+0000 7f2dee8d4740 1 -- 172.21.15.16:0/2283350989 shutdown_connections 2024-09-19T09:28:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.609+0000 7f2dee8d4740 1 -- 172.21.15.16:0/2283350989 wait complete. 2024-09-19T09:28:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.609+0000 7f2dee8d4740 20 client.15465 trim_cache size 0 max 16384 2024-09-19T09:28:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.626+0000 7f2dee8d4740 1 Processor -- start 2024-09-19T09:28:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.626+0000 7f2dee8d4740 1 -- start start 2024-09-19T09:28:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.626+0000 7f2dee8d4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 0x5649868204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.626+0000 7f2dee8d4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986809dc0 0x56498680fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.626+0000 7f2dee8d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 0x56498682be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.626+0000 7f2dee8d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41af870 con 0x564986814680 2024-09-19T09:28:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.626+0000 7f2dee8d4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41b13b0 con 0x5649868094a0 2024-09-19T09:28:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.626+0000 7f2dee8d4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41aff40 con 0x564986809dc0 2024-09-19T09:28:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2deaa98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 0x56498682be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2deaa98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 0x56498682be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42790/0 (socket says 172.21.15.16:42790) 2024-09-19T09:28:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2deaa98640 1 -- 172.21.15.16:0/3137209061 learned_addr learned my addr 172.21.15.16:0/3137209061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2dea297640 1 --2- 172.21.15.16:0/3137209061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 0x5649868204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2de9a96640 1 --2- 172.21.15.16:0/3137209061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986809dc0 0x56498680fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2deaa98640 1 -- 172.21.15.16:0/3137209061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986809dc0 msgr2=0x56498680fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2deaa98640 1 --2- 172.21.15.16:0/3137209061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986809dc0 0x56498680fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2deaa98640 1 -- 172.21.15.16:0/3137209061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 msgr2=0x5649868204e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2deaa98640 1 --2- 172.21.15.16:0/3137209061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 0x5649868204e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2deaa98640 1 -- 172.21.15.16:0/3137209061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564986806bc0 con 0x564986814680 2024-09-19T09:28:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2deaa98640 1 --2- 172.21.15.16:0/3137209061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 0x56498682be60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc004990 tx=0x7f2ddc004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c581be5e7e0c9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2dcffff640 1 -- 172.21.15.16:0/3137209061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc059070 con 0x564986814680 2024-09-19T09:28:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2dcffff640 1 -- 172.21.15.16:0/3137209061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc036040 con 0x564986814680 2024-09-19T09:28:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.627+0000 7f2dcffff640 1 -- 172.21.15.16:0/3137209061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc050040 con 0x564986814680 2024-09-19T09:28:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.628+0000 7f2dee8d4740 1 -- 172.21.15.16:0/3137209061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 msgr2=0x56498682be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.628+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/3137209061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 0x56498682be60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc004990 tx=0x7f2ddc004920 comp rx=0 tx=0).stop 2024-09-19T09:28:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.628+0000 7f2dee8d4740 1 -- 172.21.15.16:0/3137209061 shutdown_connections 2024-09-19T09:28:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.628+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/3137209061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986809dc0 0x56498680fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.628+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/3137209061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986814680 0x56498682be60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.628+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/3137209061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 0x5649868204e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.628+0000 7f2dee8d4740 1 -- 172.21.15.16:0/3137209061 >> 172.21.15.16:0/3137209061 conn(0x564986805a30 msgr2=0x56498680c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2dee8d4740 1 -- 172.21.15.16:0/3137209061 shutdown_connections 2024-09-19T09:28:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2dee8d4740 1 -- 172.21.15.16:0/3137209061 wait complete. 2024-09-19T09:28:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2dee8d4740 1 -- start start 2024-09-19T09:28:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2dee8d4740 1 Processor -- start 2024-09-19T09:28:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2dee8d4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 0x564986818550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2dee8d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 0x564986818a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2dee8d4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986814680 0x564986818fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2dee8d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41b0ce0 con 0x564986809dc0 2024-09-19T09:28:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2dee8d4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41b0610 con 0x5649868094a0 2024-09-19T09:28:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2dee8d4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41ad830 con 0x564986814680 2024-09-19T09:28:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2dee8d4740 10 client.0 _ll_register_callbacks cb 0x7f2dd4009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2de9a96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 0x564986818a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2de9a96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 0x564986818a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42792/0 (socket says 172.21.15.16:42792) 2024-09-19T09:28:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2de9a96640 1 -- 172.21.15.16:0/4286383412 learned_addr learned my addr 172.21.15.16:0/4286383412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.629+0000 7f2dea297640 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 0x564986818550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2deaa98640 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986814680 0x564986818fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2de9a96640 1 -- 172.21.15.16:0/4286383412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986814680 msgr2=0x564986818fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2de9a96640 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986814680 0x564986818fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2de9a96640 1 -- 172.21.15.16:0/4286383412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 msgr2=0x564986818550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2de9a96640 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 0x564986818550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2de9a96640 1 -- 172.21.15.16:0/4286383412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56498688dfd0 con 0x564986809dc0 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2deaa98640 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986814680 0x564986818fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2de9a96640 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 0x564986818a90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x564986812930 tx=0x7f2dd4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bc2a6230166c10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2dcdffb640 1 -- 172.21.15.16:0/4286383412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd402a040 con 0x564986809dc0 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2dcdffb640 1 -- 172.21.15.16:0/4286383412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2dd40037f0 con 0x564986809dc0 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2dee8d4740 1 -- 172.21.15.16:0/4286383412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ddc036040 con 0x564986809dc0 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2dcdffb640 1 -- 172.21.15.16:0/4286383412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2dd4011cb0 con 0x564986809dc0 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2dee8d4740 10 client.15471 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.630+0000 7f2dee8d4740 1 -- 172.21.15.16:0/4286383412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564986897b00 con 0x564986809dc0 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.631+0000 7f2dcdffb640 1 -- 172.21.15.16:0/4286383412 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc036040 con 0x564986809dc0 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.631+0000 7f2dcdffb640 1 -- 172.21.15.16:0/4286383412 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2dd4037050 con 0x564986809dc0 2024-09-19T09:28:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.631+0000 7f2dee8d4740 10 client.15471 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.631+0000 7f2dee8d4740 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.631+0000 7f2dee8d4740 20 client.15471 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.631+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5649868982b0 0x56498689a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.631+0000 7f2dee8d4740 4 client.15471 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.631+0000 7f2dee8d4740 1 -- 172.21.15.16:0/4286383412 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564986776d60 con 0x5649868982b0 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.631+0000 7f2dea297640 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5649868982b0 0x56498689a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dcdffb640 10 client.15471 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dea297640 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5649868982b0 0x56498689a6d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2362176f6c8f983 server_cookie=45844f104cb55887 in_seq=0 out_seq=0 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dcdffb640 1 -- 172.21.15.16:0/4286383412 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564986776d60 con 0x5649868982b0 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dcdffb640 10 client.15471 handle_command_reply: tid=0 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dee8d4740 1 client.15471 shutdown 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dee8d4740 1 -- 172.21.15.16:0/4286383412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 msgr2=0x564986818a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 0x564986818a90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x564986812930 tx=0x7f2dd4005dc0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dee8d4740 1 -- 172.21.15.16:0/4286383412 shutdown_connections 2024-09-19T09:28:01.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5649868982b0 0x56498689a6d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dcdffb640 0 client.15471 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:01.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564986814680 0x564986818fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564986809dc0 0x564986818a90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dee8d4740 1 --2- 172.21.15.16:0/4286383412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649868094a0 0x564986818550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dee8d4740 1 -- 172.21.15.16:0/4286383412 >> 172.21.15.16:0/4286383412 conn(0x564986805a30 msgr2=0x564986804350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:01.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dee8d4740 1 -- 172.21.15.16:0/4286383412 shutdown_connections 2024-09-19T09:28:01.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dee8d4740 1 -- 172.21.15.16:0/4286383412 wait complete. 2024-09-19T09:28:01.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.632+0000 7f2dee8d4740 20 client.15471 trim_cache size 0 max 16384 2024-09-19T09:28:01.639 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:28:01.640 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-19T09:28:01.640 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-19T09:28:01.640 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:28:01.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.633+0000 7f2dec4fb640 1 -- 172.21.15.16:0/3383750179 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2dd00a32c0 msgr2=0x7f2dd00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.633+0000 7f2dec4fb640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2dd00a32c0 0x7f2dd00a5780 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009430 tx=0x7f2ddc009800 comp rx=0 tx=0).stop 2024-09-19T09:28:01.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.633+0000 7f2dec4fb640 1 -- 172.21.15.16:0/3383750179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100a70 msgr2=0x7f2de40fcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:01.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.633+0000 7f2dec4fb640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100a70 0x7f2de40fcd80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2de000c920 tx=0x7f2de000cde0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.633+0000 7f2dec4fb640 1 -- 172.21.15.16:0/3383750179 shutdown_connections 2024-09-19T09:28:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.633+0000 7f2dec4fb640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2dd00a32c0 0x7f2dd00a5780 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.633+0000 7f2dec4fb640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2de4106a70 0x7f2de40fd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.633+0000 7f2dec4fb640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2de4101410 0x7f2de40fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.633+0000 7f2dec4fb640 1 --2- 172.21.15.16:0/3383750179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100a70 0x7f2de40fcd80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.633+0000 7f2dec4fb640 1 -- 172.21.15.16:0/3383750179 >> 172.21.15.16:0/3383750179 conn(0x7f2de40fc790 msgr2=0x7f2de406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.633+0000 7f2dec4fb640 1 -- 172.21.15.16:0/3383750179 shutdown_connections 2024-09-19T09:28:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:01.633+0000 7f2dec4fb640 1 -- 172.21.15.16:0/3383750179 wait complete. 2024-09-19T09:28:03.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:03 smithi142 ceph-mon[35032]: pgmap v351: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T09:28:03.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:03 smithi097 ceph-mon[34864]: pgmap v351: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T09:28:03.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:03 smithi016 ceph-mon[30979]: pgmap v351: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-19T09:28:04.194 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.2 2024-09-19T09:28:04.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:28:04.871 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:28:04.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:28:05.008 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:05.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:05 smithi016 ceph-mon[30979]: pgmap v352: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-09-19T09:28:05.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:05 smithi142 ceph-mon[35032]: pgmap v352: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-09-19T09:28:05.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:05 smithi097 ceph-mon[34864]: pgmap v352: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-09-19T09:28:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.573+0000 7fd7bf85e640 1 -- 172.21.15.16:0/1259496326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8102d50 msgr2=0x7fd7b81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.573+0000 7fd7bf85e640 1 --2- 172.21.15.16:0/1259496326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8102d50 0x7fd7b81031d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009900 tx=0x7fd7a802a410 comp rx=0 tx=0).stop 2024-09-19T09:28:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.573+0000 7fd7bf85e640 1 -- 172.21.15.16:0/1259496326 shutdown_connections 2024-09-19T09:28:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.573+0000 7fd7bf85e640 1 --2- 172.21.15.16:0/1259496326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7b8103710 0x7fd7b8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.573+0000 7fd7bf85e640 1 --2- 172.21.15.16:0/1259496326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8102d50 0x7fd7b81031d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.573+0000 7fd7bf85e640 1 --2- 172.21.15.16:0/1259496326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b8101b60 0x7fd7b8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.573+0000 7fd7bf85e640 1 -- 172.21.15.16:0/1259496326 >> 172.21.15.16:0/1259496326 conn(0x7fd7b80fd2e0 msgr2=0x7fd7b80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.573+0000 7fd7bf85e640 1 -- 172.21.15.16:0/1259496326 shutdown_connections 2024-09-19T09:28:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.573+0000 7fd7bf85e640 1 -- 172.21.15.16:0/1259496326 wait complete. 2024-09-19T09:28:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.574+0000 7fd7bf85e640 1 Processor -- start 2024-09-19T09:28:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.574+0000 7fd7bf85e640 1 -- start start 2024-09-19T09:28:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.574+0000 7fd7bf85e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b8101b60 0x7fd7b8197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.574+0000 7fd7bf85e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8102d50 0x7fd7b8198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.574+0000 7fd7bf85e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7b8103710 0x7fd7b819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.574+0000 7fd7bf85e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b810d7c0 con 0x7fd7b8102d50 2024-09-19T09:28:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.574+0000 7fd7bf85e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b810d640 con 0x7fd7b8101b60 2024-09-19T09:28:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.574+0000 7fd7bf85e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b810d940 con 0x7fd7b8103710 2024-09-19T09:28:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.574+0000 7fd7bcdf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8102d50 0x7fd7b8198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.574+0000 7fd7bcdf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8102d50 0x7fd7b8198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59894/0 (socket says 172.21.15.16:59894) 2024-09-19T09:28:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.574+0000 7fd7bcdf9640 1 -- 172.21.15.16:0/1149967119 learned_addr learned my addr 172.21.15.16:0/1149967119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.574+0000 7fd7bd5fa640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b8101b60 0x7fd7b8197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.574+0000 7fd7bddfb640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7b8103710 0x7fd7b819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.575+0000 7fd7bcdf9640 1 -- 172.21.15.16:0/1149967119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7b8103710 msgr2=0x7fd7b819f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.575+0000 7fd7bcdf9640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7b8103710 0x7fd7b819f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.575+0000 7fd7bcdf9640 1 -- 172.21.15.16:0/1149967119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b8101b60 msgr2=0x7fd7b8197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.575+0000 7fd7bcdf9640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b8101b60 0x7fd7b8197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.575+0000 7fd7bcdf9640 1 -- 172.21.15.16:0/1149967119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b819f970 con 0x7fd7b8102d50 2024-09-19T09:28:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.575+0000 7fd7bcdf9640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8102d50 0x7fd7b8198290 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009230 tx=0x7fd7a8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.575+0000 7fd7ae7fc640 1 -- 172.21.15.16:0/1149967119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a803f030 con 0x7fd7b8102d50 2024-09-19T09:28:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.575+0000 7fd7bf85e640 1 -- 172.21.15.16:0/1149967119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b819fc00 con 0x7fd7b8102d50 2024-09-19T09:28:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.575+0000 7fd7ae7fc640 1 -- 172.21.15.16:0/1149967119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7a8003ce0 con 0x7fd7b8102d50 2024-09-19T09:28:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.575+0000 7fd7ae7fc640 1 -- 172.21.15.16:0/1149967119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8035070 con 0x7fd7b8102d50 2024-09-19T09:28:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.575+0000 7fd7bf85e640 1 -- 172.21.15.16:0/1149967119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b80746c0 con 0x7fd7b8102d50 2024-09-19T09:28:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.578+0000 7fd7c25fa740 1 Processor -- start 2024-09-19T09:28:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.578+0000 7fd7c25fa740 1 -- start start 2024-09-19T09:28:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.578+0000 7fd7ae7fc640 1 -- 172.21.15.16:0/1149967119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd7a8042030 con 0x7fd7b8102d50 2024-09-19T09:28:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.578+0000 7fd7c25fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a5cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.578+0000 7fd7c25fa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a5d2e0 0x55b161a61da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.578+0000 7fd7c25fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a62370 0x55b161a64830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.578+0000 7fd7c25fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b1619c02f0 con 0x55b161a5c9a0 2024-09-19T09:28:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.578+0000 7fd7c25fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b161968270 con 0x55b161a62370 2024-09-19T09:28:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.578+0000 7fd7c25fa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b1619bc2b0 con 0x55b161a5d2e0 2024-09-19T09:28:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.578+0000 7fd7bddfb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a5cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7bddfb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a5cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59898/0 (socket says 172.21.15.16:59898) 2024-09-19T09:28:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7bddfb640 1 -- 172.21.15.16:0/28016404 learned_addr learned my addr 172.21.15.16:0/28016404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7bcdf9640 1 --2- 172.21.15.16:0/28016404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a62370 0x55b161a64830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7bd5fa640 1 --2- 172.21.15.16:0/28016404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a5d2e0 0x55b161a61da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7bddfb640 1 -- 172.21.15.16:0/28016404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a5d2e0 msgr2=0x55b161a61da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7bddfb640 1 --2- 172.21.15.16:0/28016404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a5d2e0 0x55b161a61da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7bddfb640 1 -- 172.21.15.16:0/28016404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a62370 msgr2=0x55b161a64830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7bddfb640 1 --2- 172.21.15.16:0/28016404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a62370 0x55b161a64830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7bddfb640 1 -- 172.21.15.16:0/28016404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b161967340 con 0x55b161a5c9a0 2024-09-19T09:28:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7bcdf9640 1 --2- 172.21.15.16:0/28016404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a62370 0x55b161a64830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7bd5fa640 1 --2- 172.21.15.16:0/28016404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a5d2e0 0x55b161a61da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7ae7fc640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd7a40a32f0 0x7fd7a40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7bddfb640 1 --2- 172.21.15.16:0/28016404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a5cda0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0007e50 tx=0x7fd7b000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a783ca96dcc64c05 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7bd5fa640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd7a40a32f0 0x7fd7a40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.579+0000 7fd7ae7fc640 1 -- 172.21.15.16:0/1149967119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd7a80f02f0 con 0x7fd7b8102d50 2024-09-19T09:28:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.580+0000 7fd7a37fe640 1 -- 172.21.15.16:0/28016404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0014070 con 0x55b161a5c9a0 2024-09-19T09:28:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.580+0000 7fd7a37fe640 1 -- 172.21.15.16:0/28016404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7b000ee90 con 0x55b161a5c9a0 2024-09-19T09:28:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.580+0000 7fd7a37fe640 1 -- 172.21.15.16:0/28016404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0005410 con 0x55b161a5c9a0 2024-09-19T09:28:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.581+0000 7fd7c25fa740 1 -- 172.21.15.16:0/28016404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 msgr2=0x55b161a5cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.581+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/28016404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a5cda0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0007e50 tx=0x7fd7b000e720 comp rx=0 tx=0).stop 2024-09-19T09:28:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.581+0000 7fd7bd5fa640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd7a40a32f0 0x7fd7a40a57b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55b161a5dd10 tx=0x7fd7b400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.581+0000 7fd7c25fa740 1 -- 172.21.15.16:0/28016404 shutdown_connections 2024-09-19T09:28:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.581+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/28016404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a5d2e0 0x55b161a61da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.581+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/28016404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a62370 0x55b161a64830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.581+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/28016404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a5cda0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.581+0000 7fd7c25fa740 1 -- 172.21.15.16:0/28016404 >> 172.21.15.16:0/28016404 conn(0x55b161a58010 msgr2=0x55b161a58410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.581+0000 7fd7c25fa740 1 -- 172.21.15.16:0/28016404 shutdown_connections 2024-09-19T09:28:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.581+0000 7fd7c25fa740 1 -- 172.21.15.16:0/28016404 wait complete. 2024-09-19T09:28:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.581+0000 7fd7c25fa740 1 -- start start 2024-09-19T09:28:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.581+0000 7fd7c25fa740 1 Processor -- start 2024-09-19T09:28:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7c25fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a66c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7c25fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a5d2e0 0x55b161a671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7c25fa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a62370 0x55b161a6e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7c25fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b161a65340 con 0x55b161a5c9a0 2024-09-19T09:28:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7c25fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b161949290 con 0x55b161a5d2e0 2024-09-19T09:28:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7c25fa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b161a654c0 con 0x55b161a62370 2024-09-19T09:28:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7c25fa740 10 client.0 _ll_register_callbacks cb 0x55b16197a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7bd5fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a66c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7bcdf9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a5d2e0 0x55b161a671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7bddfb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a62370 0x55b161a6e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7bd5fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a66c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59904/0 (socket says 172.21.15.16:59904) 2024-09-19T09:28:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7bd5fa640 1 -- 172.21.15.16:0/4219160745 learned_addr learned my addr 172.21.15.16:0/4219160745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7bddfb640 1 -- 172.21.15.16:0/4219160745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a5d2e0 msgr2=0x55b161a671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7bddfb640 1 --2- 172.21.15.16:0/4219160745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a5d2e0 0x55b161a671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7bddfb640 1 -- 172.21.15.16:0/4219160745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 msgr2=0x55b161a66c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7bddfb640 1 --2- 172.21.15.16:0/4219160745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a66c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7bddfb640 1 -- 172.21.15.16:0/4219160745 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1616d3210 con 0x55b161a62370 2024-09-19T09:28:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7bd5fa640 1 --2- 172.21.15.16:0/4219160745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a66c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.582+0000 7fd7bcdf9640 1 --2- 172.21.15.16:0/4219160745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a5d2e0 0x55b161a671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.583+0000 7fd7bddfb640 1 --2- 172.21.15.16:0/4219160745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a62370 0x55b161a6e200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0000f80 tx=0x7fd7b0007b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3caa67e8bca97dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.583+0000 7fd7a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-19T09:28:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.583+0000 7fd7a0ff9640 1 -- 172.21.15.16:0/4219160745 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0014070 con 0x55b161a62370 2024-09-19T09:28:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.583+0000 7fd7c25fa740 1 -- 172.21.15.16:0/4219160745 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7b000ee90 con 0x55b161a62370 2024-09-19T09:28:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.583+0000 7fd7a0ff9640 1 -- 172.21.15.16:0/4219160745 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd7b0003c00 con 0x55b161a62370 2024-09-19T09:28:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.583+0000 7fd7a0ff9640 1 -- 172.21.15.16:0/4219160745 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd7b0003e00 con 0x55b161a62370 2024-09-19T09:28:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.583+0000 7fd7c25fa740 10 client.25072 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.583+0000 7fd7c25fa740 1 -- 172.21.15.16:0/4219160745 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b16179f620 con 0x55b161a62370 2024-09-19T09:28:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.583+0000 7fd7a0ff9640 1 -- 172.21.15.16:0/4219160745 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7b000ee90 con 0x55b161a62370 2024-09-19T09:28:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.583+0000 7fd7a0ff9640 1 -- 172.21.15.16:0/4219160745 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd7b0004600 con 0x55b161a62370 2024-09-19T09:28:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.584+0000 7fd7c25fa740 10 client.25072 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.584+0000 7fd7c25fa740 10 client.25072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.584+0000 7fd7c25fa740 20 client.25072 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.584+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/4219160745 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55b161a726b0 0x55b161a74aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.584+0000 7fd7c25fa740 4 client.25072 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.584+0000 7fd7c25fa740 1 -- 172.21.15.16:0/4219160745 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1619cda20 con 0x55b161a726b0 2024-09-19T09:28:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.584+0000 7fd7bd5fa640 1 --2- 172.21.15.16:0/4219160745 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55b161a726b0 0x55b161a74aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.584+0000 7fd7a0ff9640 10 client.25072 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.584+0000 7fd7bd5fa640 1 --2- 172.21.15.16:0/4219160745 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55b161a726b0 0x55b161a74aa0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a3e2b45a61e1347 server_cookie=ccac1a923b5bc6a2 in_seq=0 out_seq=0 2024-09-19T09:28:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7a0ff9640 1 -- 172.21.15.16:0/4219160745 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1619cda20 con 0x55b161a726b0 2024-09-19T09:28:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7a0ff9640 10 client.25072 handle_command_reply: tid=0 2024-09-19T09:28:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7c25fa740 1 client.25072 shutdown 2024-09-19T09:28:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7c25fa740 1 -- 172.21.15.16:0/4219160745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a62370 msgr2=0x55b161a6e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/4219160745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a62370 0x55b161a6e200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0000f80 tx=0x7fd7b0007b90 comp rx=0 tx=0).stop 2024-09-19T09:28:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7c25fa740 1 -- 172.21.15.16:0/4219160745 shutdown_connections 2024-09-19T09:28:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/4219160745 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55b161a726b0 0x55b161a74aa0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/4219160745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a62370 0x55b161a6e200 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/4219160745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a5d2e0 0x55b161a671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/4219160745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a66c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7a0ff9640 0 client.25072 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7c25fa740 1 -- 172.21.15.16:0/4219160745 >> 172.21.15.16:0/4219160745 conn(0x55b161a58010 msgr2=0x55b161a59820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7c25fa740 1 -- 172.21.15.16:0/4219160745 shutdown_connections 2024-09-19T09:28:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.585+0000 7fd7c25fa740 1 -- 172.21.15.16:0/4219160745 wait complete. 2024-09-19T09:28:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.586+0000 7fd7c25fa740 20 client.25072 trim_cache size 0 max 16384 2024-09-19T09:28:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.603+0000 7fd7c25fa740 1 Processor -- start 2024-09-19T09:28:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.603+0000 7fd7c25fa740 1 -- start start 2024-09-19T09:28:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.603+0000 7fd7c25fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a5cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.603+0000 7fd7c25fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a5d2e0 0x55b161a6e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.603+0000 7fd7c25fa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a62370 0x55b161a570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.603+0000 7fd7c25fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b81ab220 con 0x55b161a5c9a0 2024-09-19T09:28:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.603+0000 7fd7c25fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b81acd60 con 0x55b161a5d2e0 2024-09-19T09:28:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.603+0000 7fd7c25fa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b81ab8f0 con 0x55b161a62370 2024-09-19T09:28:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.603+0000 7fd7bd5fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a5cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.603+0000 7fd7bd5fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a5cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59918/0 (socket says 172.21.15.16:59918) 2024-09-19T09:28:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.603+0000 7fd7bd5fa640 1 -- 172.21.15.16:0/206029881 learned_addr learned my addr 172.21.15.16:0/206029881 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.603+0000 7fd7bcdf9640 1 --2- 172.21.15.16:0/206029881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a5d2e0 0x55b161a6e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.603+0000 7fd7bddfb640 1 --2- 172.21.15.16:0/206029881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a62370 0x55b161a570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.604+0000 7fd7bcdf9640 1 -- 172.21.15.16:0/206029881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a62370 msgr2=0x55b161a570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.604+0000 7fd7bcdf9640 1 --2- 172.21.15.16:0/206029881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a62370 0x55b161a570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.604+0000 7fd7bcdf9640 1 -- 172.21.15.16:0/206029881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 msgr2=0x55b161a5cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.604+0000 7fd7bcdf9640 1 --2- 172.21.15.16:0/206029881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a5cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.604+0000 7fd7bcdf9640 1 -- 172.21.15.16:0/206029881 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b161a74e10 con 0x55b161a5d2e0 2024-09-19T09:28:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.604+0000 7fd7bd5fa640 1 --2- 172.21.15.16:0/206029881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a5cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.604+0000 7fd7bcdf9640 1 --2- 172.21.15.16:0/206029881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a5d2e0 0x55b161a6e120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80bc610 tx=0x7fd7a80f3d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8798e593efaf6f6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.604+0000 7fd7a2ffd640 1 -- 172.21.15.16:0/206029881 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8049030 con 0x55b161a5d2e0 2024-09-19T09:28:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.604+0000 7fd7a2ffd640 1 -- 172.21.15.16:0/206029881 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7a80bd010 con 0x55b161a5d2e0 2024-09-19T09:28:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.604+0000 7fd7a2ffd640 1 -- 172.21.15.16:0/206029881 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80bac10 con 0x55b161a5d2e0 2024-09-19T09:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.605+0000 7fd7c25fa740 1 -- 172.21.15.16:0/206029881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a5d2e0 msgr2=0x55b161a6e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.605+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/206029881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a5d2e0 0x55b161a6e120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80bc610 tx=0x7fd7a80f3d70 comp rx=0 tx=0).stop 2024-09-19T09:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.605+0000 7fd7c25fa740 1 -- 172.21.15.16:0/206029881 shutdown_connections 2024-09-19T09:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.605+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/206029881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a62370 0x55b161a570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.605+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/206029881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a5d2e0 0x55b161a6e120 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.605+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/206029881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a5cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.605+0000 7fd7c25fa740 1 -- 172.21.15.16:0/206029881 >> 172.21.15.16:0/206029881 conn(0x55b161a58010 msgr2=0x55b161a569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.605+0000 7fd7c25fa740 1 -- 172.21.15.16:0/206029881 shutdown_connections 2024-09-19T09:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.605+0000 7fd7c25fa740 1 -- 172.21.15.16:0/206029881 wait complete. 2024-09-19T09:28:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.606+0000 7fd7c25fa740 1 -- start start 2024-09-19T09:28:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.606+0000 7fd7c25fa740 1 Processor -- start 2024-09-19T09:28:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.606+0000 7fd7c25fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a6b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.606+0000 7fd7c25fa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a5d2e0 0x55b161a6b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.606+0000 7fd7c25fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a62370 0x55b161a6c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7bd5fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a6b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7bd5fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a6b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59926/0 (socket says 172.21.15.16:59926) 2024-09-19T09:28:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7bd5fa640 1 -- 172.21.15.16:0/1796977260 learned_addr learned my addr 172.21.15.16:0/1796977260 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.606+0000 7fd7c25fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b81ac690 con 0x55b161a5c9a0 2024-09-19T09:28:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7c25fa740 1 -- 172.21.15.16:0/1796977260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b81abfc0 con 0x55b161a62370 2024-09-19T09:28:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7c25fa740 1 -- 172.21.15.16:0/1796977260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b81a91e0 con 0x55b161a5d2e0 2024-09-19T09:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7c25fa740 10 client.0 _ll_register_callbacks cb 0x7fd7a80f3c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7bcdf9640 1 --2- 172.21.15.16:0/1796977260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a5d2e0 0x55b161a6b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7bddfb640 1 --2- 172.21.15.16:0/1796977260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a62370 0x55b161a6c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7bd5fa640 1 -- 172.21.15.16:0/1796977260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a5d2e0 msgr2=0x55b161a6b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7bd5fa640 1 --2- 172.21.15.16:0/1796977260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a5d2e0 0x55b161a6b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7bd5fa640 1 -- 172.21.15.16:0/1796977260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a62370 msgr2=0x55b161a6c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7bd5fa640 1 --2- 172.21.15.16:0/1796977260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a62370 0x55b161a6c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7bd5fa640 1 -- 172.21.15.16:0/1796977260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b161a6ccf0 con 0x55b161a5c9a0 2024-09-19T09:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7bd5fa640 1 --2- 172.21.15.16:0/1796977260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a6b190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd7b401e730 tx=0x7fd7b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2cc928e3e5f7b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7a0ff9640 1 -- 172.21.15.16:0/1796977260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b401ed60 con 0x55b161a5c9a0 2024-09-19T09:28:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7a0ff9640 1 -- 172.21.15.16:0/1796977260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fd7b402a380 con 0x55b161a5c9a0 2024-09-19T09:28:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.607+0000 7fd7c25fa740 1 -- 172.21.15.16:0/1796977260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7a80bd010 con 0x55b161a5c9a0 2024-09-19T09:28:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.608+0000 7fd7a0ff9640 1 -- 172.21.15.16:0/1796977260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd7b402a580 con 0x55b161a5c9a0 2024-09-19T09:28:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.608+0000 7fd7c25fa740 10 client.15489 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.608+0000 7fd7c25fa740 1 -- 172.21.15.16:0/1796977260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b161a6e0b0 con 0x55b161a5c9a0 2024-09-19T09:28:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.608+0000 7fd7a0ff9640 1 -- 172.21.15.16:0/1796977260 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7a80bd010 con 0x55b161a5c9a0 2024-09-19T09:28:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.608+0000 7fd7a0ff9640 1 -- 172.21.15.16:0/1796977260 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd7b4032920 con 0x55b161a5c9a0 2024-09-19T09:28:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.608+0000 7fd7c25fa740 10 client.15489 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.608+0000 7fd7c25fa740 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.608+0000 7fd7c25fa740 20 client.15489 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.608+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/1796977260 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55b161add770 0x55b161adfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.608+0000 7fd7c25fa740 4 client.15489 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.608+0000 7fd7c25fa740 1 -- 172.21.15.16:0/1796977260 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b1619c7d70 con 0x55b161add770 2024-09-19T09:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.608+0000 7fd7bcdf9640 1 --2- 172.21.15.16:0/1796977260 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55b161add770 0x55b161adfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.609+0000 7fd7a0ff9640 10 client.15489 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.609+0000 7fd7bcdf9640 1 --2- 172.21.15.16:0/1796977260 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55b161add770 0x55b161adfb90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b434167ebe9443bf server_cookie=dc57c45a9adcc5d0 in_seq=0 out_seq=0 2024-09-19T09:28:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.612+0000 7fd7a0ff9640 1 -- 172.21.15.16:0/1796977260 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b1619c7d70 con 0x55b161add770 2024-09-19T09:28:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.612+0000 7fd7a0ff9640 10 client.15489 handle_command_reply: tid=0 2024-09-19T09:28:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.612+0000 7fd7c25fa740 1 client.15489 shutdown 2024-09-19T09:28:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7c25fa740 1 -- 172.21.15.16:0/1796977260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 msgr2=0x55b161a6b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/1796977260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a6b190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd7b401e730 tx=0x7fd7b40046c0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7c25fa740 1 -- 172.21.15.16:0/1796977260 shutdown_connections 2024-09-19T09:28:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/1796977260 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55b161add770 0x55b161adfb90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/1796977260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b161a5d2e0 0x55b161a6b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/1796977260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b161a62370 0x55b161a6c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7a0ff9640 0 client.15489 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7c25fa740 1 --2- 172.21.15.16:0/1796977260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b161a5c9a0 0x55b161a6b190 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7c25fa740 1 -- 172.21.15.16:0/1796977260 >> 172.21.15.16:0/1796977260 conn(0x55b161a58010 msgr2=0x55b161a569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7c25fa740 1 -- 172.21.15.16:0/1796977260 shutdown_connections 2024-09-19T09:28:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7c25fa740 1 -- 172.21.15.16:0/1796977260 wait complete. 2024-09-19T09:28:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7c25fa740 20 client.15489 trim_cache size 0 max 16384 2024-09-19T09:28:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-19T09:28:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-19T09:28:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-19T09:28:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-19T09:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7bf85e640 1 -- 172.21.15.16:0/1149967119 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd7a40a32f0 msgr2=0x7fd7a40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7bf85e640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd7a40a32f0 0x7fd7a40a57b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55b161a5dd10 tx=0x7fd7b400b390 comp rx=0 tx=0).stop 2024-09-19T09:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7bf85e640 1 -- 172.21.15.16:0/1149967119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8102d50 msgr2=0x7fd7b8198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.613+0000 7fd7bf85e640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8102d50 0x7fd7b8198290 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009230 tx=0x7fd7a8004360 comp rx=0 tx=0).stop 2024-09-19T09:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.614+0000 7fd7bf85e640 1 -- 172.21.15.16:0/1149967119 shutdown_connections 2024-09-19T09:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.614+0000 7fd7bf85e640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fd7a40a32f0 0x7fd7a40a57b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.614+0000 7fd7bf85e640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7b8103710 0x7fd7b819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.614+0000 7fd7bf85e640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8102d50 0x7fd7b8198290 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.614+0000 7fd7bf85e640 1 --2- 172.21.15.16:0/1149967119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b8101b60 0x7fd7b8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.614+0000 7fd7bf85e640 1 -- 172.21.15.16:0/1149967119 >> 172.21.15.16:0/1149967119 conn(0x7fd7b80fd2e0 msgr2=0x7fd7b8106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.614+0000 7fd7bf85e640 1 -- 172.21.15.16:0/1149967119 shutdown_connections 2024-09-19T09:28:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:05.614+0000 7fd7bf85e640 1 -- 172.21.15.16:0/1149967119 wait complete. 2024-09-19T09:28:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.655+0000 7f5d35cfd640 1 -- 172.21.15.16:0/17689852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d30108290 msgr2=0x7f5d3010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.655+0000 7f5d35cfd640 1 --2- 172.21.15.16:0/17689852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d30108290 0x7f5d3010ec30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009cd0 tx=0x7f5d2802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.655+0000 7f5d35cfd640 1 -- 172.21.15.16:0/17689852 shutdown_connections 2024-09-19T09:28:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.655+0000 7f5d35cfd640 1 --2- 172.21.15.16:0/17689852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d30108290 0x7f5d3010ec30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.655+0000 7f5d35cfd640 1 --2- 172.21.15.16:0/17689852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d30103740 0x7f5d30107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.655+0000 7f5d35cfd640 1 --2- 172.21.15.16:0/17689852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d30102d90 0x7f5d30103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.655+0000 7f5d35cfd640 1 -- 172.21.15.16:0/17689852 >> 172.21.15.16:0/17689852 conn(0x7f5d300fc7d0 msgr2=0x7f5d300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.655+0000 7f5d35cfd640 1 -- 172.21.15.16:0/17689852 shutdown_connections 2024-09-19T09:28:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.655+0000 7f5d35cfd640 1 -- 172.21.15.16:0/17689852 wait complete. 2024-09-19T09:28:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.655+0000 7f5d35cfd640 1 Processor -- start 2024-09-19T09:28:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.656+0000 7f5d35cfd640 1 -- start start 2024-09-19T09:28:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.656+0000 7f5d35cfd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d30102d90 0x7f5d301009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.656+0000 7f5d35cfd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d30103740 0x7f5d30100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.656+0000 7f5d35cfd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d30108290 0x7f5d30101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.656+0000 7f5d35cfd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30112090 con 0x7f5d30108290 2024-09-19T09:28:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.656+0000 7f5d35cfd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30111f10 con 0x7f5d30102d90 2024-09-19T09:28:05.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.656+0000 7f5d35cfd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30112210 con 0x7f5d30103740 2024-09-19T09:28:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.656+0000 7f5d2ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d30108290 0x7f5d30101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.656+0000 7f5d2f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d30102d90 0x7f5d301009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.656+0000 7f5d2ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d30108290 0x7f5d30101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59940/0 (socket says 172.21.15.16:59940) 2024-09-19T09:28:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.656+0000 7f5d2ffff640 1 -- 172.21.15.16:0/1163970172 learned_addr learned my addr 172.21.15.16:0/1163970172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.656+0000 7f5d2effd640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d30103740 0x7f5d30100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.657+0000 7f5d2f7fe640 1 -- 172.21.15.16:0/1163970172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d30103740 msgr2=0x7f5d30100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.657+0000 7f5d2f7fe640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d30103740 0x7f5d30100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.657+0000 7f5d2f7fe640 1 -- 172.21.15.16:0/1163970172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d30108290 msgr2=0x7f5d30101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.657+0000 7f5d2f7fe640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d30108290 0x7f5d30101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.657+0000 7f5d2f7fe640 1 -- 172.21.15.16:0/1163970172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d301a6a10 con 0x7f5d30102d90 2024-09-19T09:28:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.657+0000 7f5d2effd640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d30103740 0x7f5d30100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.657+0000 7f5d2f7fe640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d30102d90 0x7f5d301009f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000e9f0 tx=0x7f5d2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.657+0000 7f5d2cff9640 1 -- 172.21.15.16:0/1163970172 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2000ccc0 con 0x7f5d30102d90 2024-09-19T09:28:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.657+0000 7f5d35cfd640 1 -- 172.21.15.16:0/1163970172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d301a6d00 con 0x7f5d30102d90 2024-09-19T09:28:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.657+0000 7f5d2cff9640 1 -- 172.21.15.16:0/1163970172 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d20010910 con 0x7f5d30102d90 2024-09-19T09:28:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.657+0000 7f5d2cff9640 1 -- 172.21.15.16:0/1163970172 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20018960 con 0x7f5d30102d90 2024-09-19T09:28:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.657+0000 7f5d35cfd640 1 -- 172.21.15.16:0/1163970172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d301a71e0 con 0x7f5d30102d90 2024-09-19T09:28:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.658+0000 7f5d35cfd640 1 -- 172.21.15.16:0/1163970172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d30103170 con 0x7f5d30102d90 2024-09-19T09:28:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.660+0000 7f5d2cff9640 1 -- 172.21.15.16:0/1163970172 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f5d20010ac0 con 0x7f5d30102d90 2024-09-19T09:28:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.661+0000 7f5d2cff9640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5d180a3240 0x7f5d180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.661+0000 7f5d2effd640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5d180a3240 0x7f5d180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.661+0000 7f5d2cff9640 1 -- 172.21.15.16:0/1163970172 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f5d200d0280 con 0x7f5d30102d90 2024-09-19T09:28:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.661+0000 7f5d2effd640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5d180a3240 0x7f5d180a5700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5d24005fd0 tx=0x7f5d24005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.664+0000 7f5d2cff9640 1 -- 172.21.15.16:0/1163970172 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d2009cfa0 con 0x7f5d30102d90 2024-09-19T09:28:05.822 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:05.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.921+0000 7f5d35cfd640 1 -- 172.21.15.16:0/1163970172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d30101f20 con 0x7f5d30102d90 2024-09-19T09:28:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.922+0000 7f5d2cff9640 1 -- 172.21.15.16:0/1163970172 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f5d2009c8a0 con 0x7f5d30102d90 2024-09-19T09:28:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:28:05.925 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:28:05.925 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:28:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.924+0000 7f5d35cfd640 1 -- 172.21.15.16:0/1163970172 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5d180a3240 msgr2=0x7f5d180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.924+0000 7f5d35cfd640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5d180a3240 0x7f5d180a5700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5d24005fd0 tx=0x7f5d24005850 comp rx=0 tx=0).stop 2024-09-19T09:28:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.925+0000 7f5d35cfd640 1 -- 172.21.15.16:0/1163970172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d30102d90 msgr2=0x7f5d301009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.925+0000 7f5d35cfd640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d30102d90 0x7f5d301009f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000e9f0 tx=0x7f5d2000eeb0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.925+0000 7f5d35cfd640 1 -- 172.21.15.16:0/1163970172 shutdown_connections 2024-09-19T09:28:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.925+0000 7f5d35cfd640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f5d180a3240 0x7f5d180a5700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.925+0000 7f5d35cfd640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d30103740 0x7f5d30100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.925+0000 7f5d35cfd640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d30108290 0x7f5d30101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.925+0000 7f5d35cfd640 1 --2- 172.21.15.16:0/1163970172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d30102d90 0x7f5d301009f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.925+0000 7f5d35cfd640 1 -- 172.21.15.16:0/1163970172 >> 172.21.15.16:0/1163970172 conn(0x7f5d300fc7d0 msgr2=0x7f5d30105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.925+0000 7f5d35cfd640 1 -- 172.21.15.16:0/1163970172 shutdown_connections 2024-09-19T09:28:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:05.925+0000 7f5d35cfd640 1 -- 172.21.15.16:0/1163970172 wait complete. 2024-09-19T09:28:06.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:06 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1163970172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:06.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:06 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1163970172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:06.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:06 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1163970172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:06.757 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0' 2024-09-19T09:28:06.757 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0 2024-09-19T09:28:07.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.081+0000 7f8bfae60640 1 -- 172.21.15.16:0/4236314255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4106f10 msgr2=0x7f8bf410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.081+0000 7f8bfae60640 1 --2- 172.21.15.16:0/4236314255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4106f10 0x7f8bf410ea90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009ca0 tx=0x7f8bec02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.082+0000 7f8bfae60640 1 -- 172.21.15.16:0/4236314255 shutdown_connections 2024-09-19T09:28:07.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.082+0000 7f8bfae60640 1 --2- 172.21.15.16:0/4236314255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf4102790 0x7f8bf41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.082+0000 7f8bfae60640 1 --2- 172.21.15.16:0/4236314255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4106f10 0x7f8bf410ea90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.082+0000 7f8bfae60640 1 --2- 172.21.15.16:0/4236314255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf4101de0 0x7f8bf41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.082+0000 7f8bfae60640 1 -- 172.21.15.16:0/4236314255 >> 172.21.15.16:0/4236314255 conn(0x7f8bf40fd880 msgr2=0x7f8bf40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.082+0000 7f8bfae60640 1 -- 172.21.15.16:0/4236314255 shutdown_connections 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.082+0000 7f8bfae60640 1 -- 172.21.15.16:0/4236314255 wait complete. 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.082+0000 7f8bfae60640 1 Processor -- start 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.082+0000 7f8bfae60640 1 -- start start 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bfae60640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf4101de0 0x7f8bf41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bfae60640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4102790 0x7f8bf4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bfae60640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf4106f10 0x7f8bf419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bfae60640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4111f70 con 0x7f8bf4102790 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bfae60640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4111df0 con 0x7f8bf4101de0 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bfae60640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41120f0 con 0x7f8bf4106f10 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bf3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4102790 0x7f8bf4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bf3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4102790 0x7f8bf4198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59952/0 (socket says 172.21.15.16:59952) 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bf3fff640 1 -- 172.21.15.16:0/3968206334 learned_addr learned my addr 172.21.15.16:0/3968206334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bf93fd640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf4106f10 0x7f8bf419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bf8bfc640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf4101de0 0x7f8bf41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bf3fff640 1 -- 172.21.15.16:0/3968206334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf4106f10 msgr2=0x7f8bf419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bf3fff640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf4106f10 0x7f8bf419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bf3fff640 1 -- 172.21.15.16:0/3968206334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf4101de0 msgr2=0x7f8bf41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bf3fff640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf4101de0 0x7f8bf41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bf3fff640 1 -- 172.21.15.16:0/3968206334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf419d380 con 0x7f8bf4102790 2024-09-19T09:28:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.083+0000 7f8bf8bfc640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf4101de0 0x7f8bf41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.084+0000 7f8bf3fff640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4102790 0x7f8bf4198910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8be000c970 tx=0x7f8be000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.084+0000 7f8bf1ffb640 1 -- 172.21.15.16:0/3968206334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0007bb0 con 0x7f8bf4102790 2024-09-19T09:28:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.084+0000 7f8bfae60640 1 -- 172.21.15.16:0/3968206334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf419d670 con 0x7f8bf4102790 2024-09-19T09:28:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.084+0000 7f8bf1ffb640 1 -- 172.21.15.16:0/3968206334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be0007d50 con 0x7f8bf4102790 2024-09-19T09:28:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.084+0000 7f8bf1ffb640 1 -- 172.21.15.16:0/3968206334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be000f450 con 0x7f8bf4102790 2024-09-19T09:28:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.084+0000 7f8bfae60640 1 -- 172.21.15.16:0/3968206334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf41a4f50 con 0x7f8bf4102790 2024-09-19T09:28:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.085+0000 7f8bfae60640 1 -- 172.21.15.16:0/3968206334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf41021c0 con 0x7f8bf4102790 2024-09-19T09:28:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.086+0000 7f8bf1ffb640 1 -- 172.21.15.16:0/3968206334 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8be0005020 con 0x7f8bf4102790 2024-09-19T09:28:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.087+0000 7f8bf1ffb640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8be40a3240 0x7f8be40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.087+0000 7f8bf8bfc640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8be40a3240 0x7f8be40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.087+0000 7f8bf1ffb640 1 -- 172.21.15.16:0/3968206334 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8be00cf680 con 0x7f8bf4102790 2024-09-19T09:28:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.087+0000 7f8bf8bfc640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8be40a3240 0x7f8be40a5700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8be8005fd0 tx=0x7f8be800b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.092+0000 7f8bf1ffb640 1 -- 172.21.15.16:0/3968206334 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8be0014030 con 0x7f8bf4102790 2024-09-19T09:28:07.213 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:07.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:07 smithi016 ceph-mon[30979]: pgmap v353: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T09:28:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.329+0000 7f8bfae60640 1 -- 172.21.15.16:0/3968206334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bf410a150 con 0x7f8bf4102790 2024-09-19T09:28:07.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.330+0000 7f8bf1ffb640 1 -- 172.21.15.16:0/3968206334 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f8be009c3a0 con 0x7f8bf4102790 2024-09-19T09:28:07.332 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:28:07.332 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:28:07.332 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:28:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.333+0000 7f8bfae60640 1 -- 172.21.15.16:0/3968206334 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8be40a3240 msgr2=0x7f8be40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.333+0000 7f8bfae60640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8be40a3240 0x7f8be40a5700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8be8005fd0 tx=0x7f8be800b5f0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.333+0000 7f8bfae60640 1 -- 172.21.15.16:0/3968206334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4102790 msgr2=0x7f8bf4198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.333+0000 7f8bfae60640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4102790 0x7f8bf4198910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8be000c970 tx=0x7f8be000ce30 comp rx=0 tx=0).stop 2024-09-19T09:28:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.333+0000 7f8bfae60640 1 -- 172.21.15.16:0/3968206334 shutdown_connections 2024-09-19T09:28:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.333+0000 7f8bfae60640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f8be40a3240 0x7f8be40a5700 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.333+0000 7f8bfae60640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf4106f10 0x7f8bf419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.333+0000 7f8bfae60640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf4102790 0x7f8bf4198910 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.333+0000 7f8bfae60640 1 --2- 172.21.15.16:0/3968206334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bf4101de0 0x7f8bf41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.333+0000 7f8bfae60640 1 -- 172.21.15.16:0/3968206334 >> 172.21.15.16:0/3968206334 conn(0x7f8bf40fd880 msgr2=0x7f8bf40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.333+0000 7f8bfae60640 1 -- 172.21.15.16:0/3968206334 shutdown_connections 2024-09-19T09:28:07.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.333+0000 7f8bfae60640 1 -- 172.21.15.16:0/3968206334 wait complete. 2024-09-19T09:28:07.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:07 smithi142 ceph-mon[35032]: pgmap v353: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T09:28:07.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:07 smithi097 ceph-mon[34864]: pgmap v353: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-19T09:28:07.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.929+0000 7f9255e0c640 1 -- 172.21.15.16:0/1580546844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9250100aa0 msgr2=0x7f925010ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.929+0000 7f9255e0c640 1 --2- 172.21.15.16:0/1580546844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9250100aa0 0x7f925010ea20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9248009ca0 tx=0x7f924802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.929+0000 7f9255e0c640 1 -- 172.21.15.16:0/1580546844 shutdown_connections 2024-09-19T09:28:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.929+0000 7f9255e0c640 1 --2- 172.21.15.16:0/1580546844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9250100aa0 0x7f925010ea20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.929+0000 7f9255e0c640 1 --2- 172.21.15.16:0/1580546844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9250100100 0x7f9250100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.929+0000 7f9255e0c640 1 --2- 172.21.15.16:0/1580546844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9250106b80 0x7f9250106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.929+0000 7f9255e0c640 1 -- 172.21.15.16:0/1580546844 >> 172.21.15.16:0/1580546844 conn(0x7f9250077230 msgr2=0x7f92500fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.929+0000 7f9255e0c640 1 -- 172.21.15.16:0/1580546844 shutdown_connections 2024-09-19T09:28:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.929+0000 7f9255e0c640 1 -- 172.21.15.16:0/1580546844 wait complete. 2024-09-19T09:28:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.930+0000 7f9255e0c640 1 Processor -- start 2024-09-19T09:28:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.930+0000 7f9255e0c640 1 -- start start 2024-09-19T09:28:07.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.930+0000 7f9255e0c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9250100100 0x7f92501012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f9255e0c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9250100aa0 0x7f9250101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f9255e0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9250106b80 0x7f9250101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f9255e0c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9250112050 con 0x7f9250106b80 2024-09-19T09:28:07.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f9255e0c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9250111ed0 con 0x7f9250100100 2024-09-19T09:28:07.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f9255e0c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92501121d0 con 0x7f9250100aa0 2024-09-19T09:28:07.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f924ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9250106b80 0x7f9250101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f924ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9250106b80 0x7f9250101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59976/0 (socket says 172.21.15.16:59976) 2024-09-19T09:28:07.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f924ffff640 1 -- 172.21.15.16:0/1608271344 learned_addr learned my addr 172.21.15.16:0/1608271344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:07.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f924f7fe640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9250100100 0x7f92501012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f924effd640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9250100aa0 0x7f9250101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f924ffff640 1 -- 172.21.15.16:0/1608271344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9250100aa0 msgr2=0x7f9250101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f924ffff640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9250100aa0 0x7f9250101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f924ffff640 1 -- 172.21.15.16:0/1608271344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9250100100 msgr2=0x7f92501012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f924ffff640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9250100100 0x7f92501012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f924ffff640 1 -- 172.21.15.16:0/1608271344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92501a68d0 con 0x7f9250106b80 2024-09-19T09:28:07.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f924f7fe640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9250100100 0x7f92501012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:07.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.931+0000 7f924ffff640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9250106b80 0x7f9250101d60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f92480319f0 tx=0x7f9248031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.932+0000 7f924cff9640 1 -- 172.21.15.16:0/1608271344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248003b70 con 0x7f9250106b80 2024-09-19T09:28:07.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.932+0000 7f924cff9640 1 -- 172.21.15.16:0/1608271344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9248003d10 con 0x7f9250106b80 2024-09-19T09:28:07.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.932+0000 7f9255e0c640 1 -- 172.21.15.16:0/1608271344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92501a6b60 con 0x7f9250106b80 2024-09-19T09:28:07.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.932+0000 7f924cff9640 1 -- 172.21.15.16:0/1608271344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924802fc40 con 0x7f9250106b80 2024-09-19T09:28:07.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.932+0000 7f9255e0c640 1 -- 172.21.15.16:0/1608271344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92501a70a0 con 0x7f9250106b80 2024-09-19T09:28:07.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.934+0000 7f924cff9640 1 -- 172.21.15.16:0/1608271344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9248004050 con 0x7f9250106b80 2024-09-19T09:28:07.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.935+0000 7f92581e5740 1 Processor -- start 2024-09-19T09:28:07.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.935+0000 7f92581e5740 1 -- start start 2024-09-19T09:28:07.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.935+0000 7f924cff9640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f92380a3240 0x7f92380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.935+0000 7f92581e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 0x56038aac2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.935+0000 7f92581e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac3080 0x56038aacd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.935+0000 7f924effd640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f92380a3240 0x7f92380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.935+0000 7f92581e5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aacd940 0x56038aacfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.935+0000 7f924cff9640 1 -- 172.21.15.16:0/1608271344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f92480f0950 con 0x7f9250106b80 2024-09-19T09:28:07.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.935+0000 7f92581e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56038aa282a0 con 0x56038aac2760 2024-09-19T09:28:07.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.935+0000 7f92581e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56038a9cfb10 con 0x56038aac3080 2024-09-19T09:28:07.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.935+0000 7f92581e5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56038aa24260 con 0x56038aacd940 2024-09-19T09:28:07.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.935+0000 7f924f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 0x56038aac2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.935+0000 7f924f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 0x56038aac2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59992/0 (socket says 172.21.15.16:59992) 2024-09-19T09:28:07.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.936+0000 7f924f7fe640 1 -- 172.21.15.16:0/2115440910 learned_addr learned my addr 172.21.15.16:0/2115440910 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:07.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.936+0000 7f924ffff640 1 --2- 172.21.15.16:0/2115440910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aacd940 0x56038aacfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.936+0000 7f924effd640 1 --2- 172.21.15.16:0/2115440910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac3080 0x56038aacd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.936+0000 7f924f7fe640 1 -- 172.21.15.16:0/2115440910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aacd940 msgr2=0x56038aacfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.936+0000 7f924f7fe640 1 --2- 172.21.15.16:0/2115440910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aacd940 0x56038aacfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.936+0000 7f924f7fe640 1 -- 172.21.15.16:0/2115440910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac3080 msgr2=0x56038aacd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.936+0000 7f924f7fe640 1 --2- 172.21.15.16:0/2115440910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac3080 0x56038aacd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.936+0000 7f924f7fe640 1 -- 172.21.15.16:0/2115440910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56038a799b90 con 0x56038aac2760 2024-09-19T09:28:07.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.936+0000 7f924effd640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f92380a3240 0x7f92380a5700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9250102d70 tx=0x7f9244013880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.936+0000 7f924f7fe640 1 --2- 172.21.15.16:0/2115440910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 0x56038aac2b40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f924000f440 tx=0x7f924000f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365067b95852a322 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.936+0000 7f9235ffb640 1 -- 172.21.15.16:0/2115440910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240005220 con 0x56038aac2760 2024-09-19T09:28:07.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.936+0000 7f9235ffb640 1 -- 172.21.15.16:0/2115440910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9240014070 con 0x56038aac2760 2024-09-19T09:28:07.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.936+0000 7f9235ffb640 1 -- 172.21.15.16:0/2115440910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240010ac0 con 0x56038aac2760 2024-09-19T09:28:07.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.937+0000 7f92581e5740 1 -- 172.21.15.16:0/2115440910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 msgr2=0x56038aac2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.937+0000 7f92581e5740 1 --2- 172.21.15.16:0/2115440910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 0x56038aac2b40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f924000f440 tx=0x7f924000f900 comp rx=0 tx=0).stop 2024-09-19T09:28:07.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.937+0000 7f92581e5740 1 -- 172.21.15.16:0/2115440910 shutdown_connections 2024-09-19T09:28:07.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.937+0000 7f92581e5740 1 --2- 172.21.15.16:0/2115440910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aacd940 0x56038aacfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.937+0000 7f92581e5740 1 --2- 172.21.15.16:0/2115440910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac3080 0x56038aacd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.937+0000 7f92581e5740 1 --2- 172.21.15.16:0/2115440910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 0x56038aac2b40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.937+0000 7f92581e5740 1 -- 172.21.15.16:0/2115440910 >> 172.21.15.16:0/2115440910 conn(0x56038aabecf0 msgr2=0x56038aabf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.937+0000 7f92581e5740 1 -- 172.21.15.16:0/2115440910 shutdown_connections 2024-09-19T09:28:07.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.937+0000 7f92581e5740 1 -- 172.21.15.16:0/2115440910 wait complete. 2024-09-19T09:28:07.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.938+0000 7f92581e5740 1 -- start start 2024-09-19T09:28:07.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.938+0000 7f92581e5740 1 Processor -- start 2024-09-19T09:28:07.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.938+0000 7f92581e5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aac2760 0x56038aaca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.938+0000 7f92581e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac3080 0x56038aacace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.938+0000 7f92581e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aacd940 0x56038aacb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.938+0000 7f92581e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56038aad0510 con 0x56038aacd940 2024-09-19T09:28:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.938+0000 7f92581e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56038a9b06e0 con 0x56038aac3080 2024-09-19T09:28:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.938+0000 7f92581e5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56038aad0690 con 0x56038aac2760 2024-09-19T09:28:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.938+0000 7f92581e5740 10 client.0 _ll_register_callbacks cb 0x56038a9d4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.938+0000 7f924ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aacd940 0x56038aacb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.938+0000 7f924ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aacd940 0x56038aacb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59996/0 (socket says 172.21.15.16:59996) 2024-09-19T09:28:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.938+0000 7f924ffff640 1 -- 172.21.15.16:0/3879707910 learned_addr learned my addr 172.21.15.16:0/3879707910 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.938+0000 7f924f7fe640 1 --2- 172.21.15.16:0/3879707910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aac2760 0x56038aaca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f924effd640 1 --2- 172.21.15.16:0/3879707910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac3080 0x56038aacace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f924ffff640 1 -- 172.21.15.16:0/3879707910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aac2760 msgr2=0x56038aaca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f924ffff640 1 --2- 172.21.15.16:0/3879707910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aac2760 0x56038aaca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f924ffff640 1 -- 172.21.15.16:0/3879707910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac3080 msgr2=0x56038aacace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f924ffff640 1 --2- 172.21.15.16:0/3879707910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac3080 0x56038aacace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f924ffff640 1 -- 172.21.15.16:0/3879707910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56038aad9980 con 0x56038aacd940 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f924ffff640 1 --2- 172.21.15.16:0/3879707910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aacd940 0x56038aacb220 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9248006010 tx=0x7f92480bcbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba7d57400bfc634 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f922effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f922effd640 1 -- 172.21.15.16:0/3879707910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92480bd090 con 0x56038aacd940 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f922effd640 1 -- 172.21.15.16:0/3879707910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9248004730 con 0x56038aacd940 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f92581e5740 1 -- 172.21.15.16:0/3879707910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9240014070 con 0x56038aacd940 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f922effd640 1 -- 172.21.15.16:0/3879707910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92480c1440 con 0x56038aacd940 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f92581e5740 10 client.15513 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.939+0000 7f92581e5740 1 -- 172.21.15.16:0/3879707910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56038aad0510 con 0x56038aacd940 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.940+0000 7f922effd640 1 -- 172.21.15.16:0/3879707910 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9248036040 con 0x56038aacd940 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.940+0000 7f922effd640 1 -- 172.21.15.16:0/3879707910 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f92480bd2f0 con 0x56038aacd940 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.940+0000 7f92581e5740 10 client.15513 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:07.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.940+0000 7f92581e5740 10 client.15513 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:28:07.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.940+0000 7f92581e5740 20 client.15513 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:07.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.940+0000 7f92581e5740 1 --2- 172.21.15.16:0/3879707910 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56038aadd730 0x56038aadfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.940+0000 7f92581e5740 4 client.15513 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:07.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.940+0000 7f92581e5740 1 -- 172.21.15.16:0/3879707910 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9240014070 con 0x56038aadd730 2024-09-19T09:28:07.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.940+0000 7f924f7fe640 1 --2- 172.21.15.16:0/3879707910 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56038aadd730 0x56038aadfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.941+0000 7f922effd640 10 client.15513 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:07.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.941+0000 7f924f7fe640 1 --2- 172.21.15.16:0/3879707910 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56038aadd730 0x56038aadfbb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d202f514385761f server_cookie=4573f7aceb446cd9 in_seq=0 out_seq=0 2024-09-19T09:28:07.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.941+0000 7f922effd640 1 -- 172.21.15.16:0/3879707910 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9240014070 con 0x56038aadd730 2024-09-19T09:28:07.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.941+0000 7f922effd640 10 client.15513 handle_command_reply: tid=0 2024-09-19T09:28:07.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.941+0000 7f92581e5740 1 client.15513 shutdown 2024-09-19T09:28:07.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.942+0000 7f92581e5740 1 -- 172.21.15.16:0/3879707910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aacd940 msgr2=0x56038aacb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.942+0000 7f92581e5740 1 --2- 172.21.15.16:0/3879707910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aacd940 0x56038aacb220 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9248006010 tx=0x7f92480bcbd0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.942+0000 7f92581e5740 1 -- 172.21.15.16:0/3879707910 shutdown_connections 2024-09-19T09:28:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.942+0000 7f92581e5740 1 --2- 172.21.15.16:0/3879707910 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56038aadd730 0x56038aadfbb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.942+0000 7f92581e5740 1 --2- 172.21.15.16:0/3879707910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aacd940 0x56038aacb220 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.942+0000 7f92581e5740 1 --2- 172.21.15.16:0/3879707910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac3080 0x56038aacace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.942+0000 7f92581e5740 1 --2- 172.21.15.16:0/3879707910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aac2760 0x56038aaca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.942+0000 7f92581e5740 1 -- 172.21.15.16:0/3879707910 >> 172.21.15.16:0/3879707910 conn(0x56038aabecf0 msgr2=0x56038aabde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.942+0000 7f922effd640 0 client.15513 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.942+0000 7f92581e5740 1 -- 172.21.15.16:0/3879707910 shutdown_connections 2024-09-19T09:28:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.942+0000 7f92581e5740 1 -- 172.21.15.16:0/3879707910 wait complete. 2024-09-19T09:28:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.942+0000 7f92581e5740 20 client.15513 trim_cache size 0 max 16384 2024-09-19T09:28:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.959+0000 7f92581e5740 1 Processor -- start 2024-09-19T09:28:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.959+0000 7f92581e5740 1 -- start start 2024-09-19T09:28:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f92581e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac2760 0x56038aac51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f92581e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac3080 0x56038aae54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f92581e5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aacd940 0x56038aae5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f92581e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92501af6f0 con 0x56038aac3080 2024-09-19T09:28:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f92581e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92501b1230 con 0x56038aac2760 2024-09-19T09:28:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f92581e5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92501afdc0 con 0x56038aacd940 2024-09-19T09:28:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f924effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac3080 0x56038aae54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f924effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac3080 0x56038aae54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60008/0 (socket says 172.21.15.16:60008) 2024-09-19T09:28:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f924effd640 1 -- 172.21.15.16:0/672863319 learned_addr learned my addr 172.21.15.16:0/672863319 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f924f7fe640 1 --2- 172.21.15.16:0/672863319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac2760 0x56038aac51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f924effd640 1 -- 172.21.15.16:0/672863319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aacd940 msgr2=0x56038aae5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f924effd640 1 --2- 172.21.15.16:0/672863319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aacd940 0x56038aae5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f924effd640 1 -- 172.21.15.16:0/672863319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac2760 msgr2=0x56038aac51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f924effd640 1 --2- 172.21.15.16:0/672863319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac2760 0x56038aac51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f924effd640 1 -- 172.21.15.16:0/672863319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56038aaba260 con 0x56038aac3080 2024-09-19T09:28:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.960+0000 7f924effd640 1 --2- 172.21.15.16:0/672863319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac3080 0x56038aae54c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9244009a20 tx=0x7f9244005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917a953d47cffd0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.961+0000 7f92357fa640 1 -- 172.21.15.16:0/672863319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244005220 con 0x56038aac3080 2024-09-19T09:28:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.961+0000 7f92357fa640 1 -- 172.21.15.16:0/672863319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9244025070 con 0x56038aac3080 2024-09-19T09:28:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.961+0000 7f92357fa640 1 -- 172.21.15.16:0/672863319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244003b10 con 0x56038aac3080 2024-09-19T09:28:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.961+0000 7f92581e5740 1 -- 172.21.15.16:0/672863319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac3080 msgr2=0x56038aae54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.961+0000 7f92581e5740 1 --2- 172.21.15.16:0/672863319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac3080 0x56038aae54c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9244009a20 tx=0x7f9244005f90 comp rx=0 tx=0).stop 2024-09-19T09:28:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.962+0000 7f92581e5740 1 -- 172.21.15.16:0/672863319 shutdown_connections 2024-09-19T09:28:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.962+0000 7f92581e5740 1 --2- 172.21.15.16:0/672863319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aacd940 0x56038aae5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.962+0000 7f92581e5740 1 --2- 172.21.15.16:0/672863319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac3080 0x56038aae54c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.962+0000 7f92581e5740 1 --2- 172.21.15.16:0/672863319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aac2760 0x56038aac51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.962+0000 7f92581e5740 1 -- 172.21.15.16:0/672863319 >> 172.21.15.16:0/672863319 conn(0x56038aabecf0 msgr2=0x56038aacc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.962+0000 7f92581e5740 1 -- 172.21.15.16:0/672863319 shutdown_connections 2024-09-19T09:28:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.962+0000 7f92581e5740 1 -- 172.21.15.16:0/672863319 wait complete. 2024-09-19T09:28:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.962+0000 7f92581e5740 1 -- start start 2024-09-19T09:28:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.962+0000 7f92581e5740 1 Processor -- start 2024-09-19T09:28:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f92581e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 0x56038ab41790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f92581e5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aac3080 0x56038ab41cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f92581e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aacd940 0x56038aad1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f92581e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92501b0b60 con 0x56038aac2760 2024-09-19T09:28:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f92581e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92501b0490 con 0x56038aacd940 2024-09-19T09:28:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f92581e5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92501ad6b0 con 0x56038aac3080 2024-09-19T09:28:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f92581e5740 10 client.0 _ll_register_callbacks cb 0x7f92480bcf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f924f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 0x56038ab41790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f924f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 0x56038ab41790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60020/0 (socket says 172.21.15.16:60020) 2024-09-19T09:28:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f924f7fe640 1 -- 172.21.15.16:0/3033497806 learned_addr learned my addr 172.21.15.16:0/3033497806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f924ffff640 1 --2- 172.21.15.16:0/3033497806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aacd940 0x56038aad1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f924effd640 1 --2- 172.21.15.16:0/3033497806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aac3080 0x56038ab41cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f924f7fe640 1 -- 172.21.15.16:0/3033497806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aac3080 msgr2=0x56038ab41cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f924f7fe640 1 --2- 172.21.15.16:0/3033497806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aac3080 0x56038ab41cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f924f7fe640 1 -- 172.21.15.16:0/3033497806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aacd940 msgr2=0x56038aad1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f924f7fe640 1 --2- 172.21.15.16:0/3033497806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aacd940 0x56038aad1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f924f7fe640 1 -- 172.21.15.16:0/3033497806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56038aabf980 con 0x56038aac2760 2024-09-19T09:28:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f924f7fe640 1 --2- 172.21.15.16:0/3033497806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 0x56038ab41790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f924000ac60 tx=0x7f924000aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea46d28076345d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.963+0000 7f922effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f922effd640 1 -- 172.21.15.16:0/3033497806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240005290 con 0x56038aac2760 2024-09-19T09:28:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f922effd640 1 -- 172.21.15.16:0/3033497806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f924001d040 con 0x56038aac2760 2024-09-19T09:28:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f92581e5740 1 -- 172.21.15.16:0/3033497806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9244025070 con 0x56038aac2760 2024-09-19T09:28:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f922effd640 1 -- 172.21.15.16:0/3033497806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9240005430 con 0x56038aac2760 2024-09-19T09:28:07.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f92581e5740 10 client.15519 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:07.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f92581e5740 1 -- 172.21.15.16:0/3033497806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56038ab48dc0 con 0x56038aac2760 2024-09-19T09:28:07.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f922effd640 1 -- 172.21.15.16:0/3033497806 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9244025070 con 0x56038aac2760 2024-09-19T09:28:07.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f922effd640 1 -- 172.21.15.16:0/3033497806 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f924003b050 con 0x56038aac2760 2024-09-19T09:28:07.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f92581e5740 10 client.15519 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:07.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f92581e5740 10 client.15519 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:28:07.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f92581e5740 20 client.15519 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:07.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f92581e5740 1 --2- 172.21.15.16:0/3033497806 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56038ab493c0 0x56038ab4b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:07.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f92581e5740 4 client.15519 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}] 2024-09-19T09:28:07.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.964+0000 7f92581e5740 1 -- 172.21.15.16:0/3033497806 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}) -- 0x56038aa359d0 con 0x56038ab493c0 2024-09-19T09:28:07.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.965+0000 7f924effd640 1 --2- 172.21.15.16:0/3033497806 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56038ab493c0 0x56038ab4b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:07.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.965+0000 7f922effd640 10 client.15519 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:07.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.965+0000 7f924effd640 1 --2- 172.21.15.16:0/3033497806 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56038ab493c0 0x56038ab4b7b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=863e8fd151afdaa8 server_cookie=ec4a2aa94148945c in_seq=0 out_seq=0 2024-09-19T09:28:07.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.977+0000 7f922effd640 1 -- 172.21.15.16:0/3033497806 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56038aa359d0 con 0x56038ab493c0 2024-09-19T09:28:07.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.977+0000 7f922effd640 10 client.15519 handle_command_reply: tid=0 2024-09-19T09:28:07.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.977+0000 7f92581e5740 1 client.15519 shutdown 2024-09-19T09:28:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.978+0000 7f92581e5740 1 -- 172.21.15.16:0/3033497806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 msgr2=0x56038ab41790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.978+0000 7f92581e5740 1 --2- 172.21.15.16:0/3033497806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 0x56038ab41790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f924000ac60 tx=0x7f924000aab0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.978+0000 7f92581e5740 1 -- 172.21.15.16:0/3033497806 shutdown_connections 2024-09-19T09:28:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.978+0000 7f92581e5740 1 --2- 172.21.15.16:0/3033497806 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56038ab493c0 0x56038ab4b7b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.978+0000 7f92581e5740 1 --2- 172.21.15.16:0/3033497806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56038aac3080 0x56038ab41cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.978+0000 7f92581e5740 1 --2- 172.21.15.16:0/3033497806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56038aacd940 0x56038aad1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.978+0000 7f922effd640 0 client.15519 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.978+0000 7f92581e5740 1 --2- 172.21.15.16:0/3033497806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56038aac2760 0x56038ab41790 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.978+0000 7f92581e5740 1 -- 172.21.15.16:0/3033497806 >> 172.21.15.16:0/3033497806 conn(0x56038aabecf0 msgr2=0x56038aabd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.978+0000 7f92581e5740 1 -- 172.21.15.16:0/3033497806 shutdown_connections 2024-09-19T09:28:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.978+0000 7f92581e5740 1 -- 172.21.15.16:0/3033497806 wait complete. 2024-09-19T09:28:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.978+0000 7f92581e5740 20 client.15519 trim_cache size 0 max 16384 2024-09-19T09:28:07.982 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:28:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-19T09:28:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24500, 2024-09-19T09:28:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 2, 2024-09-19T09:28:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-19T09:28:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "a08a4233": { 2024-09-19T09:28:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 2, 2024-09-19T09:28:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-19T09:28:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-19T09:28:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-19T09:28:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-19T09:28:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:28:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-19T09:28:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 131.2, 2024-09-19T09:28:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-19T09:28:07.983 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-19T09:28:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-19T09:28:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-19T09:28:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-19T09:28:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-19T09:28:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:07.984 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:07.985 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:28:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.979+0000 7f9255e0c640 1 -- 172.21.15.16:0/1608271344 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f92380a3240 msgr2=0x7f92380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.979+0000 7f9255e0c640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f92380a3240 0x7f92380a5700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9250102d70 tx=0x7f9244013880 comp rx=0 tx=0).stop 2024-09-19T09:28:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.979+0000 7f9255e0c640 1 -- 172.21.15.16:0/1608271344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9250106b80 msgr2=0x7f9250101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.979+0000 7f9255e0c640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9250106b80 0x7f9250101d60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f92480319f0 tx=0x7f9248031a20 comp rx=0 tx=0).stop 2024-09-19T09:28:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.979+0000 7f9255e0c640 1 -- 172.21.15.16:0/1608271344 shutdown_connections 2024-09-19T09:28:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.979+0000 7f9255e0c640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f92380a3240 0x7f92380a5700 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.979+0000 7f9255e0c640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9250100aa0 0x7f9250101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.979+0000 7f9255e0c640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9250106b80 0x7f9250101d60 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.979+0000 7f9255e0c640 1 --2- 172.21.15.16:0/1608271344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9250100100 0x7f92501012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.979+0000 7f9255e0c640 1 -- 172.21.15.16:0/1608271344 >> 172.21.15.16:0/1608271344 conn(0x7f9250077230 msgr2=0x7f9250104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.979+0000 7f9255e0c640 1 -- 172.21.15.16:0/1608271344 shutdown_connections 2024-09-19T09:28:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:07.979+0000 7f9255e0c640 1 -- 172.21.15.16:0/1608271344 wait complete. 2024-09-19T09:28:08.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:08 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3968206334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:08.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:08 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3968206334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:08.424 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:08 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3968206334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:08.765 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:28:09.187 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a08a4233', quiesce took 5.0 seconds. Will release after: 6.199999999999999 2024-09-19T09:28:09.270 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:09.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:09 smithi142 ceph-mon[35032]: pgmap v354: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T09:28:09.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:09 smithi097 ceph-mon[34864]: pgmap v354: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T09:28:09.392 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:09 smithi016 ceph-mon[30979]: pgmap v354: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T09:28:09.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.807+0000 7f48000a1640 1 -- 172.21.15.16:0/3505974390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f8100e20 msgr2=0x7f47f8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.807+0000 7f48000a1640 1 --2- 172.21.15.16:0/3505974390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f8100e20 0x7f47f8101200 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009ca0 tx=0x7f47f402a950 comp rx=0 tx=0).stop 2024-09-19T09:28:09.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.807+0000 7f48000a1640 1 -- 172.21.15.16:0/3505974390 shutdown_connections 2024-09-19T09:28:09.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.807+0000 7f48000a1640 1 --2- 172.21.15.16:0/3505974390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47f8102ef0 0x7f47f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.807+0000 7f48000a1640 1 --2- 172.21.15.16:0/3505974390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f8102570 0x7f47f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.807+0000 7f48000a1640 1 --2- 172.21.15.16:0/3505974390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f8100e20 0x7f47f8101200 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.807+0000 7f48000a1640 1 -- 172.21.15.16:0/3505974390 >> 172.21.15.16:0/3505974390 conn(0x7f47f80fc7d0 msgr2=0x7f47f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:09.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.807+0000 7f48000a1640 1 -- 172.21.15.16:0/3505974390 shutdown_connections 2024-09-19T09:28:09.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.807+0000 7f48000a1640 1 -- 172.21.15.16:0/3505974390 wait complete. 2024-09-19T09:28:09.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.808+0000 7f48000a1640 1 Processor -- start 2024-09-19T09:28:09.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.808+0000 7f48000a1640 1 -- start start 2024-09-19T09:28:09.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.808+0000 7f48000a1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f8100e20 0x7f47f8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.808+0000 7f48000a1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47f8102570 0x7f47f8199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.808+0000 7f48000a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f8102ef0 0x7f47f81a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.808+0000 7f48000a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47f8110f80 con 0x7f47f8102ef0 2024-09-19T09:28:09.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.808+0000 7f48000a1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f47f8110e00 con 0x7f47f8100e20 2024-09-19T09:28:09.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.808+0000 7f48000a1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47f8111100 con 0x7f47f8102570 2024-09-19T09:28:09.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47fe63e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f8102ef0 0x7f47f81a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47fe63e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f8102ef0 0x7f47f81a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60026/0 (socket says 172.21.15.16:60026) 2024-09-19T09:28:09.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47fe63e640 1 -- 172.21.15.16:0/2688277029 learned_addr learned my addr 172.21.15.16:0/2688277029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:09.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47fd63c640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47f8102570 0x7f47f8199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47fde3d640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f8100e20 0x7f47f8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47fe63e640 1 -- 172.21.15.16:0/2688277029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47f8102570 msgr2=0x7f47f8199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47fe63e640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47f8102570 0x7f47f8199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47fe63e640 1 -- 172.21.15.16:0/2688277029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f8100e20 msgr2=0x7f47f8199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47fe63e640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f8100e20 0x7f47f8199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47fe63e640 1 -- 172.21.15.16:0/2688277029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f81a4910 con 0x7f47f8102ef0 2024-09-19T09:28:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47fde3d640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f8100e20 0x7f47f8199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47fe63e640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f8102ef0 0x7f47f81a43d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c970 tx=0x7f47f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47eeffd640 1 -- 172.21.15.16:0/2688277029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0007bb0 con 0x7f47f8102ef0 2024-09-19T09:28:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f48000a1640 1 -- 172.21.15.16:0/2688277029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f81a4ae0 con 0x7f47f8102ef0 2024-09-19T09:28:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47eeffd640 1 -- 172.21.15.16:0/2688277029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f0007d50 con 0x7f47f8102ef0 2024-09-19T09:28:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f47eeffd640 1 -- 172.21.15.16:0/2688277029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f000f450 con 0x7f47f8102ef0 2024-09-19T09:28:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.809+0000 7f48000a1640 1 -- 172.21.15.16:0/2688277029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f81a4ff0 con 0x7f47f8102ef0 2024-09-19T09:28:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.812+0000 7f47eeffd640 1 -- 172.21.15.16:0/2688277029 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f47f0004f20 con 0x7f47f8102ef0 2024-09-19T09:28:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.812+0000 7f480247a740 1 Processor -- start 2024-09-19T09:28:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.812+0000 7f480247a740 1 -- start start 2024-09-19T09:28:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f480247a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e9b40 0x5593289e8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f480247a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289e85d0 0x5593289e8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f480247a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289f7780 0x5593289f9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f480247a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55932894e2f0 con 0x5593289f7780 2024-09-19T09:28:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f480247a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5593288f5cf0 con 0x5593289e9b40 2024-09-19T09:28:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f480247a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55932894a2b0 con 0x5593289e85d0 2024-09-19T09:28:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47eeffd640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f47e40a3240 0x7f47e40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47fd63c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289f7780 0x5593289f9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47fd63c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289f7780 0x5593289f9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60028/0 (socket says 172.21.15.16:60028) 2024-09-19T09:28:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47fd63c640 1 -- 172.21.15.16:0/3585639106 learned_addr learned my addr 172.21.15.16:0/3585639106 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47fde3d640 1 --2- 172.21.15.16:0/3585639106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289e85d0 0x5593289e8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47fe63e640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f47e40a3240 0x7f47e40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47fd63c640 1 --2- 172.21.15.16:0/3585639106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e9b40 0x5593289e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47eeffd640 1 -- 172.21.15.16:0/2688277029 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f47f00cf5d0 con 0x7f47f8102ef0 2024-09-19T09:28:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47fd63c640 1 -- 172.21.15.16:0/3585639106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289e85d0 msgr2=0x5593289e8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47fd63c640 1 --2- 172.21.15.16:0/3585639106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289e85d0 0x5593289e8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47fd63c640 1 -- 172.21.15.16:0/3585639106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e9b40 msgr2=0x5593289e8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47fd63c640 1 --2- 172.21.15.16:0/3585639106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e9b40 0x5593289e8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47fd63c640 1 -- 172.21.15.16:0/3585639106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593288da3a0 con 0x5593289f7780 2024-09-19T09:28:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.813+0000 7f47fde3d640 1 --2- 172.21.15.16:0/3585639106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289e85d0 0x5593289e8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.814+0000 7f47fe63e640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f47e40a3240 0x7f47e40a5700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c940 tx=0x7f47f0066580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.814+0000 7f47fd63c640 1 --2- 172.21.15.16:0/3585639106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289f7780 0x5593289f9b70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f47e80149b0 tx=0x7f47e800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d37d63a6503c47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:09.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.814+0000 7f47e3fff640 1 -- 172.21.15.16:0/3585639106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e80142d0 con 0x5593289f7780 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.814+0000 7f47e3fff640 1 -- 172.21.15.16:0/3585639106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e8005500 con 0x5593289f7780 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.814+0000 7f47e3fff640 1 -- 172.21.15.16:0/3585639106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e800f760 con 0x5593289f7780 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.815+0000 7f480247a740 1 -- 172.21.15.16:0/3585639106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289f7780 msgr2=0x5593289f9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.815+0000 7f480247a740 1 --2- 172.21.15.16:0/3585639106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289f7780 0x5593289f9b70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f47e80149b0 tx=0x7f47e800c4b0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.815+0000 7f480247a740 1 -- 172.21.15.16:0/3585639106 shutdown_connections 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.815+0000 7f480247a740 1 --2- 172.21.15.16:0/3585639106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289e85d0 0x5593289e8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.815+0000 7f480247a740 1 --2- 172.21.15.16:0/3585639106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289f7780 0x5593289f9b70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.815+0000 7f480247a740 1 --2- 172.21.15.16:0/3585639106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e9b40 0x5593289e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.815+0000 7f480247a740 1 -- 172.21.15.16:0/3585639106 >> 172.21.15.16:0/3585639106 conn(0x5593289b1310 msgr2=0x5593289e4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.815+0000 7f480247a740 1 -- 172.21.15.16:0/3585639106 shutdown_connections 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.815+0000 7f480247a740 1 -- 172.21.15.16:0/3585639106 wait complete. 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.815+0000 7f480247a740 1 -- start start 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.815+0000 7f480247a740 1 Processor -- start 2024-09-19T09:28:09.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f480247a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e85d0 0x5593289f7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f480247a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e9b40 0x5593289f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f480247a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289f7780 0x5593289f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f480247a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5593289fa480 con 0x5593289e85d0 2024-09-19T09:28:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f480247a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559328909060 con 0x5593289e9b40 2024-09-19T09:28:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f480247a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5593289fa600 con 0x5593289f7780 2024-09-19T09:28:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f480247a740 10 client.0 _ll_register_callbacks cb 0x55932865c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47fd63c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e85d0 0x5593289f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47fd63c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e85d0 0x5593289f7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60032/0 (socket says 172.21.15.16:60032) 2024-09-19T09:28:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47fd63c640 1 -- 172.21.15.16:0/563410570 learned_addr learned my addr 172.21.15.16:0/563410570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47fde3d640 1 --2- 172.21.15.16:0/563410570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e9b40 0x5593289f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47fd63c640 1 --2- 172.21.15.16:0/563410570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289f7780 0x5593289f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47fd63c640 1 -- 172.21.15.16:0/563410570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289f7780 msgr2=0x5593289f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:09.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47fd63c640 1 --2- 172.21.15.16:0/563410570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289f7780 0x5593289f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47fd63c640 1 -- 172.21.15.16:0/563410570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e9b40 msgr2=0x5593289f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47fd63c640 1 --2- 172.21.15.16:0/563410570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e9b40 0x5593289f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47fd63c640 1 -- 172.21.15.16:0/563410570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593286bf3c0 con 0x5593289e85d0 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47fd63c640 1 --2- 172.21.15.16:0/563410570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e85d0 0x5593289f7140 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f47e8005790 tx=0x7f47e8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703bd4a70f1ea8a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47e17fa640 1 -- 172.21.15.16:0/563410570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e802a020 con 0x5593289e85d0 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f47e17fa640 1 -- 172.21.15.16:0/563410570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f47e80048c0 con 0x5593289e85d0 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.816+0000 7f480247a740 1 -- 172.21.15.16:0/563410570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e8005500 con 0x5593289e85d0 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.817+0000 7f47e17fa640 1 -- 172.21.15.16:0/563410570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47e8004020 con 0x5593289e85d0 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.817+0000 7f480247a740 10 client.15531 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.817+0000 7f480247a740 1 -- 172.21.15.16:0/563410570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559328a05f10 con 0x5593289e85d0 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.817+0000 7f47e17fa640 1 -- 172.21.15.16:0/563410570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e8005500 con 0x5593289e85d0 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.817+0000 7f47e17fa640 1 -- 172.21.15.16:0/563410570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f47e80042e0 con 0x5593289e85d0 2024-09-19T09:28:09.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.817+0000 7f480247a740 10 client.15531 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.817+0000 7f480247a740 10 client.15531 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-19T09:28:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.817+0000 7f480247a740 20 client.15531 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.817+0000 7f480247a740 1 --2- 172.21.15.16:0/563410570 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559328a07260 0x559328a09650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.817+0000 7f480247a740 4 client.15531 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.817+0000 7f480247a740 1 -- 172.21.15.16:0/563410570 --> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55932895b860 con 0x559328a07260 2024-09-19T09:28:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.817+0000 7f47fde3d640 1 --2- 172.21.15.16:0/563410570 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559328a07260 0x559328a09650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.818+0000 7f47fde3d640 1 --2- 172.21.15.16:0/563410570 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559328a07260 0x559328a09650 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b2ec11fd8574b07f server_cookie=30b5107e3bc7f3ae in_seq=0 out_seq=0 2024-09-19T09:28:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.818+0000 7f47e17fa640 10 client.15531 ms_handle_connect on v2:172.21.15.142:6832/3906366829 2024-09-19T09:28:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f47e17fa640 1 -- 172.21.15.16:0/563410570 <== mds.4 v2:172.21.15.142:6832/3906366829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55932895b860 con 0x559328a07260 2024-09-19T09:28:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f47e17fa640 10 client.15531 handle_command_reply: tid=0 2024-09-19T09:28:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f480247a740 1 client.15531 shutdown 2024-09-19T09:28:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f480247a740 1 -- 172.21.15.16:0/563410570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e85d0 msgr2=0x5593289f7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f480247a740 1 --2- 172.21.15.16:0/563410570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e85d0 0x5593289f7140 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f47e8005790 tx=0x7f47e8005850 comp rx=0 tx=0).stop 2024-09-19T09:28:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f480247a740 1 -- 172.21.15.16:0/563410570 shutdown_connections 2024-09-19T09:28:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f480247a740 1 --2- 172.21.15.16:0/563410570 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559328a07260 0x559328a09650 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f480247a740 1 --2- 172.21.15.16:0/563410570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289f7780 0x5593289f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f480247a740 1 --2- 172.21.15.16:0/563410570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e9b40 0x5593289f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f47e17fa640 0 client.15531 ms_handle_reset on v2:172.21.15.142:6832/3906366829 2024-09-19T09:28:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f480247a740 1 --2- 172.21.15.16:0/563410570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e85d0 0x5593289f7140 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f480247a740 1 -- 172.21.15.16:0/563410570 >> 172.21.15.16:0/563410570 conn(0x5593289b1310 msgr2=0x5593289ea960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f480247a740 1 -- 172.21.15.16:0/563410570 shutdown_connections 2024-09-19T09:28:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f480247a740 1 -- 172.21.15.16:0/563410570 wait complete. 2024-09-19T09:28:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.819+0000 7f480247a740 20 client.15531 trim_cache size 0 max 16384 2024-09-19T09:28:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.833+0000 7f480247a740 1 Processor -- start 2024-09-19T09:28:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.833+0000 7f480247a740 1 -- start start 2024-09-19T09:28:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f480247a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e85d0 0x5593289f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f480247a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e9b40 0x559328a0c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f480247a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289f7780 0x559328a0cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f480247a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81ab4c0 con 0x5593289e9b40 2024-09-19T09:28:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f480247a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81ad000 con 0x5593289e85d0 2024-09-19T09:28:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f480247a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81abb90 con 0x5593289f7780 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f47fde3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e9b40 0x559328a0c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f47fd63c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e85d0 0x5593289f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f47fde3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e9b40 0x559328a0c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60046/0 (socket says 172.21.15.16:60046) 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f47fd63c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e85d0 0x5593289f32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:49866/0 (socket says 172.21.15.16:49866) 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f47fde3d640 1 -- 172.21.15.16:0/3113997606 learned_addr learned my addr 172.21.15.16:0/3113997606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f47fd63c640 1 --2- 172.21.15.16:0/3113997606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289f7780 0x559328a0cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f47fd63c640 1 -- 172.21.15.16:0/3113997606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289f7780 msgr2=0x559328a0cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f47fd63c640 1 --2- 172.21.15.16:0/3113997606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289f7780 0x559328a0cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f47fd63c640 1 -- 172.21.15.16:0/3113997606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e9b40 msgr2=0x559328a0c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f47fd63c640 1 --2- 172.21.15.16:0/3113997606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e9b40 0x559328a0c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f47fd63c640 1 -- 172.21.15.16:0/3113997606 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593289f6d60 con 0x5593289e85d0 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.834+0000 7f47fd63c640 1 --2- 172.21.15.16:0/3113997606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e85d0 0x5593289f32c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47e800cc10 tx=0x7f47e8026040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ea77c5bb10d781a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.835+0000 7f47e37fe640 1 -- 172.21.15.16:0/3113997606 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e802a020 con 0x5593289e85d0 2024-09-19T09:28:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.835+0000 7f47e37fe640 1 -- 172.21.15.16:0/3113997606 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e8037040 con 0x5593289e85d0 2024-09-19T09:28:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.835+0000 7f47e37fe640 1 -- 172.21.15.16:0/3113997606 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8036070 con 0x5593289e85d0 2024-09-19T09:28:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.835+0000 7f480247a740 1 -- 172.21.15.16:0/3113997606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e85d0 msgr2=0x5593289f32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.835+0000 7f480247a740 1 --2- 172.21.15.16:0/3113997606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e85d0 0x5593289f32c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47e800cc10 tx=0x7f47e8026040 comp rx=0 tx=0).stop 2024-09-19T09:28:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.835+0000 7f480247a740 1 -- 172.21.15.16:0/3113997606 shutdown_connections 2024-09-19T09:28:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.835+0000 7f480247a740 1 --2- 172.21.15.16:0/3113997606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289f7780 0x559328a0cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.835+0000 7f480247a740 1 --2- 172.21.15.16:0/3113997606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e9b40 0x559328a0c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.835+0000 7f480247a740 1 --2- 172.21.15.16:0/3113997606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289e85d0 0x5593289f32c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.835+0000 7f480247a740 1 -- 172.21.15.16:0/3113997606 >> 172.21.15.16:0/3113997606 conn(0x5593289b1310 msgr2=0x5593289f2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:09.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.835+0000 7f480247a740 1 -- 172.21.15.16:0/3113997606 shutdown_connections 2024-09-19T09:28:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.836+0000 7f480247a740 1 -- 172.21.15.16:0/3113997606 wait complete. 2024-09-19T09:28:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.836+0000 7f480247a740 1 -- start start 2024-09-19T09:28:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.836+0000 7f480247a740 1 Processor -- start 2024-09-19T09:28:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.836+0000 7f480247a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289e85d0 0x559328a09df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.836+0000 7f480247a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e9b40 0x559328a0a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.836+0000 7f480247a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289f7780 0x559328a69b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.836+0000 7f480247a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81ac930 con 0x5593289e9b40 2024-09-19T09:28:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.836+0000 7f480247a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81ac260 con 0x5593289f7780 2024-09-19T09:28:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.836+0000 7f480247a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47f81a9480 con 0x5593289e85d0 2024-09-19T09:28:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.836+0000 7f480247a740 10 client.0 _ll_register_callbacks cb 0x7f47e800cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47fde3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e9b40 0x559328a0a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47fde3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e9b40 0x559328a0a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60050/0 (socket says 172.21.15.16:60050) 2024-09-19T09:28:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47fde3d640 1 -- 172.21.15.16:0/554967500 learned_addr learned my addr 172.21.15.16:0/554967500 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47fd63c640 1 --2- 172.21.15.16:0/554967500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289f7780 0x559328a69b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47fde3d640 1 -- 172.21.15.16:0/554967500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289e85d0 msgr2=0x559328a09df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47fde3d640 1 --2- 172.21.15.16:0/554967500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289e85d0 0x559328a09df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47fde3d640 1 -- 172.21.15.16:0/554967500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289f7780 msgr2=0x559328a69b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47fde3d640 1 --2- 172.21.15.16:0/554967500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289f7780 0x559328a69b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47fde3d640 1 -- 172.21.15.16:0/554967500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559328a6a200 con 0x5593289e9b40 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47fd63c640 1 --2- 172.21.15.16:0/554967500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289f7780 0x559328a69b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47fde3d640 1 --2- 172.21.15.16:0/554967500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e9b40 0x559328a0a330 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009ca0 tx=0x7f47f402f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0e073153c03e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47e17fa640 1 -- 172.21.15.16:0/554967500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4054060 con 0x5593289e9b40 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47e17fa640 1 -- 172.21.15.16:0/554967500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f47f402f3a0 con 0x5593289e9b40 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f480247a740 1 -- 172.21.15.16:0/554967500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e8037040 con 0x5593289e9b40 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f47e17fa640 1 -- 172.21.15.16:0/554967500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47f4034040 con 0x5593289e9b40 2024-09-19T09:28:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f480247a740 10 client.15537 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.837+0000 7f480247a740 1 -- 172.21.15.16:0/554967500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559328a71660 con 0x5593289e9b40 2024-09-19T09:28:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.838+0000 7f47e17fa640 1 -- 172.21.15.16:0/554967500 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e8037040 con 0x5593289e9b40 2024-09-19T09:28:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.838+0000 7f47e17fa640 1 -- 172.21.15.16:0/554967500 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f47f4059070 con 0x5593289e9b40 2024-09-19T09:28:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.838+0000 7f480247a740 10 client.15537 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.838+0000 7f480247a740 10 client.15537 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-19T09:28:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.838+0000 7f480247a740 20 client.15537 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.838+0000 7f480247a740 1 --2- 172.21.15.16:0/554967500 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559328a07260 0x559328a73e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.838+0000 7f480247a740 4 client.15537 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:28:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.838+0000 7f480247a740 1 -- 172.21.15.16:0/554967500 --> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559328955d70 con 0x559328a07260 2024-09-19T09:28:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.838+0000 7f47fd63c640 1 --2- 172.21.15.16:0/554967500 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559328a07260 0x559328a73e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f47fd63c640 1 --2- 172.21.15.16:0/554967500 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559328a07260 0x559328a73e70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8ce6a01dbe9bbd19 server_cookie=802700a5809d2f56 in_seq=0 out_seq=0 2024-09-19T09:28:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f47e17fa640 10 client.15537 ms_handle_connect on v2:172.21.15.142:6832/3906366829 2024-09-19T09:28:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f47e17fa640 1 -- 172.21.15.16:0/554967500 <== mds.4 v2:172.21.15.142:6832/3906366829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559328955d70 con 0x559328a07260 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f47e17fa640 10 client.15537 handle_command_reply: tid=0 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f480247a740 1 client.15537 shutdown 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f480247a740 1 -- 172.21.15.16:0/554967500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e9b40 msgr2=0x559328a0a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f480247a740 1 --2- 172.21.15.16:0/554967500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e9b40 0x559328a0a330 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009ca0 tx=0x7f47f402f980 comp rx=0 tx=0).stop 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f480247a740 1 -- 172.21.15.16:0/554967500 shutdown_connections 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f480247a740 1 --2- 172.21.15.16:0/554967500 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x559328a07260 0x559328a73e70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f480247a740 1 --2- 172.21.15.16:0/554967500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5593289f7780 0x559328a69b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f47e17fa640 0 client.15537 ms_handle_reset on v2:172.21.15.142:6832/3906366829 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f480247a740 1 --2- 172.21.15.16:0/554967500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593289e9b40 0x559328a0a330 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f480247a740 1 --2- 172.21.15.16:0/554967500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593289e85d0 0x559328a09df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f480247a740 1 -- 172.21.15.16:0/554967500 >> 172.21.15.16:0/554967500 conn(0x5593289b1310 msgr2=0x5593289e3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f480247a740 1 -- 172.21.15.16:0/554967500 shutdown_connections 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.839+0000 7f480247a740 1 -- 172.21.15.16:0/554967500 wait complete. 2024-09-19T09:28:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.840+0000 7f480247a740 20 client.15537 trim_cache size 0 max 16384 2024-09-19T09:28:09.846 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:28:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.840+0000 7f48000a1640 1 -- 172.21.15.16:0/2688277029 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f47e40a3240 msgr2=0x7f47e40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.840+0000 7f48000a1640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f47e40a3240 0x7f47e40a5700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c940 tx=0x7f47f0066580 comp rx=0 tx=0).stop 2024-09-19T09:28:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.840+0000 7f48000a1640 1 -- 172.21.15.16:0/2688277029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f8102ef0 msgr2=0x7f47f81a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.840+0000 7f48000a1640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f8102ef0 0x7f47f81a43d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c970 tx=0x7f47f000ce30 comp rx=0 tx=0).stop 2024-09-19T09:28:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.840+0000 7f48000a1640 1 -- 172.21.15.16:0/2688277029 shutdown_connections 2024-09-19T09:28:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.840+0000 7f48000a1640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f47e40a3240 0x7f47e40a5700 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.840+0000 7f48000a1640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47f8102570 0x7f47f8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.840+0000 7f48000a1640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f8102ef0 0x7f47f81a43d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.840+0000 7f48000a1640 1 --2- 172.21.15.16:0/2688277029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f8100e20 0x7f47f8199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.840+0000 7f48000a1640 1 -- 172.21.15.16:0/2688277029 >> 172.21.15.16:0/2688277029 conn(0x7f47f80fc7d0 msgr2=0x7f47f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.840+0000 7f48000a1640 1 -- 172.21.15.16:0/2688277029 shutdown_connections 2024-09-19T09:28:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:09.840+0000 7f48000a1640 1 -- 172.21.15.16:0/2688277029 wait complete. 2024-09-19T09:28:10.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:28:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:11.349 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:11 smithi016 ceph-mon[30979]: pgmap v355: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:28:11.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:11 smithi142 ceph-mon[35032]: pgmap v355: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:28:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:11 smithi097 ceph-mon[34864]: pgmap v355: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:28:11.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.437+0000 7f2ac7506640 1 -- 172.21.15.16:0/167869646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac01082a0 msgr2=0x7f2ac010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.437+0000 7f2ac7506640 1 --2- 172.21.15.16:0/167869646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac01082a0 0x7f2ac010ec40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009cd0 tx=0x7f2ab802a8e0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.437+0000 7f2ac7506640 1 -- 172.21.15.16:0/167869646 shutdown_connections 2024-09-19T09:28:11.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.437+0000 7f2ac7506640 1 --2- 172.21.15.16:0/167869646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac01036e0 0x7f2ac0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.437+0000 7f2ac7506640 1 --2- 172.21.15.16:0/167869646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac01082a0 0x7f2ac010ec40 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.437+0000 7f2ac7506640 1 --2- 172.21.15.16:0/167869646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ac0102d30 0x7f2ac0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.437+0000 7f2ac7506640 1 -- 172.21.15.16:0/167869646 >> 172.21.15.16:0/167869646 conn(0x7f2ac00fc7d0 msgr2=0x7f2ac00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:11.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.438+0000 7f2ac7506640 1 -- 172.21.15.16:0/167869646 shutdown_connections 2024-09-19T09:28:11.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.438+0000 7f2ac7506640 1 -- 172.21.15.16:0/167869646 wait complete. 2024-09-19T09:28:11.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.438+0000 7f2ac7506640 1 Processor -- start 2024-09-19T09:28:11.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.438+0000 7f2ac7506640 1 -- start start 2024-09-19T09:28:11.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.438+0000 7f2ac7506640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ac0102d30 0x7f2ac01a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac7506640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac01036e0 0x7f2ac01a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac7506640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac01082a0 0x7f2ac019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac7506640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01120f0 con 0x7f2ac01082a0 2024-09-19T09:28:11.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac7506640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0111f70 con 0x7f2ac0102d30 2024-09-19T09:28:11.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac7506640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0112270 con 0x7f2ac01036e0 2024-09-19T09:28:11.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac5aa3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac01082a0 0x7f2ac019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac5aa3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac01082a0 0x7f2ac019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60058/0 (socket says 172.21.15.16:60058) 2024-09-19T09:28:11.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac5aa3640 1 -- 172.21.15.16:0/3047202499 learned_addr learned my addr 172.21.15.16:0/3047202499 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:11.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac01036e0 0x7f2ac01a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac52a2640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ac0102d30 0x7f2ac01a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac5aa3640 1 -- 172.21.15.16:0/3047202499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac01036e0 msgr2=0x7f2ac01a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac5aa3640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac01036e0 0x7f2ac01a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac5aa3640 1 -- 172.21.15.16:0/3047202499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ac0102d30 msgr2=0x7f2ac01a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac5aa3640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ac0102d30 0x7f2ac01a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac5aa3640 1 -- 172.21.15.16:0/3047202499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac019ad20 con 0x7f2ac01082a0 2024-09-19T09:28:11.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.439+0000 7f2ac5aa3640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac01082a0 0x7f2ac019a5f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8033040 tx=0x7f2ab8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.440+0000 7f2ab67fc640 1 -- 172.21.15.16:0/3047202499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8004040 con 0x7f2ac01082a0 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.440+0000 7f2ab67fc640 1 -- 172.21.15.16:0/3047202499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ab80046c0 con 0x7f2ac01082a0 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.440+0000 7f2ac7506640 1 -- 172.21.15.16:0/3047202499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac019afb0 con 0x7f2ac01082a0 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.440+0000 7f2ab67fc640 1 -- 172.21.15.16:0/3047202499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8046480 con 0x7f2ac01082a0 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.440+0000 7f2ac7506640 1 -- 172.21.15.16:0/3047202499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac01a7300 con 0x7f2ac01082a0 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.443+0000 7f2ab67fc640 1 -- 172.21.15.16:0/3047202499 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f2ab80305a0 con 0x7f2ac01082a0 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.443+0000 7f2ac98df740 1 Processor -- start 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.443+0000 7f2ac98df740 1 -- start start 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac98df740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d52327fb40 0x55d52327e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac98df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d52327ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac98df740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d52328b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac98df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d5231e42f0 con 0x55d52327e5d0 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac98df740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d52318bcf0 con 0x55d523289370 2024-09-19T09:28:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac98df740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d5231e02b0 con 0x55d52327fb40 2024-09-19T09:28:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac52a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d52327ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac52a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d52327ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60068/0 (socket says 172.21.15.16:60068) 2024-09-19T09:28:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac52a2640 1 -- 172.21.15.16:0/2560485119 learned_addr learned my addr 172.21.15.16:0/2560485119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/2560485119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d52327fb40 0x55d52327e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/2560485119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d52328b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac52a2640 1 -- 172.21.15.16:0/2560485119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d52327fb40 msgr2=0x55d52327e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac52a2640 1 --2- 172.21.15.16:0/2560485119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d52327fb40 0x55d52327e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac52a2640 1 -- 172.21.15.16:0/2560485119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 msgr2=0x55d52328b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac52a2640 1 --2- 172.21.15.16:0/2560485119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d52328b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.444+0000 7f2ac52a2640 1 -- 172.21.15.16:0/2560485119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5231703a0 con 0x55d52327e5d0 2024-09-19T09:28:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.445+0000 7f2ab67fc640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2aac0a32c0 0x7f2aac0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.445+0000 7f2ac52a2640 1 --2- 172.21.15.16:0/2560485119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d52327ea10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00bcd0 tx=0x7f2abc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8284bef32817dcaa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.445+0000 7f2ab67fc640 1 -- 172.21.15.16:0/3047202499 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2ab80f0950 con 0x7f2ac01082a0 2024-09-19T09:28:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.445+0000 7f2aab7fe640 1 -- 172.21.15.16:0/2560485119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc005220 con 0x55d52327e5d0 2024-09-19T09:28:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.445+0000 7f2aab7fe640 1 -- 172.21.15.16:0/2560485119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2abc004c80 con 0x55d52327e5d0 2024-09-19T09:28:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.445+0000 7f2aab7fe640 1 -- 172.21.15.16:0/2560485119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc0117c0 con 0x55d52327e5d0 2024-09-19T09:28:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.445+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2aac0a32c0 0x7f2aac0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.446+0000 7f2ac98df740 1 -- 172.21.15.16:0/2560485119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 msgr2=0x55d52327ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.446+0000 7f2ac98df740 1 --2- 172.21.15.16:0/2560485119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d52327ea10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00bcd0 tx=0x7f2abc0107a0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.446+0000 7f2ac98df740 1 -- 172.21.15.16:0/2560485119 shutdown_connections 2024-09-19T09:28:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.446+0000 7f2ac98df740 1 --2- 172.21.15.16:0/2560485119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d52328b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.446+0000 7f2ac98df740 1 --2- 172.21.15.16:0/2560485119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d52327ea10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.446+0000 7f2ac98df740 1 --2- 172.21.15.16:0/2560485119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d52327fb40 0x55d52327e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.446+0000 7f2ac98df740 1 -- 172.21.15.16:0/2560485119 >> 172.21.15.16:0/2560485119 conn(0x55d523247310 msgr2=0x55d52327ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.446+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2aac0a32c0 0x7f2aac0a5780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55d52327f070 tx=0x7f2ab00046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.446+0000 7f2ac98df740 1 -- 172.21.15.16:0/2560485119 shutdown_connections 2024-09-19T09:28:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.446+0000 7f2ac98df740 1 -- 172.21.15.16:0/2560485119 wait complete. 2024-09-19T09:28:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.447+0000 7f2ac98df740 1 -- start start 2024-09-19T09:28:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.447+0000 7f2ac98df740 1 Processor -- start 2024-09-19T09:28:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.447+0000 7f2ac98df740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d52327e5d0 0x55d523288cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.447+0000 7f2ac98df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327fb40 0x55d523285fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac98df740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523286520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac98df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d52328c0c0 con 0x55d52327fb40 2024-09-19T09:28:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac98df740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d52319f060 con 0x55d523289370 2024-09-19T09:28:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac98df740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d52328c240 con 0x55d52327e5d0 2024-09-19T09:28:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac4aa1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327fb40 0x55d523285fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac4aa1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327fb40 0x55d523285fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60080/0 (socket says 172.21.15.16:60080) 2024-09-19T09:28:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac4aa1640 1 -- 172.21.15.16:0/3621296628 learned_addr learned my addr 172.21.15.16:0/3621296628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac98df740 10 client.0 _ll_register_callbacks cb 0x55d522ef28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac52a2640 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d52327e5d0 0x55d523288cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac5aa3640 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523286520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac4aa1640 1 -- 172.21.15.16:0/3621296628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d52327e5d0 msgr2=0x55d523288cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d52327e5d0 0x55d523288cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac4aa1640 1 -- 172.21.15.16:0/3621296628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 msgr2=0x55d523286520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523286520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac4aa1640 1 -- 172.21.15.16:0/3621296628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d522f553c0 con 0x55d52327fb40 2024-09-19T09:28:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac5aa3640 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523286520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327fb40 0x55d523285fa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000bd70 tx=0x7f2ab0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee9dfb71d1906b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.448+0000 7f2aa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.449+0000 7f2aa8ff9640 1 -- 172.21.15.16:0/3621296628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab002d060 con 0x55d52327fb40 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.449+0000 7f2ac98df740 1 -- 172.21.15.16:0/3621296628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2abc004c80 con 0x55d52327fb40 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.449+0000 7f2aa8ff9640 1 -- 172.21.15.16:0/3621296628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2ab00207d0 con 0x55d52327fb40 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.449+0000 7f2aa8ff9640 1 -- 172.21.15.16:0/3621296628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ab00209d0 con 0x55d52327fb40 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.449+0000 7f2ac98df740 10 client.15549 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.449+0000 7f2ac98df740 1 -- 172.21.15.16:0/3621296628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5232a03e0 con 0x55d52327fb40 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.449+0000 7f2aa8ff9640 1 -- 172.21.15.16:0/3621296628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2abc004c80 con 0x55d52327fb40 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.449+0000 7f2aa8ff9640 1 -- 172.21.15.16:0/3621296628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2ab0032070 con 0x55d52327fb40 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.450+0000 7f2ac98df740 10 client.15549 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.450+0000 7f2ac98df740 10 client.15549 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.450+0000 7f2ac98df740 20 client.15549 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.450+0000 7f2ac98df740 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55d5232a1c30 0x55d5232a4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.450+0000 7f2ac98df740 4 client.15549 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.450+0000 7f2ac98df740 1 -- 172.21.15.16:0/3621296628 --> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5231f1860 con 0x55d5232a1c30 2024-09-19T09:28:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.450+0000 7f2ac52a2640 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55d5232a1c30 0x55d5232a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.451+0000 7f2aa8ff9640 10 client.15549 ms_handle_connect on v2:172.21.15.97:6832/108652274 2024-09-19T09:28:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.451+0000 7f2ac52a2640 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55d5232a1c30 0x55d5232a4090 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=24875f0e54c64cd0 server_cookie=c7d045f55603f544 in_seq=0 out_seq=0 2024-09-19T09:28:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.451+0000 7f2aa8ff9640 1 -- 172.21.15.16:0/3621296628 <== mds.3 v2:172.21.15.97:6832/108652274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5231f1860 con 0x55d5232a1c30 2024-09-19T09:28:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.451+0000 7f2aa8ff9640 10 client.15549 handle_command_reply: tid=0 2024-09-19T09:28:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.451+0000 7f2ac98df740 1 client.15549 shutdown 2024-09-19T09:28:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.452+0000 7f2ac98df740 1 -- 172.21.15.16:0/3621296628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327fb40 msgr2=0x55d523285fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.452+0000 7f2ac98df740 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327fb40 0x55d523285fa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000bd70 tx=0x7f2ab0004730 comp rx=0 tx=0).stop 2024-09-19T09:28:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.452+0000 7f2ac98df740 1 -- 172.21.15.16:0/3621296628 shutdown_connections 2024-09-19T09:28:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.452+0000 7f2ac98df740 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55d5232a1c30 0x55d5232a4090 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.452+0000 7f2ac98df740 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523286520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.452+0000 7f2ac98df740 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327fb40 0x55d523285fa0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.452+0000 7f2ac98df740 1 --2- 172.21.15.16:0/3621296628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d52327e5d0 0x55d523288cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.452+0000 7f2ac98df740 1 -- 172.21.15.16:0/3621296628 >> 172.21.15.16:0/3621296628 conn(0x55d523247310 msgr2=0x55d523280930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:11.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.452+0000 7f2aa8ff9640 0 client.15549 ms_handle_reset on v2:172.21.15.97:6832/108652274 2024-09-19T09:28:11.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.452+0000 7f2ac98df740 1 -- 172.21.15.16:0/3621296628 shutdown_connections 2024-09-19T09:28:11.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.452+0000 7f2ac98df740 1 -- 172.21.15.16:0/3621296628 wait complete. 2024-09-19T09:28:11.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.452+0000 7f2ac98df740 20 client.15549 trim_cache size 0 max 16384 2024-09-19T09:28:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.466+0000 7f2ac98df740 1 Processor -- start 2024-09-19T09:28:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.466+0000 7f2ac98df740 1 -- start start 2024-09-19T09:28:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.466+0000 7f2ac98df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d523287f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.466+0000 7f2ac98df740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523281120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.466+0000 7f2ac98df740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d523288450 0x55d523281660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.466+0000 7f2ac98df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01ab480 con 0x55d52327e5d0 2024-09-19T09:28:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.466+0000 7f2ac98df740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01acfc0 con 0x55d523289370 2024-09-19T09:28:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.466+0000 7f2ac98df740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01abb50 con 0x55d523288450 2024-09-19T09:28:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.466+0000 7f2ac52a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d523287f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2ac52a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d523287f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60090/0 (socket says 172.21.15.16:60090) 2024-09-19T09:28:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2ac52a2640 1 -- 172.21.15.16:0/2832283560 learned_addr learned my addr 172.21.15.16:0/2832283560 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2ac5aa3640 1 --2- 172.21.15.16:0/2832283560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d523288450 0x55d523281660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/2832283560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523281120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2ac52a2640 1 -- 172.21.15.16:0/2832283560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d523288450 msgr2=0x55d523281660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2ac52a2640 1 --2- 172.21.15.16:0/2832283560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d523288450 0x55d523281660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2ac52a2640 1 -- 172.21.15.16:0/2832283560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 msgr2=0x55d523281120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2ac52a2640 1 --2- 172.21.15.16:0/2832283560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523281120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2ac52a2640 1 -- 172.21.15.16:0/2832283560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5232a4710 con 0x55d52327e5d0 2024-09-19T09:28:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/2832283560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523281120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2ac52a2640 1 --2- 172.21.15.16:0/2832283560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d523287f10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2abc011010 tx=0x7f2abc00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdf28c4e061556f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2aaaffd640 1 -- 172.21.15.16:0/2832283560 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc005200 con 0x55d52327e5d0 2024-09-19T09:28:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2aaaffd640 1 -- 172.21.15.16:0/2832283560 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2abc004b40 con 0x55d52327e5d0 2024-09-19T09:28:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.467+0000 7f2aaaffd640 1 -- 172.21.15.16:0/2832283560 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc029850 con 0x55d52327e5d0 2024-09-19T09:28:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.468+0000 7f2ac98df740 1 -- 172.21.15.16:0/2832283560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 msgr2=0x55d523287f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.468+0000 7f2ac98df740 1 --2- 172.21.15.16:0/2832283560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d523287f10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2abc011010 tx=0x7f2abc00b410 comp rx=0 tx=0).stop 2024-09-19T09:28:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.468+0000 7f2ac98df740 1 -- 172.21.15.16:0/2832283560 shutdown_connections 2024-09-19T09:28:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.468+0000 7f2ac98df740 1 --2- 172.21.15.16:0/2832283560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d523288450 0x55d523281660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.468+0000 7f2ac98df740 1 --2- 172.21.15.16:0/2832283560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523281120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.468+0000 7f2ac98df740 1 --2- 172.21.15.16:0/2832283560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d523287f10 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.468+0000 7f2ac98df740 1 -- 172.21.15.16:0/2832283560 >> 172.21.15.16:0/2832283560 conn(0x55d523247310 msgr2=0x55d523287800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.469+0000 7f2ac98df740 1 -- 172.21.15.16:0/2832283560 shutdown_connections 2024-09-19T09:28:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.469+0000 7f2ac98df740 1 -- 172.21.15.16:0/2832283560 wait complete. 2024-09-19T09:28:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.469+0000 7f2ac98df740 1 -- start start 2024-09-19T09:28:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.469+0000 7f2ac98df740 1 Processor -- start 2024-09-19T09:28:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.469+0000 7f2ac98df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d523283170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.469+0000 7f2ac98df740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d523288450 0x55d5232836b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac98df740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523283bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac52a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d523283170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac98df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01ac8f0 con 0x55d52327e5d0 2024-09-19T09:28:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac98df740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01ac220 con 0x55d523289370 2024-09-19T09:28:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac98df740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01a9440 con 0x55d523288450 2024-09-19T09:28:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac52a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d523283170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60096/0 (socket says 172.21.15.16:60096) 2024-09-19T09:28:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac52a2640 1 -- 172.21.15.16:0/3505767931 learned_addr learned my addr 172.21.15.16:0/3505767931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac98df740 10 client.0 _ll_register_callbacks cb 0x7f2ab0011f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d523288450 0x55d5232836b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac5aa3640 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523283bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac4aa1640 1 -- 172.21.15.16:0/3505767931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 msgr2=0x55d523283bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523283bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac4aa1640 1 -- 172.21.15.16:0/3505767931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 msgr2=0x55d523283170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d523283170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac4aa1640 1 -- 172.21.15.16:0/3505767931 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d52330bb10 con 0x55d523288450 2024-09-19T09:28:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac52a2640 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d523283170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2ac4aa1640 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d523288450 0x55d5232836b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55d523287090 tx=0x7f2ab0020e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2dc1fa657d61e49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.470+0000 7f2aa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-19T09:28:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2aa8ff9640 1 -- 172.21.15.16:0/3505767931 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0038030 con 0x55d523288450 2024-09-19T09:28:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2aa8ff9640 1 -- 172.21.15.16:0/3505767931 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2ab00207d0 con 0x55d523288450 2024-09-19T09:28:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2ac98df740 1 -- 172.21.15.16:0/3505767931 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2abc004b40 con 0x55d523288450 2024-09-19T09:28:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2aa8ff9640 1 -- 172.21.15.16:0/3505767931 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ab002d020 con 0x55d523288450 2024-09-19T09:28:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2ac98df740 10 client.25129 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2ac98df740 1 -- 172.21.15.16:0/3505767931 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d52330cd00 con 0x55d523288450 2024-09-19T09:28:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2aa8ff9640 1 -- 172.21.15.16:0/3505767931 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ab0033de0 con 0x55d523288450 2024-09-19T09:28:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2aa8ff9640 1 -- 172.21.15.16:0/3505767931 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2ab0032050 con 0x55d523288450 2024-09-19T09:28:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2ac98df740 10 client.25129 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2ac98df740 10 client.25129 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-19T09:28:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2ac98df740 20 client.25129 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2ac98df740 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55d52330d420 0x55d52330f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2ac98df740 4 client.25129 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:28:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.471+0000 7f2ac98df740 1 -- 172.21.15.16:0/3505767931 --> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2abc004b40 con 0x55d52330d420 2024-09-19T09:28:11.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.472+0000 7f2ac52a2640 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55d52330d420 0x55d52330f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.472+0000 7f2ac52a2640 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55d52330d420 0x55d52330f810 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a56b0c29982d775d server_cookie=2b4036fbdcb38882 in_seq=0 out_seq=0 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.472+0000 7f2aa8ff9640 10 client.25129 ms_handle_connect on v2:172.21.15.97:6832/108652274 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.472+0000 7f2aa8ff9640 1 -- 172.21.15.16:0/3505767931 <== mds.3 v2:172.21.15.97:6832/108652274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2abc004b40 con 0x55d52330d420 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.472+0000 7f2aa8ff9640 10 client.25129 handle_command_reply: tid=0 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.472+0000 7f2ac98df740 1 client.25129 shutdown 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.472+0000 7f2ac98df740 1 -- 172.21.15.16:0/3505767931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d523288450 msgr2=0x55d5232836b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.472+0000 7f2ac98df740 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d523288450 0x55d5232836b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55d523287090 tx=0x7f2ab0020e20 comp rx=0 tx=0).stop 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.473+0000 7f2ac98df740 1 -- 172.21.15.16:0/3505767931 shutdown_connections 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.473+0000 7f2ac98df740 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55d52330d420 0x55d52330f810 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.473+0000 7f2ac98df740 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d523288450 0x55d5232836b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.473+0000 7f2ac98df740 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d523289370 0x55d523283bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.473+0000 7f2ac98df740 1 --2- 172.21.15.16:0/3505767931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d52327e5d0 0x55d523283170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.473+0000 7f2ac98df740 1 -- 172.21.15.16:0/3505767931 >> 172.21.15.16:0/3505767931 conn(0x55d523247310 msgr2=0x55d5232794e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:11.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.473+0000 7f2aa8ff9640 0 client.25129 ms_handle_reset on v2:172.21.15.97:6832/108652274 2024-09-19T09:28:11.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.473+0000 7f2ac98df740 1 -- 172.21.15.16:0/3505767931 shutdown_connections 2024-09-19T09:28:11.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.473+0000 7f2ac98df740 1 -- 172.21.15.16:0/3505767931 wait complete. 2024-09-19T09:28:11.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.473+0000 7f2ac98df740 20 client.25129 trim_cache size 0 max 16384 2024-09-19T09:28:11.487 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:28:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.474+0000 7f2ac7506640 1 -- 172.21.15.16:0/3047202499 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2aac0a32c0 msgr2=0x7f2aac0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.474+0000 7f2ac7506640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2aac0a32c0 0x7f2aac0a5780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55d52327f070 tx=0x7f2ab00046c0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.474+0000 7f2ac7506640 1 -- 172.21.15.16:0/3047202499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac01082a0 msgr2=0x7f2ac019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.474+0000 7f2ac7506640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac01082a0 0x7f2ac019a5f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8033040 tx=0x7f2ab8003970 comp rx=0 tx=0).stop 2024-09-19T09:28:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.474+0000 7f2ac7506640 1 -- 172.21.15.16:0/3047202499 shutdown_connections 2024-09-19T09:28:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.474+0000 7f2ac7506640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2aac0a32c0 0x7f2aac0a5780 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.474+0000 7f2ac7506640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac01036e0 0x7f2ac01a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.474+0000 7f2ac7506640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac01082a0 0x7f2ac019a5f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.474+0000 7f2ac7506640 1 --2- 172.21.15.16:0/3047202499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ac0102d30 0x7f2ac01a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.474+0000 7f2ac7506640 1 -- 172.21.15.16:0/3047202499 >> 172.21.15.16:0/3047202499 conn(0x7f2ac00fc7d0 msgr2=0x7f2ac0101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.474+0000 7f2ac7506640 1 -- 172.21.15.16:0/3047202499 shutdown_connections 2024-09-19T09:28:11.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:11.474+0000 7f2ac7506640 1 -- 172.21.15.16:0/3047202499 wait complete. 2024-09-19T09:28:12.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:28:12.573 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.132+0000 7f687b3fa640 1 -- 172.21.15.16:0/831973658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6874102570 msgr2=0x7f68741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.132+0000 7f687b3fa640 1 --2- 172.21.15.16:0/831973658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6874102570 0x7f68741029b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6864009900 tx=0x7f686402a480 comp rx=0 tx=0).stop 2024-09-19T09:28:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.132+0000 7f687b3fa640 1 -- 172.21.15.16:0/831973658 shutdown_connections 2024-09-19T09:28:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.132+0000 7f687b3fa640 1 --2- 172.21.15.16:0/831973658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6874102ef0 0x7f687410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.132+0000 7f687b3fa640 1 --2- 172.21.15.16:0/831973658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6874102570 0x7f68741029b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.132+0000 7f687b3fa640 1 --2- 172.21.15.16:0/831973658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6874100e20 0x7f6874101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.132+0000 7f687b3fa640 1 -- 172.21.15.16:0/831973658 >> 172.21.15.16:0/831973658 conn(0x7f68740fc7d0 msgr2=0x7f68740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.133+0000 7f687b3fa640 1 -- 172.21.15.16:0/831973658 shutdown_connections 2024-09-19T09:28:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.133+0000 7f687b3fa640 1 -- 172.21.15.16:0/831973658 wait complete. 2024-09-19T09:28:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.134+0000 7f687b3fa640 1 Processor -- start 2024-09-19T09:28:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.134+0000 7f687b3fa640 1 -- start start 2024-09-19T09:28:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.134+0000 7f687b3fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6874100e20 0x7f6874198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.134+0000 7f687b3fa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6874102570 0x7f6874198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.134+0000 7f687b3fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6874102ef0 0x7f687419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.134+0000 7f687b3fa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6874110f50 con 0x7f6874100e20 2024-09-19T09:28:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.134+0000 7f687b3fa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6874110dd0 con 0x7f6874102ef0 2024-09-19T09:28:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.134+0000 7f687b3fa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f68741110d0 con 0x7f6874102570 2024-09-19T09:28:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.134+0000 7f6879196640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6874100e20 0x7f6874198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.134+0000 7f6879196640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6874100e20 0x7f6874198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60122/0 (socket says 172.21.15.16:60122) 2024-09-19T09:28:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.134+0000 7f6879196640 1 -- 172.21.15.16:0/1099333974 learned_addr learned my addr 172.21.15.16:0/1099333974 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.135+0000 7f6879997640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6874102ef0 0x7f687419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.135+0000 7f6878995640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6874102570 0x7f6874198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.135+0000 7f6879196640 1 -- 172.21.15.16:0/1099333974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6874102570 msgr2=0x7f6874198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.135+0000 7f6879196640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6874102570 0x7f6874198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.135+0000 7f6879196640 1 -- 172.21.15.16:0/1099333974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6874102ef0 msgr2=0x7f687419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.135+0000 7f6879196640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6874102ef0 0x7f687419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.135+0000 7f6879196640 1 -- 172.21.15.16:0/1099333974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687419d500 con 0x7f6874100e20 2024-09-19T09:28:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.135+0000 7f6879196640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6874100e20 0x7f6874198550 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f687000d950 tx=0x7f687000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.135+0000 7f686a7fc640 1 -- 172.21.15.16:0/1099333974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870017070 con 0x7f6874100e20 2024-09-19T09:28:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.135+0000 7f687b3fa640 1 -- 172.21.15.16:0/1099333974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687419d7f0 con 0x7f6874100e20 2024-09-19T09:28:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.135+0000 7f686a7fc640 1 -- 172.21.15.16:0/1099333974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6870009c30 con 0x7f6874100e20 2024-09-19T09:28:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.135+0000 7f686a7fc640 1 -- 172.21.15.16:0/1099333974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870017070 con 0x7f6874100e20 2024-09-19T09:28:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.136+0000 7f687b3fa640 1 -- 172.21.15.16:0/1099333974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68740fcbb0 con 0x7f6874100e20 2024-09-19T09:28:13.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.139+0000 7f687d7d3740 1 Processor -- start 2024-09-19T09:28:13.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.139+0000 7f686a7fc640 1 -- 172.21.15.16:0/1099333974 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f68700109d0 con 0x7f6874100e20 2024-09-19T09:28:13.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.140+0000 7f686a7fc640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f68600a34c0 0x7f68600a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f686a7fc640 1 -- 172.21.15.16:0/1099333974 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f68700cee70 con 0x7f6874100e20 2024-09-19T09:28:13.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f6879997640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f68600a34c0 0x7f68600a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f687d7d3740 1 -- start start 2024-09-19T09:28:13.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f687d7d3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653f350 0x56361653d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f687d7d3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56361653dde0 0x56361653e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f687d7d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563616548b80 0x56361654af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f687d7d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56361649b2f0 con 0x563616548b80 2024-09-19T09:28:13.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f687d7d3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563616442cf0 con 0x56361653f350 2024-09-19T09:28:13.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f687d7d3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5636164972b0 con 0x56361653dde0 2024-09-19T09:28:13.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f6879196640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653f350 0x56361653d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f6879997640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563616548b80 0x56361654af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f6879997640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563616548b80 0x56361654af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60130/0 (socket says 172.21.15.16:60130) 2024-09-19T09:28:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f6879196640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653f350 0x56361653d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:49932/0 (socket says 172.21.15.16:49932) 2024-09-19T09:28:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.141+0000 7f6879997640 1 -- 172.21.15.16:0/3846097397 learned_addr learned my addr 172.21.15.16:0/3846097397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.142+0000 7f6879997640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f68600a34c0 0x7f68600a5980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f686c0092f0 tx=0x7f686c009280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.142+0000 7f6878995640 1 --2- 172.21.15.16:0/3846097397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56361653dde0 0x56361653e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.142+0000 7f6879997640 1 -- 172.21.15.16:0/3846097397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56361653dde0 msgr2=0x56361653e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:13.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.142+0000 7f6879997640 1 --2- 172.21.15.16:0/3846097397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56361653dde0 0x56361653e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.142+0000 7f6879997640 1 -- 172.21.15.16:0/3846097397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653f350 msgr2=0x56361653d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.142+0000 7f6879997640 1 --2- 172.21.15.16:0/3846097397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653f350 0x56361653d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.142+0000 7f6879997640 1 -- 172.21.15.16:0/3846097397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636164273a0 con 0x563616548b80 2024-09-19T09:28:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.142+0000 7f6879196640 1 --2- 172.21.15.16:0/3846097397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653f350 0x56361653d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.142+0000 7f6879997640 1 --2- 172.21.15.16:0/3846097397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563616548b80 0x56361654af70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f686c012580 tx=0x7f686c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbbc8cfd00d3b86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.142+0000 7f685f7fe640 1 -- 172.21.15.16:0/3846097397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c022030 con 0x563616548b80 2024-09-19T09:28:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.143+0000 7f685f7fe640 1 -- 172.21.15.16:0/3846097397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686c011980 con 0x563616548b80 2024-09-19T09:28:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.143+0000 7f687d7d3740 1 -- 172.21.15.16:0/3846097397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563616548b80 msgr2=0x56361654af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.143+0000 7f685f7fe640 1 -- 172.21.15.16:0/3846097397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c01ea30 con 0x563616548b80 2024-09-19T09:28:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.143+0000 7f687d7d3740 1 --2- 172.21.15.16:0/3846097397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563616548b80 0x56361654af70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f686c012580 tx=0x7f686c013000 comp rx=0 tx=0).stop 2024-09-19T09:28:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.144+0000 7f687d7d3740 1 -- 172.21.15.16:0/3846097397 shutdown_connections 2024-09-19T09:28:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.144+0000 7f687d7d3740 1 --2- 172.21.15.16:0/3846097397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56361653dde0 0x56361653e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.144+0000 7f687d7d3740 1 --2- 172.21.15.16:0/3846097397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563616548b80 0x56361654af70 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.144+0000 7f687d7d3740 1 --2- 172.21.15.16:0/3846097397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653f350 0x56361653d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.144+0000 7f687d7d3740 1 -- 172.21.15.16:0/3846097397 >> 172.21.15.16:0/3846097397 conn(0x5636164fe310 msgr2=0x56361653a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.144+0000 7f687d7d3740 1 -- 172.21.15.16:0/3846097397 shutdown_connections 2024-09-19T09:28:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.144+0000 7f687d7d3740 1 -- 172.21.15.16:0/3846097397 wait complete. 2024-09-19T09:28:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.144+0000 7f687d7d3740 1 -- start start 2024-09-19T09:28:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.144+0000 7f687d7d3740 1 Processor -- start 2024-09-19T09:28:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f687d7d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653dde0 0x563616539980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f687d7d3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56361653f350 0x563616547b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f687d7d3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563616548b80 0x563616548300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f687d7d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563616531aa0 con 0x56361653dde0 2024-09-19T09:28:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f687d7d3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563616456060 con 0x563616548b80 2024-09-19T09:28:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f687d7d3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563616531c20 con 0x56361653f350 2024-09-19T09:28:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f687d7d3740 10 client.0 _ll_register_callbacks cb 0x5636161a98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f6879196640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653dde0 0x563616539980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f6879196640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653dde0 0x563616539980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60132/0 (socket says 172.21.15.16:60132) 2024-09-19T09:28:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f6879196640 1 -- 172.21.15.16:0/2786597128 learned_addr learned my addr 172.21.15.16:0/2786597128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f6878995640 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56361653f350 0x563616547b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f6879997640 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563616548b80 0x563616548300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f6879196640 1 -- 172.21.15.16:0/2786597128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56361653f350 msgr2=0x563616547b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f6879196640 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56361653f350 0x563616547b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f6879196640 1 -- 172.21.15.16:0/2786597128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563616548b80 msgr2=0x563616548300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:13.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f6879196640 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563616548b80 0x563616548300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f6879196640 1 -- 172.21.15.16:0/2786597128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56361620c3c0 con 0x56361653dde0 2024-09-19T09:28:13.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.145+0000 7f6879997640 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563616548b80 0x563616548300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:13.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.146+0000 7f6879196640 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653dde0 0x563616539980 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6870097e20 tx=0x7f687009b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b25d09e12e7df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.146+0000 7f685cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.146+0000 7f685cff9640 1 -- 172.21.15.16:0/2786597128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68700b3020 con 0x56361653dde0 2024-09-19T09:28:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.146+0000 7f687d7d3740 1 -- 172.21.15.16:0/2786597128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f686c011980 con 0x56361653dde0 2024-09-19T09:28:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.146+0000 7f685cff9640 1 -- 172.21.15.16:0/2786597128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f6870099750 con 0x56361653dde0 2024-09-19T09:28:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.146+0000 7f685cff9640 1 -- 172.21.15.16:0/2786597128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6870020040 con 0x56361653dde0 2024-09-19T09:28:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.146+0000 7f687d7d3740 10 client.15567 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.146+0000 7f687d7d3740 1 -- 172.21.15.16:0/2786597128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56361655fc30 con 0x56361653dde0 2024-09-19T09:28:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.146+0000 7f685cff9640 1 -- 172.21.15.16:0/2786597128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686c011980 con 0x56361653dde0 2024-09-19T09:28:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.147+0000 7f685cff9640 1 -- 172.21.15.16:0/2786597128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6870023060 con 0x56361653dde0 2024-09-19T09:28:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.147+0000 7f687d7d3740 10 client.15567 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.147+0000 7f687d7d3740 10 client.15567 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T09:28:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.147+0000 7f687d7d3740 20 client.15567 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.147+0000 7f687d7d3740 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x563616561040 0x563616563430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.147+0000 7f687d7d3740 4 client.15567 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.147+0000 7f687d7d3740 1 -- 172.21.15.16:0/2786597128 --> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636164a8860 con 0x563616561040 2024-09-19T09:28:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.147+0000 7f6878995640 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x563616561040 0x563616563430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.148+0000 7f6878995640 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x563616561040 0x563616563430 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc04ec12fe3d5841 server_cookie=ab122d8e409391b6 in_seq=0 out_seq=0 2024-09-19T09:28:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.148+0000 7f685cff9640 10 client.15567 ms_handle_connect on v2:172.21.15.97:6836/2018284957 2024-09-19T09:28:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.149+0000 7f685cff9640 1 -- 172.21.15.16:0/2786597128 <== mds.2 v2:172.21.15.97:6836/2018284957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636164a8860 con 0x563616561040 2024-09-19T09:28:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.149+0000 7f685cff9640 10 client.15567 handle_command_reply: tid=0 2024-09-19T09:28:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.149+0000 7f687d7d3740 1 client.15567 shutdown 2024-09-19T09:28:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.149+0000 7f687d7d3740 1 -- 172.21.15.16:0/2786597128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653dde0 msgr2=0x563616539980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.149+0000 7f687d7d3740 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653dde0 0x563616539980 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6870097e20 tx=0x7f687009b450 comp rx=0 tx=0).stop 2024-09-19T09:28:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.149+0000 7f687d7d3740 1 -- 172.21.15.16:0/2786597128 shutdown_connections 2024-09-19T09:28:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.149+0000 7f687d7d3740 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x563616561040 0x563616563430 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.149+0000 7f685cff9640 0 client.15567 ms_handle_reset on v2:172.21.15.97:6836/2018284957 2024-09-19T09:28:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.149+0000 7f687d7d3740 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56361653f350 0x563616547b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.149+0000 7f687d7d3740 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563616548b80 0x563616548300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.149+0000 7f687d7d3740 1 --2- 172.21.15.16:0/2786597128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653dde0 0x563616539980 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.149+0000 7f687d7d3740 1 -- 172.21.15.16:0/2786597128 >> 172.21.15.16:0/2786597128 conn(0x5636164fe310 msgr2=0x563616540170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.150+0000 7f687d7d3740 1 -- 172.21.15.16:0/2786597128 shutdown_connections 2024-09-19T09:28:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.150+0000 7f687d7d3740 1 -- 172.21.15.16:0/2786597128 wait complete. 2024-09-19T09:28:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.150+0000 7f687d7d3740 20 client.15567 trim_cache size 0 max 16384 2024-09-19T09:28:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.176+0000 7f687d7d3740 1 Processor -- start 2024-09-19T09:28:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.176+0000 7f687d7d3740 1 -- start start 2024-09-19T09:28:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.176+0000 7f687d7d3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653dde0 0x563616568c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.176+0000 7f687d7d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 0x5636165691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.177+0000 7f687d7d3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563616548b80 0x563616533e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.177+0000 7f687d7d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68741ab490 con 0x56361653f350 2024-09-19T09:28:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.177+0000 7f687d7d3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68741acfd0 con 0x56361653dde0 2024-09-19T09:28:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.177+0000 7f687d7d3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f68741abb60 con 0x563616548b80 2024-09-19T09:28:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.177+0000 7f6878995640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 0x5636165691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.177+0000 7f6878995640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 0x5636165691a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60134/0 (socket says 172.21.15.16:60134) 2024-09-19T09:28:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.177+0000 7f6878995640 1 -- 172.21.15.16:0/1446837778 learned_addr learned my addr 172.21.15.16:0/1446837778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.177+0000 7f6879997640 1 --2- 172.21.15.16:0/1446837778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563616548b80 0x563616533e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.177+0000 7f6879196640 1 --2- 172.21.15.16:0/1446837778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653dde0 0x563616568c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.177+0000 7f6879997640 1 -- 172.21.15.16:0/1446837778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653dde0 msgr2=0x563616568c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.178+0000 7f6879997640 1 --2- 172.21.15.16:0/1446837778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653dde0 0x563616568c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.178+0000 7f6879997640 1 -- 172.21.15.16:0/1446837778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 msgr2=0x5636165691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.178+0000 7f6879997640 1 --2- 172.21.15.16:0/1446837778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 0x5636165691a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.178+0000 7f6879997640 1 -- 172.21.15.16:0/1446837778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563616548630 con 0x563616548b80 2024-09-19T09:28:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.178+0000 7f6878995640 1 --2- 172.21.15.16:0/1446837778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 0x5636165691a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.178+0000 7f6879196640 1 --2- 172.21.15.16:0/1446837778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653dde0 0x563616568c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.178+0000 7f6879997640 1 --2- 172.21.15.16:0/1446837778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563616548b80 0x563616533e90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f686c012e10 tx=0x7f686c010c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ebbcf4c1d5f89cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.178+0000 7f685effd640 1 -- 172.21.15.16:0/1446837778 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c016030 con 0x563616548b80 2024-09-19T09:28:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.178+0000 7f685effd640 1 -- 172.21.15.16:0/1446837778 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686c023040 con 0x563616548b80 2024-09-19T09:28:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.178+0000 7f685effd640 1 -- 172.21.15.16:0/1446837778 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c022070 con 0x563616548b80 2024-09-19T09:28:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.179+0000 7f687d7d3740 1 -- 172.21.15.16:0/1446837778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563616548b80 msgr2=0x563616533e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.179+0000 7f687d7d3740 1 --2- 172.21.15.16:0/1446837778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563616548b80 0x563616533e90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f686c012e10 tx=0x7f686c010c90 comp rx=0 tx=0).stop 2024-09-19T09:28:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.179+0000 7f687d7d3740 1 -- 172.21.15.16:0/1446837778 shutdown_connections 2024-09-19T09:28:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.179+0000 7f687d7d3740 1 --2- 172.21.15.16:0/1446837778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563616548b80 0x563616533e90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.179+0000 7f687d7d3740 1 --2- 172.21.15.16:0/1446837778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 0x5636165691a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.179+0000 7f687d7d3740 1 --2- 172.21.15.16:0/1446837778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653dde0 0x563616568c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.179+0000 7f687d7d3740 1 -- 172.21.15.16:0/1446837778 >> 172.21.15.16:0/1446837778 conn(0x5636164fe310 msgr2=0x56361653a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.180+0000 7f687d7d3740 1 -- 172.21.15.16:0/1446837778 shutdown_connections 2024-09-19T09:28:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.180+0000 7f687d7d3740 1 -- 172.21.15.16:0/1446837778 wait complete. 2024-09-19T09:28:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.180+0000 7f687d7d3740 1 -- start start 2024-09-19T09:28:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.180+0000 7f687d7d3740 1 Processor -- start 2024-09-19T09:28:13.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.181+0000 7f687d7d3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653dde0 0x563616542640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.181+0000 7f687d7d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 0x563616542b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f687d7d3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563616548b80 0x5636165430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f6878995640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 0x563616542b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f687d7d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68741ac900 con 0x56361653f350 2024-09-19T09:28:13.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f6878995640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 0x563616542b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60140/0 (socket says 172.21.15.16:60140) 2024-09-19T09:28:13.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f6878995640 1 -- 172.21.15.16:0/306703604 learned_addr learned my addr 172.21.15.16:0/306703604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:13.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f687d7d3740 1 -- 172.21.15.16:0/306703604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68741ac230 con 0x56361653dde0 2024-09-19T09:28:13.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f687d7d3740 1 -- 172.21.15.16:0/306703604 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f68741a9450 con 0x563616548b80 2024-09-19T09:28:13.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f687d7d3740 10 client.0 _ll_register_callbacks cb 0x7f687009bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:13.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f6879196640 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653dde0 0x563616542640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f6879997640 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563616548b80 0x5636165430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f6879196640 1 -- 172.21.15.16:0/306703604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563616548b80 msgr2=0x5636165430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f6879196640 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563616548b80 0x5636165430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f6879196640 1 -- 172.21.15.16:0/306703604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 msgr2=0x563616542b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f6879196640 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 0x563616542b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.182+0000 7f6879196640 1 -- 172.21.15.16:0/306703604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636165cac40 con 0x56361653dde0 2024-09-19T09:28:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.183+0000 7f6878995640 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 0x563616542b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.183+0000 7f6879196640 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653dde0 0x563616542640 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f687009b3e0 tx=0x7f687000df40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d304bd4e6c414d22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.183+0000 7f685cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-19T09:28:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.183+0000 7f685cff9640 1 -- 172.21.15.16:0/306703604 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68700b3070 con 0x56361653dde0 2024-09-19T09:28:13.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.183+0000 7f685cff9640 1 -- 172.21.15.16:0/306703604 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f687009a420 con 0x56361653dde0 2024-09-19T09:28:13.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.183+0000 7f687d7d3740 1 -- 172.21.15.16:0/306703604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f686c023040 con 0x56361653dde0 2024-09-19T09:28:13.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.184+0000 7f685cff9640 1 -- 172.21.15.16:0/306703604 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f687009a620 con 0x56361653dde0 2024-09-19T09:28:13.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.184+0000 7f687d7d3740 10 client.25202 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:13.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.184+0000 7f687d7d3740 1 -- 172.21.15.16:0/306703604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636165cbdb0 con 0x56361653dde0 2024-09-19T09:28:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.184+0000 7f685cff9640 1 -- 172.21.15.16:0/306703604 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6870015080 con 0x56361653dde0 2024-09-19T09:28:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.185+0000 7f685cff9640 1 -- 172.21.15.16:0/306703604 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6870080070 con 0x56361653dde0 2024-09-19T09:28:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.185+0000 7f687d7d3740 10 client.25202 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.185+0000 7f687d7d3740 10 client.25202 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T09:28:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.185+0000 7f687d7d3740 20 client.25202 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.185+0000 7f687d7d3740 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x563616561040 0x5636165ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.185+0000 7f687d7d3740 4 client.25202 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:28:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.185+0000 7f687d7d3740 1 -- 172.21.15.16:0/306703604 --> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f686c023040 con 0x563616561040 2024-09-19T09:28:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.185+0000 7f6878995640 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x563616561040 0x5636165ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:13.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.186+0000 7f6878995640 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x563616561040 0x5636165ce510 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82011d91a6765174 server_cookie=39e49c8fb833b04f in_seq=0 out_seq=0 2024-09-19T09:28:13.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.186+0000 7f685cff9640 10 client.25202 ms_handle_connect on v2:172.21.15.97:6836/2018284957 2024-09-19T09:28:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.186+0000 7f685cff9640 1 -- 172.21.15.16:0/306703604 <== mds.2 v2:172.21.15.97:6836/2018284957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f686c023040 con 0x563616561040 2024-09-19T09:28:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.186+0000 7f685cff9640 10 client.25202 handle_command_reply: tid=0 2024-09-19T09:28:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.186+0000 7f687d7d3740 1 client.25202 shutdown 2024-09-19T09:28:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.187+0000 7f687d7d3740 1 -- 172.21.15.16:0/306703604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653dde0 msgr2=0x563616542640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.187+0000 7f687d7d3740 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653dde0 0x563616542640 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f687009b3e0 tx=0x7f687000df40 comp rx=0 tx=0).stop 2024-09-19T09:28:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.187+0000 7f687d7d3740 1 -- 172.21.15.16:0/306703604 shutdown_connections 2024-09-19T09:28:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.187+0000 7f687d7d3740 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x563616561040 0x5636165ce510 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.187+0000 7f687d7d3740 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563616548b80 0x5636165430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.187+0000 7f687d7d3740 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56361653f350 0x563616542b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.187+0000 7f687d7d3740 1 --2- 172.21.15.16:0/306703604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56361653dde0 0x563616542640 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.187+0000 7f687d7d3740 1 -- 172.21.15.16:0/306703604 >> 172.21.15.16:0/306703604 conn(0x5636164fe310 msgr2=0x563616535500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.187+0000 7f685cff9640 0 client.25202 ms_handle_reset on v2:172.21.15.97:6836/2018284957 2024-09-19T09:28:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.187+0000 7f687d7d3740 1 -- 172.21.15.16:0/306703604 shutdown_connections 2024-09-19T09:28:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.187+0000 7f687d7d3740 1 -- 172.21.15.16:0/306703604 wait complete. 2024-09-19T09:28:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.188+0000 7f687d7d3740 20 client.25202 trim_cache size 0 max 16384 2024-09-19T09:28:13.191 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:28:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.188+0000 7f687b3fa640 1 -- 172.21.15.16:0/1099333974 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f68600a34c0 msgr2=0x7f68600a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.188+0000 7f687b3fa640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f68600a34c0 0x7f68600a5980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f686c0092f0 tx=0x7f686c009280 comp rx=0 tx=0).stop 2024-09-19T09:28:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.188+0000 7f687b3fa640 1 -- 172.21.15.16:0/1099333974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6874100e20 msgr2=0x7f6874198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.189+0000 7f687b3fa640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6874100e20 0x7f6874198550 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f687000d950 tx=0x7f687000de10 comp rx=0 tx=0).stop 2024-09-19T09:28:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.189+0000 7f687b3fa640 1 -- 172.21.15.16:0/1099333974 shutdown_connections 2024-09-19T09:28:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.189+0000 7f687b3fa640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f68600a34c0 0x7f68600a5980 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.189+0000 7f687b3fa640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6874102570 0x7f6874198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.189+0000 7f687b3fa640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6874102ef0 0x7f687419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.189+0000 7f687b3fa640 1 --2- 172.21.15.16:0/1099333974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6874100e20 0x7f6874198550 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.189+0000 7f687b3fa640 1 -- 172.21.15.16:0/1099333974 >> 172.21.15.16:0/1099333974 conn(0x7f68740fc7d0 msgr2=0x7f687406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.190+0000 7f687b3fa640 1 -- 172.21.15.16:0/1099333974 shutdown_connections 2024-09-19T09:28:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:13.190+0000 7f687b3fa640 1 -- 172.21.15.16:0/1099333974 wait complete. 2024-09-19T09:28:13.317 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:13 smithi016 ceph-mon[30979]: pgmap v356: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T09:28:13.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:13 smithi142 ceph-mon[35032]: pgmap v356: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T09:28:13.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:13 smithi097 ceph-mon[34864]: pgmap v356: 97 pgs: 97 active+clean; 660 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T09:28:13.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:28:14.142 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.629+0000 7f28e1d7a640 1 -- 172.21.15.16:0/197527910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc100e20 msgr2=0x7f28dc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.629+0000 7f28e1d7a640 1 --2- 172.21.15.16:0/197527910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc100e20 0x7f28dc101200 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009900 tx=0x7f28cc02a480 comp rx=0 tx=0).stop 2024-09-19T09:28:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.629+0000 7f28e1d7a640 1 -- 172.21.15.16:0/197527910 shutdown_connections 2024-09-19T09:28:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.629+0000 7f28e1d7a640 1 --2- 172.21.15.16:0/197527910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28dc102570 0x7f28dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.629+0000 7f28e1d7a640 1 --2- 172.21.15.16:0/197527910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28dc102ef0 0x7f28dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.629+0000 7f28e1d7a640 1 --2- 172.21.15.16:0/197527910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc100e20 0x7f28dc101200 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.629+0000 7f28e1d7a640 1 -- 172.21.15.16:0/197527910 >> 172.21.15.16:0/197527910 conn(0x7f28dc0fc7d0 msgr2=0x7f28dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.630+0000 7f28e1d7a640 1 -- 172.21.15.16:0/197527910 shutdown_connections 2024-09-19T09:28:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.630+0000 7f28e1d7a640 1 -- 172.21.15.16:0/197527910 wait complete. 2024-09-19T09:28:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.631+0000 7f28e1d7a640 1 Processor -- start 2024-09-19T09:28:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.631+0000 7f28e1d7a640 1 -- start start 2024-09-19T09:28:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.631+0000 7f28e1d7a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28dc100e20 0x7f28dc197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.631+0000 7f28e1d7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc102570 0x7f28dc197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.631+0000 7f28e1d7a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28dc102ef0 0x7f28dc19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.631+0000 7f28e1d7a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc110f50 con 0x7f28dc102570 2024-09-19T09:28:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.631+0000 7f28e1d7a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc110dd0 con 0x7f28dc102ef0 2024-09-19T09:28:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.631+0000 7f28e1d7a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1110d0 con 0x7f28dc100e20 2024-09-19T09:28:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.631+0000 7f28daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc102570 0x7f28dc197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.631+0000 7f28daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc102570 0x7f28dc197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60156/0 (socket says 172.21.15.16:60156) 2024-09-19T09:28:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.631+0000 7f28daffd640 1 -- 172.21.15.16:0/1121004310 learned_addr learned my addr 172.21.15.16:0/1121004310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.632+0000 7f28dbfff640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28dc102ef0 0x7f28dc19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.632+0000 7f28db7fe640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28dc100e20 0x7f28dc197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.632+0000 7f28daffd640 1 -- 172.21.15.16:0/1121004310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28dc100e20 msgr2=0x7f28dc197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.632+0000 7f28daffd640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28dc100e20 0x7f28dc197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.632+0000 7f28daffd640 1 -- 172.21.15.16:0/1121004310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28dc102ef0 msgr2=0x7f28dc19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.632+0000 7f28daffd640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28dc102ef0 0x7f28dc19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.632+0000 7f28daffd640 1 -- 172.21.15.16:0/1121004310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc19c400 con 0x7f28dc102570 2024-09-19T09:28:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.632+0000 7f28dbfff640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28dc102ef0 0x7f28dc19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.632+0000 7f28daffd640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc102570 0x7f28dc197990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f28d000d950 tx=0x7f28d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.632+0000 7f28d8ff9640 1 -- 172.21.15.16:0/1121004310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0017070 con 0x7f28dc102570 2024-09-19T09:28:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.632+0000 7f28e1d7a640 1 -- 172.21.15.16:0/1121004310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28dc19c6f0 con 0x7f28dc102570 2024-09-19T09:28:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.632+0000 7f28d8ff9640 1 -- 172.21.15.16:0/1121004310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d0009c30 con 0x7f28dc102570 2024-09-19T09:28:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.633+0000 7f28e1d7a640 1 -- 172.21.15.16:0/1121004310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28dc069f70 con 0x7f28dc102570 2024-09-19T09:28:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.633+0000 7f28d8ff9640 1 -- 172.21.15.16:0/1121004310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d000f6b0 con 0x7f28dc102570 2024-09-19T09:28:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.635+0000 7f28d8ff9640 1 -- 172.21.15.16:0/1121004310 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f28d0010460 con 0x7f28dc102570 2024-09-19T09:28:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.636+0000 7f28d8ff9640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f28c40a32c0 0x7f28c40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.636+0000 7f28db7fe640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f28c40a32c0 0x7f28c40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.636+0000 7f28d8ff9640 1 -- 172.21.15.16:0/1121004310 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f28d00cf8e0 con 0x7f28dc102570 2024-09-19T09:28:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.637+0000 7f28db7fe640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f28c40a32c0 0x7f28c40a5780 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f28cc005ab0 tx=0x7f28cc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.638+0000 7f28e4153740 1 Processor -- start 2024-09-19T09:28:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.638+0000 7f28e4153740 1 -- start start 2024-09-19T09:28:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.638+0000 7f28e4153740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 0x56207f758090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.638+0000 7f28e4153740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 0x56207f758a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.638+0000 7f28e4153740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f763370 0x56207f765760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.638+0000 7f28e4153740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56207f6be2f0 con 0x56207f7585d0 2024-09-19T09:28:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.638+0000 7f28e4153740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56207f665cf0 con 0x56207f759b40 2024-09-19T09:28:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.638+0000 7f28e4153740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56207f6ba2b0 con 0x56207f763370 2024-09-19T09:28:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.638+0000 7f28daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 0x56207f758a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.638+0000 7f28daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 0x56207f758a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60160/0 (socket says 172.21.15.16:60160) 2024-09-19T09:28:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.638+0000 7f28db7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 0x56207f758090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.638+0000 7f28daffd640 1 -- 172.21.15.16:0/3310835561 learned_addr learned my addr 172.21.15.16:0/3310835561 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.638+0000 7f28dbfff640 1 --2- 172.21.15.16:0/3310835561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f763370 0x56207f765760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.639+0000 7f28daffd640 1 -- 172.21.15.16:0/3310835561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f763370 msgr2=0x56207f765760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.639+0000 7f28daffd640 1 --2- 172.21.15.16:0/3310835561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f763370 0x56207f765760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.639+0000 7f28daffd640 1 -- 172.21.15.16:0/3310835561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 msgr2=0x56207f758090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.639+0000 7f28daffd640 1 --2- 172.21.15.16:0/3310835561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 0x56207f758090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.639+0000 7f28daffd640 1 -- 172.21.15.16:0/3310835561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56207f64a3a0 con 0x56207f7585d0 2024-09-19T09:28:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.639+0000 7f28daffd640 1 --2- 172.21.15.16:0/3310835561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 0x56207f758a10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f28d000dfe0 tx=0x7f28d0009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8e4b158790824a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.639+0000 7f28c1ffb640 1 -- 172.21.15.16:0/3310835561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0020070 con 0x56207f7585d0 2024-09-19T09:28:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.639+0000 7f28c1ffb640 1 -- 172.21.15.16:0/3310835561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d00a4b70 con 0x56207f7585d0 2024-09-19T09:28:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.639+0000 7f28c1ffb640 1 -- 172.21.15.16:0/3310835561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d00109d0 con 0x56207f7585d0 2024-09-19T09:28:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.640+0000 7f28e4153740 1 -- 172.21.15.16:0/3310835561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 msgr2=0x56207f758a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.640+0000 7f28e4153740 1 --2- 172.21.15.16:0/3310835561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 0x56207f758a10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f28d000dfe0 tx=0x7f28d0009ea0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.640+0000 7f28e4153740 1 -- 172.21.15.16:0/3310835561 shutdown_connections 2024-09-19T09:28:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.640+0000 7f28e4153740 1 --2- 172.21.15.16:0/3310835561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f763370 0x56207f765760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.640+0000 7f28e4153740 1 --2- 172.21.15.16:0/3310835561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 0x56207f758a10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.640+0000 7f28e4153740 1 --2- 172.21.15.16:0/3310835561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 0x56207f758090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.640+0000 7f28e4153740 1 -- 172.21.15.16:0/3310835561 >> 172.21.15.16:0/3310835561 conn(0x56207f721310 msgr2=0x56207f754c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.640+0000 7f28e4153740 1 -- 172.21.15.16:0/3310835561 shutdown_connections 2024-09-19T09:28:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.640+0000 7f28e4153740 1 -- 172.21.15.16:0/3310835561 wait complete. 2024-09-19T09:28:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.640+0000 7f28e4153740 1 -- start start 2024-09-19T09:28:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.641+0000 7f28e4153740 1 Processor -- start 2024-09-19T09:28:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.641+0000 7f28e4153740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 0x56207f762d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.641+0000 7f28e4153740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 0x56207f760030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.641+0000 7f28e4153740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f763370 0x56207f760640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.641+0000 7f28e4153740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56207f765fb0 con 0x56207f7585d0 2024-09-19T09:28:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.641+0000 7f28e4153740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56207f679060 con 0x56207f759b40 2024-09-19T09:28:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.641+0000 7f28e4153740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56207f766130 con 0x56207f763370 2024-09-19T09:28:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.641+0000 7f28e4153740 10 client.0 _ll_register_callbacks cb 0x56207f3cc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.641+0000 7f28db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 0x56207f762d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.641+0000 7f28db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 0x56207f762d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60172/0 (socket says 172.21.15.16:60172) 2024-09-19T09:28:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.641+0000 7f28db7fe640 1 -- 172.21.15.16:0/1009529387 learned_addr learned my addr 172.21.15.16:0/1009529387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.641+0000 7f28daffd640 1 --2- 172.21.15.16:0/1009529387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 0x56207f760030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.642+0000 7f28db7fe640 1 -- 172.21.15.16:0/1009529387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f763370 msgr2=0x56207f760640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.642+0000 7f28db7fe640 1 --2- 172.21.15.16:0/1009529387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f763370 0x56207f760640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.642+0000 7f28db7fe640 1 -- 172.21.15.16:0/1009529387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 msgr2=0x56207f760030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.642+0000 7f28db7fe640 1 --2- 172.21.15.16:0/1009529387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 0x56207f760030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.642+0000 7f28db7fe640 1 -- 172.21.15.16:0/1009529387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56207f42f3c0 con 0x56207f7585d0 2024-09-19T09:28:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.642+0000 7f28db7fe640 1 --2- 172.21.15.16:0/1009529387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 0x56207f762d80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009900 tx=0x7f28cc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e3b03ccb034483 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.642+0000 7f28baffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.642+0000 7f28baffd640 1 -- 172.21.15.16:0/1009529387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc04d070 con 0x56207f7585d0 2024-09-19T09:28:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.642+0000 7f28baffd640 1 -- 172.21.15.16:0/1009529387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f28cc0048c0 con 0x56207f7585d0 2024-09-19T09:28:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.642+0000 7f28e4153740 1 -- 172.21.15.16:0/1009529387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d00a4b70 con 0x56207f7585d0 2024-09-19T09:28:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.642+0000 7f28baffd640 1 -- 172.21.15.16:0/1009529387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f28cc057050 con 0x56207f7585d0 2024-09-19T09:28:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.642+0000 7f28e4153740 10 client.15585 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.642+0000 7f28e4153740 1 -- 172.21.15.16:0/1009529387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56207f771ad0 con 0x56207f7585d0 2024-09-19T09:28:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.643+0000 7f28baffd640 1 -- 172.21.15.16:0/1009529387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d00a4b70 con 0x56207f7585d0 2024-09-19T09:28:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.643+0000 7f28baffd640 1 -- 172.21.15.16:0/1009529387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f28cc032550 con 0x56207f7585d0 2024-09-19T09:28:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.643+0000 7f28e4153740 10 client.15585 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.643+0000 7f28e4153740 10 client.15585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T09:28:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.643+0000 7f28e4153740 20 client.15585 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.643+0000 7f28e4153740 1 --2- 172.21.15.16:0/1009529387 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56207f772f90 0x56207f775430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.643+0000 7f28e4153740 4 client.15585 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.643+0000 7f28e4153740 1 -- 172.21.15.16:0/1009529387 --> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56207f6cb860 con 0x56207f772f90 2024-09-19T09:28:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.644+0000 7f28daffd640 1 --2- 172.21.15.16:0/1009529387 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56207f772f90 0x56207f775430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.644+0000 7f28baffd640 10 client.15585 ms_handle_connect on v2:172.21.15.142:6838/3822134597 2024-09-19T09:28:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.644+0000 7f28daffd640 1 --2- 172.21.15.16:0/1009529387 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56207f772f90 0x56207f775430 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15f51e78c7731227 server_cookie=327ced9485b1a9d in_seq=0 out_seq=0 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28baffd640 1 -- 172.21.15.16:0/1009529387 <== mds.1 v2:172.21.15.142:6838/3822134597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56207f6cb860 con 0x56207f772f90 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28baffd640 10 client.15585 handle_command_reply: tid=0 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28e4153740 1 client.15585 shutdown 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28e4153740 1 -- 172.21.15.16:0/1009529387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 msgr2=0x56207f762d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28e4153740 1 --2- 172.21.15.16:0/1009529387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 0x56207f762d80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009900 tx=0x7f28cc009260 comp rx=0 tx=0).stop 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28e4153740 1 -- 172.21.15.16:0/1009529387 shutdown_connections 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28e4153740 1 --2- 172.21.15.16:0/1009529387 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56207f772f90 0x56207f775430 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28baffd640 0 client.15585 ms_handle_reset on v2:172.21.15.142:6838/3822134597 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28e4153740 1 --2- 172.21.15.16:0/1009529387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f763370 0x56207f760640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28e4153740 1 --2- 172.21.15.16:0/1009529387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 0x56207f760030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28e4153740 1 --2- 172.21.15.16:0/1009529387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f7585d0 0x56207f762d80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28e4153740 1 -- 172.21.15.16:0/1009529387 >> 172.21.15.16:0/1009529387 conn(0x56207f721310 msgr2=0x56207f75a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28e4153740 1 -- 172.21.15.16:0/1009529387 shutdown_connections 2024-09-19T09:28:14.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.645+0000 7f28e4153740 1 -- 172.21.15.16:0/1009529387 wait complete. 2024-09-19T09:28:14.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.646+0000 7f28e4153740 20 client.15585 trim_cache size 0 max 16384 2024-09-19T09:28:14.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.660+0000 7f28e4153740 1 Processor -- start 2024-09-19T09:28:14.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.660+0000 7f28e4153740 1 -- start start 2024-09-19T09:28:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.660+0000 7f28e4153740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f759b40 0x56207f76f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.660+0000 7f28e4153740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f763370 0x56207f75e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.660+0000 7f28e4153740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f75b9d0 0x56207f75ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.660+0000 7f28e4153740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1af810 con 0x56207f75b9d0 2024-09-19T09:28:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.660+0000 7f28e4153740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1b1350 con 0x56207f763370 2024-09-19T09:28:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.660+0000 7f28e4153740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1afee0 con 0x56207f759b40 2024-09-19T09:28:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.660+0000 7f28dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f75b9d0 0x56207f75ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.660+0000 7f28dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f75b9d0 0x56207f75ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60182/0 (socket says 172.21.15.16:60182) 2024-09-19T09:28:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.660+0000 7f28dbfff640 1 -- 172.21.15.16:0/2616233696 learned_addr learned my addr 172.21.15.16:0/2616233696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.660+0000 7f28daffd640 1 --2- 172.21.15.16:0/2616233696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f763370 0x56207f75e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.660+0000 7f28db7fe640 1 --2- 172.21.15.16:0/2616233696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f759b40 0x56207f76f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.661+0000 7f28daffd640 1 -- 172.21.15.16:0/2616233696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f759b40 msgr2=0x56207f76f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.661+0000 7f28daffd640 1 --2- 172.21.15.16:0/2616233696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f759b40 0x56207f76f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.661+0000 7f28daffd640 1 -- 172.21.15.16:0/2616233696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f75b9d0 msgr2=0x56207f75ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.661+0000 7f28daffd640 1 --2- 172.21.15.16:0/2616233696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f75b9d0 0x56207f75ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.661+0000 7f28daffd640 1 -- 172.21.15.16:0/2616233696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56207f7583f0 con 0x56207f763370 2024-09-19T09:28:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.661+0000 7f28dbfff640 1 --2- 172.21.15.16:0/2616233696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f75b9d0 0x56207f75ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.661+0000 7f28daffd640 1 --2- 172.21.15.16:0/2616233696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f763370 0x56207f75e900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f28d000dfe0 tx=0x7f28d009c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=291b119abbd5d39f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.661+0000 7f28c17fa640 1 -- 172.21.15.16:0/2616233696 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0020040 con 0x56207f763370 2024-09-19T09:28:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.661+0000 7f28c17fa640 1 -- 172.21.15.16:0/2616233696 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d009a200 con 0x56207f763370 2024-09-19T09:28:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.661+0000 7f28c17fa640 1 -- 172.21.15.16:0/2616233696 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d00a46e0 con 0x56207f763370 2024-09-19T09:28:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.662+0000 7f28e4153740 1 -- 172.21.15.16:0/2616233696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f763370 msgr2=0x56207f75e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.662+0000 7f28e4153740 1 --2- 172.21.15.16:0/2616233696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f763370 0x56207f75e900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f28d000dfe0 tx=0x7f28d009c040 comp rx=0 tx=0).stop 2024-09-19T09:28:14.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.662+0000 7f28e4153740 1 -- 172.21.15.16:0/2616233696 shutdown_connections 2024-09-19T09:28:14.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.662+0000 7f28e4153740 1 --2- 172.21.15.16:0/2616233696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f75b9d0 0x56207f75ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.662+0000 7f28e4153740 1 --2- 172.21.15.16:0/2616233696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f763370 0x56207f75e900 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.662+0000 7f28e4153740 1 --2- 172.21.15.16:0/2616233696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f759b40 0x56207f76f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.662+0000 7f28e4153740 1 -- 172.21.15.16:0/2616233696 >> 172.21.15.16:0/2616233696 conn(0x56207f721310 msgr2=0x56207f75b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:14.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.662+0000 7f28e4153740 1 -- 172.21.15.16:0/2616233696 shutdown_connections 2024-09-19T09:28:14.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.663+0000 7f28e4153740 1 -- 172.21.15.16:0/2616233696 wait complete. 2024-09-19T09:28:14.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.663+0000 7f28e4153740 1 -- start start 2024-09-19T09:28:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.663+0000 7f28e4153740 1 Processor -- start 2024-09-19T09:28:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28e4153740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 0x56207f76b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28e4153740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f75b9d0 0x56207f766d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28e4153740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f763370 0x56207f7672b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28e4153740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1b0c80 con 0x56207f763370 2024-09-19T09:28:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28e4153740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1b05b0 con 0x56207f759b40 2024-09-19T09:28:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28e4153740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1ad7d0 con 0x56207f75b9d0 2024-09-19T09:28:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28e4153740 10 client.0 _ll_register_callbacks cb 0x7f28cc03f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f763370 0x56207f7672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f763370 0x56207f7672b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60190/0 (socket says 172.21.15.16:60190) 2024-09-19T09:28:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28dbfff640 1 -- 172.21.15.16:0/253989676 learned_addr learned my addr 172.21.15.16:0/253989676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28daffd640 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f75b9d0 0x56207f766d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28db7fe640 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 0x56207f76b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28dbfff640 1 -- 172.21.15.16:0/253989676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f75b9d0 msgr2=0x56207f766d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28dbfff640 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f75b9d0 0x56207f766d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28dbfff640 1 -- 172.21.15.16:0/253989676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 msgr2=0x56207f76b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28dbfff640 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 0x56207f76b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28dbfff640 1 -- 172.21.15.16:0/253989676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56207f7679a0 con 0x56207f763370 2024-09-19T09:28:14.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.664+0000 7f28daffd640 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f75b9d0 0x56207f766d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:14.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.665+0000 7f28dbfff640 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f763370 0x56207f7672b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f28d40050c0 tx=0x7f28d4005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf90828f5297fdab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:14.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.665+0000 7f28baffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:14.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.665+0000 7f28baffd640 1 -- 172.21.15.16:0/253989676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d40127e0 con 0x56207f763370 2024-09-19T09:28:14.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.665+0000 7f28baffd640 1 -- 172.21.15.16:0/253989676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f28d4012980 con 0x56207f763370 2024-09-19T09:28:14.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.665+0000 7f28e4153740 1 -- 172.21.15.16:0/253989676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d009a200 con 0x56207f763370 2024-09-19T09:28:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.665+0000 7f28baffd640 1 -- 172.21.15.16:0/253989676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f28d4012b80 con 0x56207f763370 2024-09-19T09:28:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.665+0000 7f28e4153740 10 client.15591 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.665+0000 7f28e4153740 1 -- 172.21.15.16:0/253989676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56207f7ddc00 con 0x56207f763370 2024-09-19T09:28:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.665+0000 7f28baffd640 1 -- 172.21.15.16:0/253989676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d4012d20 con 0x56207f763370 2024-09-19T09:28:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.666+0000 7f28baffd640 1 -- 172.21.15.16:0/253989676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f28d401b6b0 con 0x56207f763370 2024-09-19T09:28:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.666+0000 7f28e4153740 10 client.15591 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.666+0000 7f28e4153740 10 client.15591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T09:28:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.666+0000 7f28e4153740 20 client.15591 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.666+0000 7f28e4153740 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56207f7de3d0 0x56207f7e07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.666+0000 7f28e4153740 4 client.15591 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.666+0000 7f28e4153740 1 -- 172.21.15.16:0/253989676 --> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f28d009a200 con 0x56207f7de3d0 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.666+0000 7f28db7fe640 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56207f7de3d0 0x56207f7e07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.666+0000 7f28baffd640 10 client.15591 ms_handle_connect on v2:172.21.15.142:6838/3822134597 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.666+0000 7f28db7fe640 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56207f7de3d0 0x56207f7e07f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34e8661eb7701b88 server_cookie=ee646492487fd084 in_seq=0 out_seq=0 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28baffd640 1 -- 172.21.15.16:0/253989676 <== mds.1 v2:172.21.15.142:6838/3822134597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f28d009a200 con 0x56207f7de3d0 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28baffd640 10 client.15591 handle_command_reply: tid=0 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28e4153740 1 client.15591 shutdown 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28e4153740 1 -- 172.21.15.16:0/253989676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f763370 msgr2=0x56207f7672b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28e4153740 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f763370 0x56207f7672b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f28d40050c0 tx=0x7f28d4005580 comp rx=0 tx=0).stop 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28e4153740 1 -- 172.21.15.16:0/253989676 shutdown_connections 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28e4153740 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56207f7de3d0 0x56207f7e07f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28e4153740 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56207f75b9d0 0x56207f766d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28e4153740 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56207f763370 0x56207f7672b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28e4153740 1 --2- 172.21.15.16:0/253989676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56207f759b40 0x56207f76b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28e4153740 1 -- 172.21.15.16:0/253989676 >> 172.21.15.16:0/253989676 conn(0x56207f721310 msgr2=0x56207f75e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28baffd640 0 client.15591 ms_handle_reset on v2:172.21.15.142:6838/3822134597 2024-09-19T09:28:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28e4153740 1 -- 172.21.15.16:0/253989676 shutdown_connections 2024-09-19T09:28:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28e4153740 1 -- 172.21.15.16:0/253989676 wait complete. 2024-09-19T09:28:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.667+0000 7f28e4153740 20 client.15591 trim_cache size 0 max 16384 2024-09-19T09:28:14.672 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:28:14.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.668+0000 7f28e1d7a640 1 -- 172.21.15.16:0/1121004310 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f28c40a32c0 msgr2=0x7f28c40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.668+0000 7f28e1d7a640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f28c40a32c0 0x7f28c40a5780 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f28cc005ab0 tx=0x7f28cc005a20 comp rx=0 tx=0).stop 2024-09-19T09:28:14.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.668+0000 7f28e1d7a640 1 -- 172.21.15.16:0/1121004310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc102570 msgr2=0x7f28dc197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:14.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.668+0000 7f28e1d7a640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc102570 0x7f28dc197990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f28d000d950 tx=0x7f28d000de10 comp rx=0 tx=0).stop 2024-09-19T09:28:14.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.668+0000 7f28e1d7a640 1 -- 172.21.15.16:0/1121004310 shutdown_connections 2024-09-19T09:28:14.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.668+0000 7f28e1d7a640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28dc102ef0 0x7f28dc19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.668+0000 7f28e1d7a640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc102570 0x7f28dc197990 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.668+0000 7f28e1d7a640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f28c40a32c0 0x7f28c40a5780 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.669+0000 7f28e1d7a640 1 --2- 172.21.15.16:0/1121004310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28dc100e20 0x7f28dc197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:14.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.669+0000 7f28e1d7a640 1 -- 172.21.15.16:0/1121004310 >> 172.21.15.16:0/1121004310 conn(0x7f28dc0fc7d0 msgr2=0x7f28dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:14.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.669+0000 7f28e1d7a640 1 -- 172.21.15.16:0/1121004310 shutdown_connections 2024-09-19T09:28:14.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:14.669+0000 7f28e1d7a640 1 -- 172.21.15.16:0/1121004310 wait complete. 2024-09-19T09:28:15.301 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:15 smithi016 ceph-mon[30979]: pgmap v357: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T09:28:15.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:15 smithi142 ceph-mon[35032]: pgmap v357: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T09:28:15.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:15 smithi097 ceph-mon[34864]: pgmap v357: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-19T09:28:15.388 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:28:15.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:28:15.702 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:16.241 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:16.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.258+0000 7f443d022640 1 -- 172.21.15.16:0/3067706708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4438106f00 msgr2=0x7f443810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.258+0000 7f443d022640 1 --2- 172.21.15.16:0/3067706708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4438106f00 0x7f443810ea80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4430009ca0 tx=0x7f443002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.258+0000 7f443d022640 1 -- 172.21.15.16:0/3067706708 shutdown_connections 2024-09-19T09:28:16.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.258+0000 7f443d022640 1 --2- 172.21.15.16:0/3067706708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4438106f00 0x7f443810ea80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.258+0000 7f443d022640 1 --2- 172.21.15.16:0/3067706708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4438102780 0x7f44381067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.258+0000 7f443d022640 1 --2- 172.21.15.16:0/3067706708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4438101dd0 0x7f44381021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.258+0000 7f443d022640 1 -- 172.21.15.16:0/3067706708 >> 172.21.15.16:0/3067706708 conn(0x7f44380fd870 msgr2=0x7f44380ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:16.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.258+0000 7f443d022640 1 -- 172.21.15.16:0/3067706708 shutdown_connections 2024-09-19T09:28:16.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.258+0000 7f443d022640 1 -- 172.21.15.16:0/3067706708 wait complete. 2024-09-19T09:28:16.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.259+0000 7f443d022640 1 Processor -- start 2024-09-19T09:28:16.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.260+0000 7f443d022640 1 -- start start 2024-09-19T09:28:16.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.260+0000 7f443d022640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4438101dd0 0x7f443819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.260+0000 7f443d022640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4438102780 0x7f443819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.260+0000 7f443d022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4438106f00 0x7f443819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.260+0000 7f443d022640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4438111f80 con 0x7f4438106f00 2024-09-19T09:28:16.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.261+0000 7f443659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4438102780 0x7f443819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.261+0000 7f443659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4438102780 0x7f443819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:33100/0 (socket says 172.21.15.16:33100) 2024-09-19T09:28:16.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.261+0000 7f443659c640 1 -- 172.21.15.16:0/1187985293 learned_addr learned my addr 172.21.15.16:0/1187985293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:16.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.261+0000 7f443759e640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4438106f00 0x7f443819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.261+0000 7f4436d9d640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4438101dd0 0x7f443819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.261+0000 7f443d022640 1 -- 172.21.15.16:0/1187985293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4438111e00 con 0x7f4438102780 2024-09-19T09:28:16.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.261+0000 7f443d022640 1 -- 172.21.15.16:0/1187985293 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4438112100 con 0x7f4438101dd0 2024-09-19T09:28:16.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.262+0000 7f443759e640 1 -- 172.21.15.16:0/1187985293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4438101dd0 msgr2=0x7f443819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.262+0000 7f443759e640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4438101dd0 0x7f443819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.262+0000 7f443759e640 1 -- 172.21.15.16:0/1187985293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4438102780 msgr2=0x7f443819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.262+0000 7f443759e640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4438102780 0x7f443819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.262+0000 7f443759e640 1 -- 172.21.15.16:0/1187985293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443819f410 con 0x7f4438106f00 2024-09-19T09:28:16.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.262+0000 7f4436d9d640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4438101dd0 0x7f443819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:16.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.262+0000 7f443759e640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4438106f00 0x7f443819ed30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4430009bf0 tx=0x7f44300095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:16.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.262+0000 7f441ffff640 1 -- 172.21.15.16:0/1187985293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443002fba0 con 0x7f4438106f00 2024-09-19T09:28:16.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.262+0000 7f441ffff640 1 -- 172.21.15.16:0/1187985293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f443002fd40 con 0x7f4438106f00 2024-09-19T09:28:16.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.262+0000 7f441ffff640 1 -- 172.21.15.16:0/1187985293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44300468f0 con 0x7f4438106f00 2024-09-19T09:28:16.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.263+0000 7f443d022640 1 -- 172.21.15.16:0/1187985293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443819f6a0 con 0x7f4438106f00 2024-09-19T09:28:16.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.263+0000 7f443d022640 1 -- 172.21.15.16:0/1187985293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443819fad0 con 0x7f4438106f00 2024-09-19T09:28:16.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.265+0000 7f443d022640 1 -- 172.21.15.16:0/1187985293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4424005290 con 0x7f4438106f00 2024-09-19T09:28:16.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.266+0000 7f441ffff640 1 -- 172.21.15.16:0/1187985293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4430046a90 con 0x7f4438106f00 2024-09-19T09:28:16.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.267+0000 7f441ffff640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f44200a3390 0x7f44200a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.267+0000 7f441ffff640 1 -- 172.21.15.16:0/1187985293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f44300f0b50 con 0x7f4438106f00 2024-09-19T09:28:16.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.268+0000 7f4436d9d640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f44200a3390 0x7f44200a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.269+0000 7f4436d9d640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f44200a3390 0x7f44200a5850 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4428009930 tx=0x7f4428008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:16.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.272+0000 7f441ffff640 1 -- 172.21.15.16:0/1187985293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f44300bd830 con 0x7f4438106f00 2024-09-19T09:28:16.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.492+0000 7f443d022640 1 -- 172.21.15.16:0/1187985293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4424005580 con 0x7f4438106f00 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.493+0000 7f441ffff640 1 -- 172.21.15.16:0/1187985293 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f44300bd130 con 0x7f4438106f00 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:28:16.494 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.495+0000 7f443d022640 1 -- 172.21.15.16:0/1187985293 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f44200a3390 msgr2=0x7f44200a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.495+0000 7f443d022640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f44200a3390 0x7f44200a5850 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4428009930 tx=0x7f4428008040 comp rx=0 tx=0).stop 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.495+0000 7f443d022640 1 -- 172.21.15.16:0/1187985293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4438106f00 msgr2=0x7f443819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.495+0000 7f443d022640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4438106f00 0x7f443819ed30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4430009bf0 tx=0x7f44300095b0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.495+0000 7f443d022640 1 -- 172.21.15.16:0/1187985293 shutdown_connections 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.495+0000 7f443d022640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4438106f00 0x7f443819ed30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.495+0000 7f443d022640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4438102780 0x7f443819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.495+0000 7f443d022640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f44200a3390 0x7f44200a5850 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.495+0000 7f443d022640 1 --2- 172.21.15.16:0/1187985293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4438101dd0 0x7f443819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.496+0000 7f443d022640 1 -- 172.21.15.16:0/1187985293 >> 172.21.15.16:0/1187985293 conn(0x7f44380fd870 msgr2=0x7f443810cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.496+0000 7f443d022640 1 -- 172.21.15.16:0/1187985293 shutdown_connections 2024-09-19T09:28:16.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.496+0000 7f443d022640 1 -- 172.21.15.16:0/1187985293 wait complete. 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.942+0000 7fb13dd9e640 1 -- 172.21.15.16:0/632249753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1381068d0 msgr2=0x7fb138106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.942+0000 7fb13dd9e640 1 --2- 172.21.15.16:0/632249753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1381068d0 0x7fb138106cb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009930 tx=0x7fb12c02a420 comp rx=0 tx=0).stop 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.942+0000 7fb13dd9e640 1 -- 172.21.15.16:0/632249753 shutdown_connections 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.942+0000 7fb13dd9e640 1 --2- 172.21.15.16:0/632249753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1381008d0 0x7fb138100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.942+0000 7fb13dd9e640 1 --2- 172.21.15.16:0/632249753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb138101270 0x7fb13810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.942+0000 7fb13dd9e640 1 --2- 172.21.15.16:0/632249753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1381068d0 0x7fb138106cb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.942+0000 7fb13dd9e640 1 -- 172.21.15.16:0/632249753 >> 172.21.15.16:0/632249753 conn(0x7fb1380fc610 msgr2=0x7fb1380fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.942+0000 7fb13dd9e640 1 -- 172.21.15.16:0/632249753 shutdown_connections 2024-09-19T09:28:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.942+0000 7fb13dd9e640 1 -- 172.21.15.16:0/632249753 wait complete. 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.943+0000 7fb13dd9e640 1 Processor -- start 2024-09-19T09:28:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.943+0000 7fb13dd9e640 1 -- start start 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.944+0000 7fb13dd9e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1381008d0 0x7fb138196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.944+0000 7fb13dd9e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb138101270 0x7fb138197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.944+0000 7fb13dd9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1381068d0 0x7fb1381979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.944+0000 7fb13dd9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb138110c90 con 0x7fb1381068d0 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.944+0000 7fb13dd9e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb138110b10 con 0x7fb1381008d0 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.944+0000 7fb13dd9e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb138110e10 con 0x7fb138101270 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.944+0000 7fb137fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb138101270 0x7fb138197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.944+0000 7fb137fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb138101270 0x7fb138197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:43944/0 (socket says 172.21.15.16:43944) 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.944+0000 7fb137fff640 1 -- 172.21.15.16:0/1219104683 learned_addr learned my addr 172.21.15.16:0/1219104683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.944+0000 7fb13d59d640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1381068d0 0x7fb1381979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.945+0000 7fb13cd9c640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1381008d0 0x7fb138196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.945+0000 7fb13d59d640 1 -- 172.21.15.16:0/1219104683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb138101270 msgr2=0x7fb138197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.945+0000 7fb13d59d640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb138101270 0x7fb138197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.945+0000 7fb13d59d640 1 -- 172.21.15.16:0/1219104683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1381008d0 msgr2=0x7fb138196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.945+0000 7fb13d59d640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1381008d0 0x7fb138196f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.945+0000 7fb13d59d640 1 -- 172.21.15.16:0/1219104683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1381a67a0 con 0x7fb1381068d0 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.945+0000 7fb13d59d640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1381068d0 0x7fb1381979b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb13000ecb0 tx=0x7fb13000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.945+0000 7fb135ffb640 1 -- 172.21.15.16:0/1219104683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130004b20 con 0x7fb1381068d0 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.945+0000 7fb13dd9e640 1 -- 172.21.15.16:0/1219104683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1381a6a90 con 0x7fb1381068d0 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.945+0000 7fb135ffb640 1 -- 172.21.15.16:0/1219104683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb130004cc0 con 0x7fb1381068d0 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.945+0000 7fb135ffb640 1 -- 172.21.15.16:0/1219104683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300187a0 con 0x7fb1381068d0 2024-09-19T09:28:16.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.946+0000 7fb13dd9e640 1 -- 172.21.15.16:0/1219104683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1381a6fd0 con 0x7fb1381068d0 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.948+0000 7fb1413d9740 1 Processor -- start 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.949+0000 7fb135ffb640 1 -- 172.21.15.16:0/1219104683 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb130018960 con 0x7fb1381068d0 2024-09-19T09:28:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.949+0000 7fb1413d9740 1 -- start start 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.950+0000 7fb1413d9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a12530 0x560fc9a10a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.950+0000 7fb1413d9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a10fc0 0x560fc9a11400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.950+0000 7fb1413d9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a1bd60 0x560fc9a1e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.950+0000 7fb1413d9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560fc996e2f0 con 0x560fc9a12530 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.950+0000 7fb1413d9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560fc9915cf0 con 0x560fc9a1bd60 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.950+0000 7fb1413d9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560fc996a2b0 con 0x560fc9a10fc0 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.950+0000 7fb137fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a12530 0x560fc9a10a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.950+0000 7fb137fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a12530 0x560fc9a10a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55222/0 (socket says 172.21.15.16:55222) 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.950+0000 7fb137fff640 1 -- 172.21.15.16:0/1982012667 learned_addr learned my addr 172.21.15.16:0/1982012667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.950+0000 7fb135ffb640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb1280a3240 0x7fb1280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.950+0000 7fb137fff640 1 -- 172.21.15.16:0/1982012667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a10fc0 msgr2=0x560fc9a11400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.950+0000 7fb137fff640 1 --2- 172.21.15.16:0/1982012667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a10fc0 0x560fc9a11400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.951+0000 7fb137fff640 1 -- 172.21.15.16:0/1982012667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a1bd60 msgr2=0x560fc9a1e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.951+0000 7fb137fff640 1 --2- 172.21.15.16:0/1982012667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a1bd60 0x560fc9a1e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.951+0000 7fb137fff640 1 -- 172.21.15.16:0/1982012667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fc98fa3a0 con 0x560fc9a12530 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.951+0000 7fb135ffb640 1 -- 172.21.15.16:0/1219104683 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb1300d0030 con 0x7fb1381068d0 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.951+0000 7fb137fff640 1 --2- 172.21.15.16:0/1982012667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a12530 0x560fc9a10a80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb124005e00 tx=0x7fb12400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec1390a440d4aec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.951+0000 7fb13d59d640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb1280a3240 0x7fb1280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.951+0000 7fb122ffd640 1 -- 172.21.15.16:0/1982012667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124003dd0 con 0x560fc9a12530 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.951+0000 7fb122ffd640 1 -- 172.21.15.16:0/1982012667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb124016030 con 0x560fc9a12530 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.951+0000 7fb122ffd640 1 -- 172.21.15.16:0/1982012667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12401a750 con 0x560fc9a12530 2024-09-19T09:28:16.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.952+0000 7fb1413d9740 1 -- 172.21.15.16:0/1982012667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a12530 msgr2=0x560fc9a10a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.952+0000 7fb13d59d640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb1280a3240 0x7fb1280a5700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb1300cdaf0 tx=0x7fb130005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.952+0000 7fb1413d9740 1 --2- 172.21.15.16:0/1982012667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a12530 0x560fc9a10a80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb124005e00 tx=0x7fb12400bd70 comp rx=0 tx=0).stop 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.953+0000 7fb1413d9740 1 -- 172.21.15.16:0/1982012667 shutdown_connections 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.953+0000 7fb1413d9740 1 --2- 172.21.15.16:0/1982012667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a10fc0 0x560fc9a11400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.953+0000 7fb1413d9740 1 --2- 172.21.15.16:0/1982012667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a1bd60 0x560fc9a1e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.953+0000 7fb1413d9740 1 --2- 172.21.15.16:0/1982012667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a12530 0x560fc9a10a80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.953+0000 7fb1413d9740 1 -- 172.21.15.16:0/1982012667 >> 172.21.15.16:0/1982012667 conn(0x560fc99d1310 msgr2=0x560fc9a0d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.953+0000 7fb1413d9740 1 -- 172.21.15.16:0/1982012667 shutdown_connections 2024-09-19T09:28:16.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.953+0000 7fb1413d9740 1 -- 172.21.15.16:0/1982012667 wait complete. 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.954+0000 7fb1413d9740 1 -- start start 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.954+0000 7fb1413d9740 1 Processor -- start 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.954+0000 7fb1413d9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a10fc0 0x560fc9a1b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.954+0000 7fb1413d9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a12530 0x560fc9a18aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.954+0000 7fb1413d9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a1bd60 0x560fc9a19230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.954+0000 7fb1413d9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560fc9a1ea00 con 0x560fc9a1bd60 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.954+0000 7fb1413d9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560fc9929060 con 0x560fc9a10fc0 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.954+0000 7fb1413d9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560fc9a1eb80 con 0x560fc9a12530 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.954+0000 7fb1413d9740 10 client.0 _ll_register_callbacks cb 0x560fc967c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.955+0000 7fb13cd9c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a12530 0x560fc9a18aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.955+0000 7fb137fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a10fc0 0x560fc9a1b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.955+0000 7fb13cd9c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a12530 0x560fc9a18aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:43954/0 (socket says 172.21.15.16:43954) 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.955+0000 7fb13cd9c640 1 -- 172.21.15.16:0/1237715382 learned_addr learned my addr 172.21.15.16:0/1237715382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.955+0000 7fb137fff640 1 --2- 172.21.15.16:0/1237715382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a1bd60 0x560fc9a19230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.955+0000 7fb13cd9c640 1 -- 172.21.15.16:0/1237715382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a10fc0 msgr2=0x560fc9a1b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.955+0000 7fb13cd9c640 1 --2- 172.21.15.16:0/1237715382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a10fc0 0x560fc9a1b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.955+0000 7fb13cd9c640 1 -- 172.21.15.16:0/1237715382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a1bd60 msgr2=0x560fc9a19230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.956+0000 7fb13cd9c640 1 --2- 172.21.15.16:0/1237715382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a1bd60 0x560fc9a19230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.956+0000 7fb13cd9c640 1 -- 172.21.15.16:0/1237715382 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fc96df3c0 con 0x560fc9a12530 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.956+0000 7fb137fff640 1 --2- 172.21.15.16:0/1237715382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a1bd60 0x560fc9a19230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.956+0000 7fb13cd9c640 1 --2- 172.21.15.16:0/1237715382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a12530 0x560fc9a18aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb12c032e40 tx=0x7fb12c032e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a4299c98bf4ea1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.956+0000 7fb117fff640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.956+0000 7fb117fff640 1 -- 172.21.15.16:0/1237715382 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c03f030 con 0x560fc9a12530 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.956+0000 7fb117fff640 1 -- 172.21.15.16:0/1237715382 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb12c0317c0 con 0x560fc9a12530 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.956+0000 7fb1413d9740 1 -- 172.21.15.16:0/1237715382 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb124016030 con 0x560fc9a12530 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.957+0000 7fb117fff640 1 -- 172.21.15.16:0/1237715382 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb12c049050 con 0x560fc9a12530 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.957+0000 7fb1413d9740 10 client.25174 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.957+0000 7fb1413d9740 1 -- 172.21.15.16:0/1237715382 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560fc9a2a580 con 0x560fc9a12530 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.957+0000 7fb117fff640 1 -- 172.21.15.16:0/1237715382 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb124016030 con 0x560fc9a12530 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.957+0000 7fb117fff640 1 -- 172.21.15.16:0/1237715382 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb12c005480 con 0x560fc9a12530 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.958+0000 7fb1413d9740 10 client.25174 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.958+0000 7fb1413d9740 10 client.25174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.958+0000 7fb1413d9740 20 client.25174 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.958+0000 7fb1413d9740 1 --2- 172.21.15.16:0/1237715382 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fc9a2ba90 0x560fc9a2ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.958+0000 7fb1413d9740 4 client.25174 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.958+0000 7fb1413d9740 1 -- 172.21.15.16:0/1237715382 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fc997b860 con 0x560fc9a2ba90 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.958+0000 7fb137fff640 1 --2- 172.21.15.16:0/1237715382 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fc9a2ba90 0x560fc9a2ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.959+0000 7fb117fff640 10 client.25174 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.959+0000 7fb137fff640 1 --2- 172.21.15.16:0/1237715382 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fc9a2ba90 0x560fc9a2ded0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5157e44b7d643d3 server_cookie=857617ec8263444e in_seq=0 out_seq=0 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.959+0000 7fb117fff640 1 -- 172.21.15.16:0/1237715382 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fc997b860 con 0x560fc9a2ba90 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.959+0000 7fb117fff640 10 client.25174 handle_command_reply: tid=0 2024-09-19T09:28:16.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.959+0000 7fb1413d9740 1 client.25174 shutdown 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.960+0000 7fb1413d9740 1 -- 172.21.15.16:0/1237715382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a12530 msgr2=0x560fc9a18aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.960+0000 7fb1413d9740 1 --2- 172.21.15.16:0/1237715382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a12530 0x560fc9a18aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb12c032e40 tx=0x7fb12c032e70 comp rx=0 tx=0).stop 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.960+0000 7fb1413d9740 1 -- 172.21.15.16:0/1237715382 shutdown_connections 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.960+0000 7fb1413d9740 1 --2- 172.21.15.16:0/1237715382 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fc9a2ba90 0x560fc9a2ded0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.960+0000 7fb117fff640 0 client.25174 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.960+0000 7fb1413d9740 1 --2- 172.21.15.16:0/1237715382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a12530 0x560fc9a18aa0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb12c032e40 tx=0x7fb12c032e70 comp rx=0 tx=0).stop 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.960+0000 7fb1413d9740 1 --2- 172.21.15.16:0/1237715382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a1bd60 0x560fc9a19230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.960+0000 7fb1413d9740 1 --2- 172.21.15.16:0/1237715382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a10fc0 0x560fc9a1b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.960+0000 7fb1413d9740 1 -- 172.21.15.16:0/1237715382 >> 172.21.15.16:0/1237715382 conn(0x560fc99d1310 msgr2=0x560fc9a03fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.960+0000 7fb1413d9740 1 -- 172.21.15.16:0/1237715382 shutdown_connections 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.960+0000 7fb1413d9740 1 -- 172.21.15.16:0/1237715382 wait complete. 2024-09-19T09:28:16.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.961+0000 7fb1413d9740 20 client.25174 trim_cache size 0 max 16384 2024-09-19T09:28:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.975+0000 7fb1413d9740 1 Processor -- start 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.975+0000 7fb1413d9740 1 -- start start 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.975+0000 7fb1413d9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a10fc0 0x560fc9a0d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.975+0000 7fb1413d9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a1bd60 0x560fc9a1aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.975+0000 7fb1413d9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a18b80 0x560fc9a337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.975+0000 7fb1413d9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1381ab1e0 con 0x560fc9a18b80 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.975+0000 7fb1413d9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb1381acd20 con 0x560fc9a1bd60 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.975+0000 7fb1413d9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb1381ab8b0 con 0x560fc9a10fc0 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.976+0000 7fb137fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a18b80 0x560fc9a337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.976+0000 7fb13cd9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a1bd60 0x560fc9a1aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.976+0000 7fb137fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a18b80 0x560fc9a337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55244/0 (socket says 172.21.15.16:55244) 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.976+0000 7fb137fff640 1 -- 172.21.15.16:0/2580009898 learned_addr learned my addr 172.21.15.16:0/2580009898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.976+0000 7fb13cd9c640 1 -- 172.21.15.16:0/2580009898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a10fc0 msgr2=0x560fc9a0d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.976+0000 7fb13cd9c640 1 --2- 172.21.15.16:0/2580009898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a10fc0 0x560fc9a0d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.976+0000 7fb13cd9c640 1 -- 172.21.15.16:0/2580009898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a18b80 msgr2=0x560fc9a337a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.976+0000 7fb13cd9c640 1 --2- 172.21.15.16:0/2580009898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a18b80 0x560fc9a337a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.976+0000 7fb13cd9c640 1 -- 172.21.15.16:0/2580009898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fc9a195a0 con 0x560fc9a1bd60 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.976+0000 7fb137fff640 1 --2- 172.21.15.16:0/2580009898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a18b80 0x560fc9a337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.976+0000 7fb13cd9c640 1 --2- 172.21.15.16:0/2580009898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a1bd60 0x560fc9a1aa10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x560fc9a19b90 tx=0x7fb12c009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b9f3a7ef72d248 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.977+0000 7fb1227fc640 1 -- 172.21.15.16:0/2580009898 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c03f040 con 0x560fc9a1bd60 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.977+0000 7fb1227fc640 1 -- 172.21.15.16:0/2580009898 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb12c004bb0 con 0x560fc9a1bd60 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.977+0000 7fb1227fc640 1 -- 172.21.15.16:0/2580009898 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c035070 con 0x560fc9a1bd60 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.977+0000 7fb1413d9740 1 -- 172.21.15.16:0/2580009898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a1bd60 msgr2=0x560fc9a1aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.977+0000 7fb1413d9740 1 --2- 172.21.15.16:0/2580009898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a1bd60 0x560fc9a1aa10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x560fc9a19b90 tx=0x7fb12c009300 comp rx=0 tx=0).stop 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.978+0000 7fb1413d9740 1 -- 172.21.15.16:0/2580009898 shutdown_connections 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.978+0000 7fb1413d9740 1 --2- 172.21.15.16:0/2580009898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a18b80 0x560fc9a337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.978+0000 7fb1413d9740 1 --2- 172.21.15.16:0/2580009898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a1bd60 0x560fc9a1aa10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.978+0000 7fb1413d9740 1 --2- 172.21.15.16:0/2580009898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a10fc0 0x560fc9a0d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.978+0000 7fb1413d9740 1 -- 172.21.15.16:0/2580009898 >> 172.21.15.16:0/2580009898 conn(0x560fc99d1310 msgr2=0x560fc9a1a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.978+0000 7fb1413d9740 1 -- 172.21.15.16:0/2580009898 shutdown_connections 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.978+0000 7fb1413d9740 1 -- 172.21.15.16:0/2580009898 wait complete. 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.978+0000 7fb1413d9740 1 -- start start 2024-09-19T09:28:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.978+0000 7fb1413d9740 1 Processor -- start 2024-09-19T09:28:16.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.979+0000 7fb1413d9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a10fc0 0x560fc9a24600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.979+0000 7fb1413d9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a18b80 0x560fc9a15860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.979+0000 7fb1413d9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a1bd60 0x560fc9a15dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.979+0000 7fb1413d9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1381ac650 con 0x560fc9a1bd60 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.979+0000 7fb1413d9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb1381abf80 con 0x560fc9a18b80 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.979+0000 7fb1413d9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb1381a91a0 con 0x560fc9a10fc0 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.979+0000 7fb1413d9740 10 client.0 _ll_register_callbacks cb 0x7fb1300093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.979+0000 7fb13cd9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a18b80 0x560fc9a15860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.979+0000 7fb137fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a1bd60 0x560fc9a15dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.979+0000 7fb137fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a1bd60 0x560fc9a15dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55248/0 (socket says 172.21.15.16:55248) 2024-09-19T09:28:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.979+0000 7fb137fff640 1 -- 172.21.15.16:0/2099914408 learned_addr learned my addr 172.21.15.16:0/2099914408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.980+0000 7fb137fff640 1 --2- 172.21.15.16:0/2099914408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a10fc0 0x560fc9a24600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.980+0000 7fb13cd9c640 1 -- 172.21.15.16:0/2099914408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a10fc0 msgr2=0x560fc9a24600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.980+0000 7fb13cd9c640 1 --2- 172.21.15.16:0/2099914408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a10fc0 0x560fc9a24600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.980+0000 7fb13cd9c640 1 -- 172.21.15.16:0/2099914408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a1bd60 msgr2=0x560fc9a15dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.980+0000 7fb13cd9c640 1 --2- 172.21.15.16:0/2099914408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a1bd60 0x560fc9a15dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.980+0000 7fb13cd9c640 1 -- 172.21.15.16:0/2099914408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fc9a165f0 con 0x560fc9a18b80 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.980+0000 7fb137fff640 1 --2- 172.21.15.16:0/2099914408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a1bd60 0x560fc9a15dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.980+0000 7fb13cd9c640 1 --2- 172.21.15.16:0/2099914408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a18b80 0x560fc9a15860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x560fc9a1e5d0 tx=0x7fb12c005670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e939ba1bf626f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.980+0000 7fb117fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-19T09:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.980+0000 7fb117fff640 1 -- 172.21.15.16:0/2099914408 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c03f030 con 0x560fc9a18b80 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.980+0000 7fb117fff640 1 -- 172.21.15.16:0/2099914408 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb12c0049b0 con 0x560fc9a18b80 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.980+0000 7fb1413d9740 1 -- 172.21.15.16:0/2099914408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb12c004bb0 con 0x560fc9a18b80 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.981+0000 7fb117fff640 1 -- 172.21.15.16:0/2099914408 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb12c035040 con 0x560fc9a18b80 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.981+0000 7fb1413d9740 10 client.25238 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.981+0000 7fb1413d9740 1 -- 172.21.15.16:0/2099914408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560fc9a96800 con 0x560fc9a18b80 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.981+0000 7fb117fff640 1 -- 172.21.15.16:0/2099914408 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb12c004bb0 con 0x560fc9a18b80 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.981+0000 7fb117fff640 1 -- 172.21.15.16:0/2099914408 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb12c060020 con 0x560fc9a18b80 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.981+0000 7fb1413d9740 10 client.25238 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:16.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.981+0000 7fb1413d9740 10 client.25238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.981+0000 7fb1413d9740 20 client.25238 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.981+0000 7fb1413d9740 1 --2- 172.21.15.16:0/2099914408 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fc9a96eb0 0x560fc9a992d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.981+0000 7fb1413d9740 4 client.25238 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.981+0000 7fb1413d9740 1 -- 172.21.15.16:0/2099914408 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fc9975d70 con 0x560fc9a96eb0 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.982+0000 7fb137fff640 1 --2- 172.21.15.16:0/2099914408 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fc9a96eb0 0x560fc9a992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.982+0000 7fb117fff640 10 client.25238 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.982+0000 7fb137fff640 1 --2- 172.21.15.16:0/2099914408 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fc9a96eb0 0x560fc9a992d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1c865dd1010e025 server_cookie=3ae2a72b01101321 in_seq=0 out_seq=0 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.982+0000 7fb117fff640 1 -- 172.21.15.16:0/2099914408 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fc9975d70 con 0x560fc9a96eb0 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.982+0000 7fb117fff640 10 client.25238 handle_command_reply: tid=0 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.982+0000 7fb1413d9740 1 client.25238 shutdown 2024-09-19T09:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.983+0000 7fb1413d9740 1 -- 172.21.15.16:0/2099914408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a18b80 msgr2=0x560fc9a15860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.983+0000 7fb1413d9740 1 --2- 172.21.15.16:0/2099914408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a18b80 0x560fc9a15860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x560fc9a1e5d0 tx=0x7fb12c005670 comp rx=0 tx=0).stop 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.983+0000 7fb1413d9740 1 -- 172.21.15.16:0/2099914408 shutdown_connections 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.983+0000 7fb1413d9740 1 --2- 172.21.15.16:0/2099914408 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fc9a96eb0 0x560fc9a992d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.983+0000 7fb1413d9740 1 --2- 172.21.15.16:0/2099914408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fc9a1bd60 0x560fc9a15dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.983+0000 7fb1413d9740 1 --2- 172.21.15.16:0/2099914408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fc9a18b80 0x560fc9a15860 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.983+0000 7fb1413d9740 1 --2- 172.21.15.16:0/2099914408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fc9a10fc0 0x560fc9a24600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.983+0000 7fb1413d9740 1 -- 172.21.15.16:0/2099914408 >> 172.21.15.16:0/2099914408 conn(0x560fc99d1310 msgr2=0x560fc9a1a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.983+0000 7fb117fff640 0 client.25238 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.983+0000 7fb1413d9740 1 -- 172.21.15.16:0/2099914408 shutdown_connections 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.983+0000 7fb1413d9740 1 -- 172.21.15.16:0/2099914408 wait complete. 2024-09-19T09:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.983+0000 7fb1413d9740 20 client.25238 trim_cache size 0 max 16384 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.984+0000 7fb13dd9e640 1 -- 172.21.15.16:0/1219104683 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb1280a3240 msgr2=0x7fb1280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.984+0000 7fb13dd9e640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb1280a3240 0x7fb1280a5700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb1300cdaf0 tx=0x7fb130005f70 comp rx=0 tx=0).stop 2024-09-19T09:28:16.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.984+0000 7fb13dd9e640 1 -- 172.21.15.16:0/1219104683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1381068d0 msgr2=0x7fb1381979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.984+0000 7fb13dd9e640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1381068d0 0x7fb1381979b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb13000ecb0 tx=0x7fb13000c6a0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.984+0000 7fb13dd9e640 1 -- 172.21.15.16:0/1219104683 shutdown_connections 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.984+0000 7fb13dd9e640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb1280a3240 0x7fb1280a5700 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.984+0000 7fb13dd9e640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb138101270 0x7fb138197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.984+0000 7fb13dd9e640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1381068d0 0x7fb1381979b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.984+0000 7fb13dd9e640 1 --2- 172.21.15.16:0/1219104683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1381008d0 0x7fb138196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.984+0000 7fb13dd9e640 1 -- 172.21.15.16:0/1219104683 >> 172.21.15.16:0/1219104683 conn(0x7fb1380fc610 msgr2=0x7fb1380fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.984+0000 7fb13dd9e640 1 -- 172.21.15.16:0/1219104683 shutdown_connections 2024-09-19T09:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:16.984+0000 7fb13dd9e640 1 -- 172.21.15.16:0/1219104683 wait complete. 2024-09-19T09:28:17.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:17 smithi016 ceph-mon[30979]: pgmap v358: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T09:28:17.302 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:17 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1187985293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:17.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:17 smithi142 ceph-mon[35032]: pgmap v358: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T09:28:17.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:17 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1187985293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:17.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:17 smithi097 ceph-mon[34864]: pgmap v358: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-19T09:28:17.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:17 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1187985293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:17.503 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id a08a4233 --release --await' 2024-09-19T09:28:17.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id a08a4233 --release --await 2024-09-19T09:28:18.298 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.739+0000 7f0da460a640 1 -- 172.21.15.16:0/4262613695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9c107820 msgr2=0x7f0d9c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.739+0000 7f0da460a640 1 --2- 172.21.15.16:0/4262613695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9c107820 0x7f0d9c109c90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009ca0 tx=0x7f0d9402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.739+0000 7f0da460a640 1 -- 172.21.15.16:0/4262613695 shutdown_connections 2024-09-19T09:28:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.739+0000 7f0da460a640 1 --2- 172.21.15.16:0/4262613695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9c107820 0x7f0d9c109c90 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.739+0000 7f0da460a640 1 --2- 172.21.15.16:0/4262613695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d9c104ef0 0x7f0d9c1072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.739+0000 7f0da460a640 1 --2- 172.21.15.16:0/4262613695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c1025c0 0x7f0d9c1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.739+0000 7f0da460a640 1 -- 172.21.15.16:0/4262613695 >> 172.21.15.16:0/4262613695 conn(0x7f0d9c0fc1e0 msgr2=0x7f0d9c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.740+0000 7f0da460a640 1 -- 172.21.15.16:0/4262613695 shutdown_connections 2024-09-19T09:28:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.740+0000 7f0da460a640 1 -- 172.21.15.16:0/4262613695 wait complete. 2024-09-19T09:28:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.740+0000 7f0da460a640 1 Processor -- start 2024-09-19T09:28:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.740+0000 7f0da460a640 1 -- start start 2024-09-19T09:28:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da460a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d9c1025c0 0x7f0d9c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da460a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9c104ef0 0x7f0d9c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da460a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c107820 0x7f0d9c19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da460a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10d7e0 con 0x7f0d9c104ef0 2024-09-19T09:28:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da460a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10d660 con 0x7f0d9c1025c0 2024-09-19T09:28:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da1ba5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9c104ef0 0x7f0d9c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da23a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d9c1025c0 0x7f0d9c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da23a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d9c1025c0 0x7f0d9c197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:33176/0 (socket says 172.21.15.16:33176) 2024-09-19T09:28:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da23a6640 1 -- 172.21.15.16:0/2869408025 learned_addr learned my addr 172.21.15.16:0/2869408025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da1ba5640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9c104ef0 0x7f0d9c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55270/0 (socket says 172.21.15.16:55270) 2024-09-19T09:28:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da460a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10d960 con 0x7f0d9c107820 2024-09-19T09:28:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da2ba7640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c107820 0x7f0d9c19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da1ba5640 1 -- 172.21.15.16:0/2869408025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c107820 msgr2=0x7f0d9c19f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da1ba5640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c107820 0x7f0d9c19f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da1ba5640 1 -- 172.21.15.16:0/2869408025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d9c1025c0 msgr2=0x7f0d9c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da1ba5640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d9c1025c0 0x7f0d9c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.741+0000 7f0da1ba5640 1 -- 172.21.15.16:0/2869408025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c19fa50 con 0x7f0d9c104ef0 2024-09-19T09:28:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.742+0000 7f0da2ba7640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c107820 0x7f0d9c19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.742+0000 7f0da1ba5640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9c104ef0 0x7f0d9c198370 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00e9c0 tx=0x7f0d8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.742+0000 7f0d937fe640 1 -- 172.21.15.16:0/2869408025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00ccc0 con 0x7f0d9c104ef0 2024-09-19T09:28:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.742+0000 7f0d937fe640 1 -- 172.21.15.16:0/2869408025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d8c010910 con 0x7f0d9c104ef0 2024-09-19T09:28:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.742+0000 7f0d937fe640 1 -- 172.21.15.16:0/2869408025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c0188c0 con 0x7f0d9c104ef0 2024-09-19T09:28:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.742+0000 7f0da460a640 1 -- 172.21.15.16:0/2869408025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c19fd40 con 0x7f0d9c104ef0 2024-09-19T09:28:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.743+0000 7f0da460a640 1 -- 172.21.15.16:0/2869408025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c1a01e0 con 0x7f0d9c104ef0 2024-09-19T09:28:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.746+0000 7f0d937fe640 1 -- 172.21.15.16:0/2869408025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f0d8c010ac0 con 0x7f0d9c104ef0 2024-09-19T09:28:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.747+0000 7f0da7477740 1 Processor -- start 2024-09-19T09:28:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.747+0000 7f0da7477740 1 -- start start 2024-09-19T09:28:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.747+0000 7f0da7477740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633865319a0 0x563386531da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.747+0000 7f0da7477740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865322e0 0x563386541680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.747+0000 7f0da7477740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563386541bc0 0x563386544030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.747+0000 7f0da7477740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5633864972f0 con 0x5633865322e0 2024-09-19T09:28:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.747+0000 7f0da7477740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56338643f270 con 0x563386541bc0 2024-09-19T09:28:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.747+0000 7f0da7477740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5633864932b0 con 0x5633865319a0 2024-09-19T09:28:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.747+0000 7f0da23a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865322e0 0x563386541680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.747+0000 7f0da2ba7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633865319a0 0x563386531da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.747+0000 7f0da23a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865322e0 0x563386541680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55288/0 (socket says 172.21.15.16:55288) 2024-09-19T09:28:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.747+0000 7f0da23a6640 1 -- 172.21.15.16:0/2708858502 learned_addr learned my addr 172.21.15.16:0/2708858502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.748+0000 7f0da1ba5640 1 --2- 172.21.15.16:0/2708858502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563386541bc0 0x563386544030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.748+0000 7f0da23a6640 1 -- 172.21.15.16:0/2708858502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633865319a0 msgr2=0x563386531da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.748+0000 7f0da23a6640 1 --2- 172.21.15.16:0/2708858502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633865319a0 0x563386531da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.748+0000 7f0da23a6640 1 -- 172.21.15.16:0/2708858502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563386541bc0 msgr2=0x563386544030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.748+0000 7f0da23a6640 1 --2- 172.21.15.16:0/2708858502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563386541bc0 0x563386544030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.748+0000 7f0da23a6640 1 -- 172.21.15.16:0/2708858502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56338643e340 con 0x5633865322e0 2024-09-19T09:28:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.748+0000 7f0da23a6640 1 --2- 172.21.15.16:0/2708858502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865322e0 0x563386541680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800bda0 tx=0x7f0d980107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d9ba508549730c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.748+0000 7f0d937fe640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0d880a3460 0x7f0d880a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.749+0000 7f0d90bfe640 1 -- 172.21.15.16:0/2708858502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9801c680 con 0x5633865322e0 2024-09-19T09:28:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.749+0000 7f0d90bfe640 1 -- 172.21.15.16:0/2708858502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d98011920 con 0x5633865322e0 2024-09-19T09:28:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.749+0000 7f0d90bfe640 1 -- 172.21.15.16:0/2708858502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9801a930 con 0x5633865322e0 2024-09-19T09:28:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.749+0000 7f0d937fe640 1 -- 172.21.15.16:0/2869408025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0d8c0cfde0 con 0x7f0d9c104ef0 2024-09-19T09:28:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.749+0000 7f0da2ba7640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0d880a3460 0x7f0d880a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.749+0000 7f0da7477740 1 -- 172.21.15.16:0/2708858502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865322e0 msgr2=0x563386541680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.749+0000 7f0da7477740 1 --2- 172.21.15.16:0/2708858502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865322e0 0x563386541680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800bda0 tx=0x7f0d980107a0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.750+0000 7f0da2ba7640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0d880a3460 0x7f0d880a5920 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0d94005cc0 tx=0x7f0d94005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.750+0000 7f0da7477740 1 -- 172.21.15.16:0/2708858502 shutdown_connections 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.750+0000 7f0da7477740 1 --2- 172.21.15.16:0/2708858502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563386541bc0 0x563386544030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.750+0000 7f0da7477740 1 --2- 172.21.15.16:0/2708858502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865322e0 0x563386541680 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.750+0000 7f0da7477740 1 --2- 172.21.15.16:0/2708858502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633865319a0 0x563386531da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.750+0000 7f0da7477740 1 -- 172.21.15.16:0/2708858502 >> 172.21.15.16:0/2708858502 conn(0x56338652ef90 msgr2=0x56338652f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.750+0000 7f0da7477740 1 -- 172.21.15.16:0/2708858502 shutdown_connections 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.750+0000 7f0da7477740 1 -- 172.21.15.16:0/2708858502 wait complete. 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.750+0000 7f0da7477740 1 -- start start 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.750+0000 7f0da7477740 1 Processor -- start 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.750+0000 7f0da7477740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 0x563386546480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.750+0000 7f0da7477740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865322e0 0x5633865469c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da7477740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563386541bc0 0x56338654da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da23a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 0x563386546480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da23a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 0x563386546480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55290/0 (socket says 172.21.15.16:55290) 2024-09-19T09:28:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da23a6640 1 -- 172.21.15.16:0/1922487076 learned_addr learned my addr 172.21.15.16:0/1922487076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da7477740 1 -- 172.21.15.16:0/1922487076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563386544b40 con 0x5633865319a0 2024-09-19T09:28:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da7477740 1 -- 172.21.15.16:0/1922487076 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563386420290 con 0x5633865322e0 2024-09-19T09:28:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da7477740 1 -- 172.21.15.16:0/1922487076 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563386544cc0 con 0x563386541bc0 2024-09-19T09:28:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da1ba5640 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865322e0 0x5633865469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da7477740 10 client.0 _ll_register_callbacks cb 0x563386451140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da2ba7640 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563386541bc0 0x56338654da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da23a6640 1 -- 172.21.15.16:0/1922487076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563386541bc0 msgr2=0x56338654da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da23a6640 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563386541bc0 0x56338654da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da23a6640 1 -- 172.21.15.16:0/1922487076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865322e0 msgr2=0x5633865469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da23a6640 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865322e0 0x5633865469c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da23a6640 1 -- 172.21.15.16:0/1922487076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633861aa210 con 0x5633865319a0 2024-09-19T09:28:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.751+0000 7f0da2ba7640 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563386541bc0 0x56338654da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.752+0000 7f0da23a6640 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 0x563386546480 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800bd70 tx=0x7f0d98000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c4ce6c9885e736 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.752+0000 7f0d85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.752+0000 7f0d85ffb640 1 -- 172.21.15.16:0/1922487076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9802a060 con 0x5633865319a0 2024-09-19T09:28:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.752+0000 7f0d85ffb640 1 -- 172.21.15.16:0/1922487076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0d98011650 con 0x5633865319a0 2024-09-19T09:28:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.752+0000 7f0da7477740 1 -- 172.21.15.16:0/1922487076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d98011920 con 0x5633865319a0 2024-09-19T09:28:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.752+0000 7f0d85ffb640 1 -- 172.21.15.16:0/1922487076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d98005040 con 0x5633865319a0 2024-09-19T09:28:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.752+0000 7f0da7477740 10 client.15624 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.752+0000 7f0da7477740 1 -- 172.21.15.16:0/1922487076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563386276620 con 0x5633865319a0 2024-09-19T09:28:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.753+0000 7f0d85ffb640 1 -- 172.21.15.16:0/1922487076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d98011920 con 0x5633865319a0 2024-09-19T09:28:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.753+0000 7f0d85ffb640 1 -- 172.21.15.16:0/1922487076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0d9802f070 con 0x5633865319a0 2024-09-19T09:28:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.753+0000 7f0da7477740 10 client.15624 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.753+0000 7f0da7477740 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.753+0000 7f0da7477740 20 client.15624 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.753+0000 7f0da7477740 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563386551e40 0x563386554230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.753+0000 7f0da7477740 4 client.15624 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.753+0000 7f0da7477740 1 -- 172.21.15.16:0/1922487076 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633864a4a20 con 0x563386551e40 2024-09-19T09:28:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.753+0000 7f0da1ba5640 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563386551e40 0x563386554230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.754+0000 7f0da1ba5640 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563386551e40 0x563386554230 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fa9d7d9b08d658f server_cookie=5f63c1399eb7f136 in_seq=0 out_seq=0 2024-09-19T09:28:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.754+0000 7f0d85ffb640 10 client.15624 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.754+0000 7f0d85ffb640 1 -- 172.21.15.16:0/1922487076 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633864a4a20 con 0x563386551e40 2024-09-19T09:28:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.754+0000 7f0d85ffb640 10 client.15624 handle_command_reply: tid=0 2024-09-19T09:28:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.754+0000 7f0da7477740 1 client.15624 shutdown 2024-09-19T09:28:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.755+0000 7f0da7477740 1 -- 172.21.15.16:0/1922487076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 msgr2=0x563386546480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.755+0000 7f0da7477740 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 0x563386546480 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800bd70 tx=0x7f0d98000bc0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.755+0000 7f0da7477740 1 -- 172.21.15.16:0/1922487076 shutdown_connections 2024-09-19T09:28:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.755+0000 7f0da7477740 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563386551e40 0x563386554230 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.755+0000 7f0d85ffb640 0 client.15624 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:18.760 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.755+0000 7f0da7477740 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563386541bc0 0x56338654da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.755+0000 7f0da7477740 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865322e0 0x5633865469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.755+0000 7f0da7477740 1 --2- 172.21.15.16:0/1922487076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 0x563386546480 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.755+0000 7f0da7477740 1 -- 172.21.15.16:0/1922487076 >> 172.21.15.16:0/1922487076 conn(0x56338652ef90 msgr2=0x5633865307a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.755+0000 7f0da7477740 1 -- 172.21.15.16:0/1922487076 shutdown_connections 2024-09-19T09:28:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.755+0000 7f0da7477740 1 -- 172.21.15.16:0/1922487076 wait complete. 2024-09-19T09:28:18.761 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.755+0000 7f0da7477740 20 client.15624 trim_cache size 0 max 16384 2024-09-19T09:28:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.784+0000 7f0da7477740 1 Processor -- start 2024-09-19T09:28:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.784+0000 7f0da7477740 1 -- start start 2024-09-19T09:28:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.784+0000 7f0da7477740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865319a0 0x5633865468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.784+0000 7f0da7477740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633865322e0 0x56338654d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.784+0000 7f0da7477740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563386541bc0 0x563386558a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.784+0000 7f0da7477740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1a6e40 con 0x563386541bc0 2024-09-19T09:28:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.784+0000 7f0da7477740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1a8980 con 0x5633865319a0 2024-09-19T09:28:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.784+0000 7f0da7477740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1a7510 con 0x5633865322e0 2024-09-19T09:28:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.784+0000 7f0da2ba7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563386541bc0 0x563386558a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.784+0000 7f0da2ba7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563386541bc0 0x563386558a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55292/0 (socket says 172.21.15.16:55292) 2024-09-19T09:28:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.784+0000 7f0da2ba7640 1 -- 172.21.15.16:0/2025367857 learned_addr learned my addr 172.21.15.16:0/2025367857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.784+0000 7f0da23a6640 1 --2- 172.21.15.16:0/2025367857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865319a0 0x5633865468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.784+0000 7f0da1ba5640 1 --2- 172.21.15.16:0/2025367857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633865322e0 0x56338654d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.785+0000 7f0da23a6640 1 -- 172.21.15.16:0/2025367857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633865322e0 msgr2=0x56338654d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.785+0000 7f0da23a6640 1 --2- 172.21.15.16:0/2025367857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633865322e0 0x56338654d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.785+0000 7f0da23a6640 1 -- 172.21.15.16:0/2025367857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563386541bc0 msgr2=0x563386558a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.785+0000 7f0da23a6640 1 --2- 172.21.15.16:0/2025367857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563386541bc0 0x563386558a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.785+0000 7f0da23a6640 1 -- 172.21.15.16:0/2025367857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633865320e0 con 0x5633865319a0 2024-09-19T09:28:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.785+0000 7f0da2ba7640 1 --2- 172.21.15.16:0/2025367857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563386541bc0 0x563386558a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.785+0000 7f0da23a6640 1 --2- 172.21.15.16:0/2025367857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865319a0 0x5633865468a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0d980107e0 tx=0x7f0d98005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5013bf7b40ea8ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.785+0000 7f0d87fff640 1 -- 172.21.15.16:0/2025367857 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9802a040 con 0x5633865319a0 2024-09-19T09:28:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.785+0000 7f0d87fff640 1 -- 172.21.15.16:0/2025367857 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d9801a8d0 con 0x5633865319a0 2024-09-19T09:28:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.785+0000 7f0d87fff640 1 -- 172.21.15.16:0/2025367857 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98011650 con 0x5633865319a0 2024-09-19T09:28:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.786+0000 7f0da7477740 1 -- 172.21.15.16:0/2025367857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865319a0 msgr2=0x5633865468a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.786+0000 7f0da7477740 1 --2- 172.21.15.16:0/2025367857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865319a0 0x5633865468a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0d980107e0 tx=0x7f0d98005900 comp rx=0 tx=0).stop 2024-09-19T09:28:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.786+0000 7f0da7477740 1 -- 172.21.15.16:0/2025367857 shutdown_connections 2024-09-19T09:28:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.786+0000 7f0da7477740 1 --2- 172.21.15.16:0/2025367857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633865322e0 0x56338654d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.786+0000 7f0da7477740 1 --2- 172.21.15.16:0/2025367857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563386541bc0 0x563386558a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.786+0000 7f0da7477740 1 --2- 172.21.15.16:0/2025367857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865319a0 0x5633865468a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.786+0000 7f0da7477740 1 -- 172.21.15.16:0/2025367857 >> 172.21.15.16:0/2025367857 conn(0x56338652ef90 msgr2=0x56338652e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.786+0000 7f0da7477740 1 -- 172.21.15.16:0/2025367857 shutdown_connections 2024-09-19T09:28:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.786+0000 7f0da7477740 1 -- 172.21.15.16:0/2025367857 wait complete. 2024-09-19T09:28:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.787+0000 7f0da7477740 1 -- start start 2024-09-19T09:28:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.787+0000 7f0da7477740 1 Processor -- start 2024-09-19T09:28:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.787+0000 7f0da7477740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 0x56338654b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.787+0000 7f0da7477740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865322e0 0x56338654b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.787+0000 7f0da7477740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563386541bc0 0x56338654c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.787+0000 7f0da23a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 0x56338654b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.787+0000 7f0da23a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 0x56338654b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55296/0 (socket says 172.21.15.16:55296) 2024-09-19T09:28:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.787+0000 7f0da23a6640 1 -- 172.21.15.16:0/1231527059 learned_addr learned my addr 172.21.15.16:0/1231527059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.787+0000 7f0da7477740 1 -- 172.21.15.16:0/1231527059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1a82b0 con 0x5633865319a0 2024-09-19T09:28:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.787+0000 7f0da7477740 1 -- 172.21.15.16:0/1231527059 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1a7be0 con 0x5633865322e0 2024-09-19T09:28:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.787+0000 7f0da7477740 1 -- 172.21.15.16:0/1231527059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1a4e00 con 0x563386541bc0 2024-09-19T09:28:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.787+0000 7f0da2ba7640 1 --2- 172.21.15.16:0/1231527059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563386541bc0 0x56338654c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.787+0000 7f0da1ba5640 1 --2- 172.21.15.16:0/1231527059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865322e0 0x56338654b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0da23a6640 1 -- 172.21.15.16:0/1231527059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563386541bc0 msgr2=0x56338654c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0da23a6640 1 --2- 172.21.15.16:0/1231527059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563386541bc0 0x56338654c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0da23a6640 1 -- 172.21.15.16:0/1231527059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865322e0 msgr2=0x56338654b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0da23a6640 1 --2- 172.21.15.16:0/1231527059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865322e0 0x56338654b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0da23a6640 1 -- 172.21.15.16:0/1231527059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56338654ccf0 con 0x5633865319a0 2024-09-19T09:28:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0da7477740 10 client.0 _ll_register_callbacks cb 0x7f0d8c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0da7477740 1 -- 172.21.15.16:0/1231527059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d9801a8d0 con 0x5633865319a0 2024-09-19T09:28:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0da23a6640 1 --2- 172.21.15.16:0/1231527059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 0x56338654b280 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0d98011010 tx=0x7f0d98005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89dfce1b616907c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0d85ffb640 10 client.15630 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0d85ffb640 1 -- 172.21.15.16:0/1231527059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9803f050 con 0x5633865319a0 2024-09-19T09:28:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0d85ffb640 1 -- 172.21.15.16:0/1231527059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f0d98011650 con 0x5633865319a0 2024-09-19T09:28:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0d85ffb640 1 -- 172.21.15.16:0/1231527059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d98035040 con 0x5633865319a0 2024-09-19T09:28:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0da7477740 10 client.15630 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.788+0000 7f0da7477740 1 -- 172.21.15.16:0/1231527059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5633865bcac0 con 0x5633865319a0 2024-09-19T09:28:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.789+0000 7f0d85ffb640 1 -- 172.21.15.16:0/1231527059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0d9802f070 con 0x5633865319a0 2024-09-19T09:28:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.789+0000 7f0da7477740 10 client.15630 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.789+0000 7f0da7477740 10 client.15630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.789+0000 7f0da7477740 20 client.15630 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.789+0000 7f0da7477740 1 --2- 172.21.15.16:0/1231527059 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563386551e40 0x5633865bf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.789+0000 7f0da7477740 4 client.15630 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:28:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.789+0000 7f0da7477740 1 -- 172.21.15.16:0/1231527059 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56338649ed70 con 0x563386551e40 2024-09-19T09:28:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.789+0000 7f0da1ba5640 1 --2- 172.21.15.16:0/1231527059 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563386551e40 0x5633865bf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.790+0000 7f0d85ffb640 10 client.15630 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.790+0000 7f0da1ba5640 1 --2- 172.21.15.16:0/1231527059 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563386551e40 0x5633865bf450 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d0b378982abc677 server_cookie=84032403c4588677 in_seq=0 out_seq=0 2024-09-19T09:28:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0d85ffb640 1 -- 172.21.15.16:0/1231527059 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56338649ed70 con 0x563386551e40 2024-09-19T09:28:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0d85ffb640 10 client.15630 handle_command_reply: tid=0 2024-09-19T09:28:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0da7477740 1 client.15630 shutdown 2024-09-19T09:28:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0da7477740 1 -- 172.21.15.16:0/1231527059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 msgr2=0x56338654b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0da7477740 1 --2- 172.21.15.16:0/1231527059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 0x56338654b280 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0d98011010 tx=0x7f0d98005ec0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0da7477740 1 -- 172.21.15.16:0/1231527059 shutdown_connections 2024-09-19T09:28:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0da7477740 1 --2- 172.21.15.16:0/1231527059 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563386551e40 0x5633865bf450 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0da7477740 1 --2- 172.21.15.16:0/1231527059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563386541bc0 0x56338654c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0da7477740 1 --2- 172.21.15.16:0/1231527059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633865322e0 0x56338654b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0da7477740 1 --2- 172.21.15.16:0/1231527059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633865319a0 0x56338654b280 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0da7477740 1 -- 172.21.15.16:0/1231527059 >> 172.21.15.16:0/1231527059 conn(0x56338652ef90 msgr2=0x56338652fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0d85ffb640 0 client.15630 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0da7477740 1 -- 172.21.15.16:0/1231527059 shutdown_connections 2024-09-19T09:28:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0da7477740 1 -- 172.21.15.16:0/1231527059 wait complete. 2024-09-19T09:28:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.798+0000 7f0da7477740 20 client.15630 trim_cache size 0 max 16384 2024-09-19T09:28:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-19T09:28:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-19T09:28:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-19T09:28:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-19T09:28:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.799+0000 7f0da460a640 1 -- 172.21.15.16:0/2869408025 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0d880a3460 msgr2=0x7f0d880a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.799+0000 7f0da460a640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0d880a3460 0x7f0d880a5920 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0d94005cc0 tx=0x7f0d94005c30 comp rx=0 tx=0).stop 2024-09-19T09:28:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.799+0000 7f0da460a640 1 -- 172.21.15.16:0/2869408025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9c104ef0 msgr2=0x7f0d9c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.799+0000 7f0da460a640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9c104ef0 0x7f0d9c198370 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00e9c0 tx=0x7f0d8c00ee80 comp rx=0 tx=0).stop 2024-09-19T09:28:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.799+0000 7f0da460a640 1 -- 172.21.15.16:0/2869408025 shutdown_connections 2024-09-19T09:28:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.799+0000 7f0da460a640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f0d880a3460 0x7f0d880a5920 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.799+0000 7f0da460a640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d9c107820 0x7f0d9c19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.799+0000 7f0da460a640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9c104ef0 0x7f0d9c198370 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.799+0000 7f0da460a640 1 --2- 172.21.15.16:0/2869408025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d9c1025c0 0x7f0d9c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.799+0000 7f0da460a640 1 -- 172.21.15.16:0/2869408025 >> 172.21.15.16:0/2869408025 conn(0x7f0d9c0fc1e0 msgr2=0x7f0d9c102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.800+0000 7f0da460a640 1 -- 172.21.15.16:0/2869408025 shutdown_connections 2024-09-19T09:28:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:18.800+0000 7f0da460a640 1 -- 172.21.15.16:0/2869408025 wait complete. 2024-09-19T09:28:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.829+0000 7f4acc9a6640 1 -- 172.21.15.16:0/1250053941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac8101de0 msgr2=0x7f4ac81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.829+0000 7f4acc9a6640 1 --2- 172.21.15.16:0/1250053941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac8101de0 0x7f4ac81021c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009900 tx=0x7f4ab802a410 comp rx=0 tx=0).stop 2024-09-19T09:28:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.830+0000 7f4acc9a6640 1 -- 172.21.15.16:0/1250053941 shutdown_connections 2024-09-19T09:28:18.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.830+0000 7f4acc9a6640 1 --2- 172.21.15.16:0/1250053941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ac8106f10 0x7f4ac810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.830+0000 7f4acc9a6640 1 --2- 172.21.15.16:0/1250053941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ac8102790 0x7f4ac81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.830+0000 7f4acc9a6640 1 --2- 172.21.15.16:0/1250053941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac8101de0 0x7f4ac81021c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.830+0000 7f4acc9a6640 1 -- 172.21.15.16:0/1250053941 >> 172.21.15.16:0/1250053941 conn(0x7f4ac80fd8c0 msgr2=0x7f4ac80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.830+0000 7f4acc9a6640 1 -- 172.21.15.16:0/1250053941 shutdown_connections 2024-09-19T09:28:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.830+0000 7f4acc9a6640 1 -- 172.21.15.16:0/1250053941 wait complete. 2024-09-19T09:28:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.831+0000 7f4acc9a6640 1 Processor -- start 2024-09-19T09:28:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.831+0000 7f4acc9a6640 1 -- start start 2024-09-19T09:28:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.831+0000 7f4acc9a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ac8101de0 0x7f4ac80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.831+0000 7f4acc9a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac8102790 0x7f4ac8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.831+0000 7f4acc9a6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ac8106f10 0x7f4ac8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.831+0000 7f4ac5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac8102790 0x7f4ac8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.831+0000 7f4acc9a6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8111fc0 con 0x7f4ac8102790 2024-09-19T09:28:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.831+0000 7f4ac5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac8102790 0x7f4ac8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55314/0 (socket says 172.21.15.16:55314) 2024-09-19T09:28:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.831+0000 7f4ac5d9b640 1 -- 172.21.15.16:0/3400884160 learned_addr learned my addr 172.21.15.16:0/3400884160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:18.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.831+0000 7f4acc9a6640 1 -- 172.21.15.16:0/3400884160 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8111e40 con 0x7f4ac8101de0 2024-09-19T09:28:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.831+0000 7f4acc9a6640 1 -- 172.21.15.16:0/3400884160 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8112140 con 0x7f4ac8106f10 2024-09-19T09:28:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.832+0000 7f4ac6d9d640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ac8106f10 0x7f4ac8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.832+0000 7f4ac5d9b640 1 -- 172.21.15.16:0/3400884160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ac8106f10 msgr2=0x7f4ac8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.832+0000 7f4ac5d9b640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ac8106f10 0x7f4ac8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.832+0000 7f4ac5d9b640 1 -- 172.21.15.16:0/3400884160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ac8101de0 msgr2=0x7f4ac80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.832+0000 7f4ac659c640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ac8101de0 0x7f4ac80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.832+0000 7f4ac5d9b640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ac8101de0 0x7f4ac80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.832+0000 7f4ac5d9b640 1 -- 172.21.15.16:0/3400884160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac81a68f0 con 0x7f4ac8102790 2024-09-19T09:28:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.832+0000 7f4ac5d9b640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac8102790 0x7f4ac8074a30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00ed60 tx=0x7f4abc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.833+0000 7f4ab77fe640 1 -- 172.21.15.16:0/3400884160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc009e10 con 0x7f4ac8102790 2024-09-19T09:28:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.833+0000 7f4ab77fe640 1 -- 172.21.15.16:0/3400884160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4abc010910 con 0x7f4ac8102790 2024-09-19T09:28:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.833+0000 7f4acc9a6640 1 -- 172.21.15.16:0/3400884160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac81a6be0 con 0x7f4ac8102790 2024-09-19T09:28:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.833+0000 7f4acc9a6640 1 -- 172.21.15.16:0/3400884160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac81a7170 con 0x7f4ac8102790 2024-09-19T09:28:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.833+0000 7f4ab77fe640 1 -- 172.21.15.16:0/3400884160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc018990 con 0x7f4ac8102790 2024-09-19T09:28:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.836+0000 7f4aced7f740 1 Processor -- start 2024-09-19T09:28:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.836+0000 7f4aced7f740 1 -- start start 2024-09-19T09:28:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4aced7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a280361870 0x55a280361c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4aced7f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280362190 0x55a28036c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4aced7f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a28036ca50 0x55a28036ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4aced7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a2802be620 con 0x55a280361870 2024-09-19T09:28:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4aced7f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a280265e50 con 0x55a28036ca50 2024-09-19T09:28:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4aced7f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a2802ba5e0 con 0x55a280362190 2024-09-19T09:28:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4ab77fe640 1 -- 172.21.15.16:0/3400884160 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4abc010ac0 con 0x7f4ac8102790 2024-09-19T09:28:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4ac6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a280361870 0x55a280361c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4ac5d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a28036ca50 0x55a28036ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4ac6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a280361870 0x55a280361c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55320/0 (socket says 172.21.15.16:55320) 2024-09-19T09:28:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4ac6d9d640 1 -- 172.21.15.16:0/3779081300 learned_addr learned my addr 172.21.15.16:0/3779081300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4ac659c640 1 --2- 172.21.15.16:0/3779081300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280362190 0x55a28036c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4ac6d9d640 1 -- 172.21.15.16:0/3779081300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280362190 msgr2=0x55a28036c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4ac6d9d640 1 --2- 172.21.15.16:0/3779081300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280362190 0x55a28036c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4ac6d9d640 1 -- 172.21.15.16:0/3779081300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a28036ca50 msgr2=0x55a28036ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4ac6d9d640 1 --2- 172.21.15.16:0/3779081300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a28036ca50 0x55a28036ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.837+0000 7f4ac6d9d640 1 -- 172.21.15.16:0/3779081300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a280256d40 con 0x55a280361870 2024-09-19T09:28:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.838+0000 7f4ac6d9d640 1 --2- 172.21.15.16:0/3779081300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a280361870 0x55a280361c50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009d70 tx=0x7f4ac0020740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4d4ca89a940c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.838+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/3779081300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0003b70 con 0x55a280361870 2024-09-19T09:28:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.838+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/3779081300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac0020e50 con 0x55a280361870 2024-09-19T09:28:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.838+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/3779081300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00227d0 con 0x55a280361870 2024-09-19T09:28:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.839+0000 7f4ab77fe640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4ab00a3340 0x7f4ab00a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.839+0000 7f4aced7f740 1 -- 172.21.15.16:0/3779081300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a280361870 msgr2=0x55a280361c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.839+0000 7f4aced7f740 1 --2- 172.21.15.16:0/3779081300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a280361870 0x55a280361c50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009d70 tx=0x7f4ac0020740 comp rx=0 tx=0).stop 2024-09-19T09:28:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.839+0000 7f4aced7f740 1 -- 172.21.15.16:0/3779081300 shutdown_connections 2024-09-19T09:28:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.839+0000 7f4aced7f740 1 --2- 172.21.15.16:0/3779081300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280362190 0x55a28036c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.839+0000 7f4aced7f740 1 --2- 172.21.15.16:0/3779081300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a28036ca50 0x55a28036ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.839+0000 7f4aced7f740 1 --2- 172.21.15.16:0/3779081300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a280361870 0x55a280361c50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.839+0000 7f4aced7f740 1 -- 172.21.15.16:0/3779081300 >> 172.21.15.16:0/3779081300 conn(0x55a28035de00 msgr2=0x55a28035e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.839+0000 7f4ab77fe640 1 -- 172.21.15.16:0/3400884160 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4abc0cfe90 con 0x7f4ac8102790 2024-09-19T09:28:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.839+0000 7f4aced7f740 1 -- 172.21.15.16:0/3779081300 shutdown_connections 2024-09-19T09:28:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.839+0000 7f4aced7f740 1 -- 172.21.15.16:0/3779081300 wait complete. 2024-09-19T09:28:18.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.840+0000 7f4aced7f740 1 -- start start 2024-09-19T09:28:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.840+0000 7f4aced7f740 1 Processor -- start 2024-09-19T09:28:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.840+0000 7f4ac659c640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4ab00a3340 0x7f4ab00a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.840+0000 7f4aced7f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280361870 0x55a280369960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.840+0000 7f4aced7f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280362190 0x55a280369ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.841+0000 7f4aced7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28036ca50 0x55a28036a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.841+0000 7f4ac6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28036ca50 0x55a28036a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.841+0000 7f4ac6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28036ca50 0x55a28036a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55322/0 (socket says 172.21.15.16:55322) 2024-09-19T09:28:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.841+0000 7f4ac5d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280362190 0x55a280369ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.841+0000 7f4ac5d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280362190 0x55a280369ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:33242/0 (socket says 172.21.15.16:33242) 2024-09-19T09:28:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.841+0000 7f4ac659c640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4ab00a3340 0x7f4ab00a5800 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8006010 tx=0x7f4ab8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.841+0000 7f4ac5d9b640 1 -- 172.21.15.16:0/1523103877 learned_addr learned my addr 172.21.15.16:0/1523103877 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.841+0000 7f4aced7f740 1 -- 172.21.15.16:0/1523103877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a28036f620 con 0x55a28036ca50 2024-09-19T09:28:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.841+0000 7f4aced7f740 1 -- 172.21.15.16:0/1523103877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a280246f10 con 0x55a280362190 2024-09-19T09:28:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.841+0000 7f4aced7f740 1 -- 172.21.15.16:0/1523103877 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a28036f7a0 con 0x55a280361870 2024-09-19T09:28:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.841+0000 7f4aced7f740 10 client.0 _ll_register_callbacks cb 0x55a28026adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.841+0000 7f4ac659c640 1 --2- 172.21.15.16:0/1523103877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280361870 0x55a280369960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.842+0000 7f4ac6d9d640 1 -- 172.21.15.16:0/1523103877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280361870 msgr2=0x55a280369960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.842+0000 7f4ac6d9d640 1 --2- 172.21.15.16:0/1523103877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280361870 0x55a280369960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.842+0000 7f4ac6d9d640 1 -- 172.21.15.16:0/1523103877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280362190 msgr2=0x55a280369ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.842+0000 7f4ac6d9d640 1 --2- 172.21.15.16:0/1523103877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280362190 0x55a280369ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.842+0000 7f4ac6d9d640 1 -- 172.21.15.16:0/1523103877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a27ffcd580 con 0x55a28036ca50 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.842+0000 7f4ac6d9d640 1 --2- 172.21.15.16:0/1523103877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28036ca50 0x55a28036a3e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006010 tx=0x7f4ac0004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8d86daacee7f07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.842+0000 7f4aadffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.842+0000 7f4aadffb640 1 -- 172.21.15.16:0/1523103877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac003b060 con 0x55a28036ca50 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.842+0000 7f4aced7f740 1 -- 172.21.15.16:0/1523103877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac0020e50 con 0x55a28036ca50 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.842+0000 7f4aadffb640 1 -- 172.21.15.16:0/1523103877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4ac0003970 con 0x55a28036ca50 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.843+0000 7f4aadffb640 1 -- 172.21.15.16:0/1523103877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ac00043a0 con 0x55a28036ca50 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.843+0000 7f4aced7f740 10 client.15642 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.843+0000 7f4aced7f740 1 -- 172.21.15.16:0/1523103877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a280097960 con 0x55a28036ca50 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.843+0000 7f4aadffb640 1 -- 172.21.15.16:0/1523103877 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac0020e50 con 0x55a28036ca50 2024-09-19T09:28:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.843+0000 7f4aadffb640 1 -- 172.21.15.16:0/1523103877 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4ac0040070 con 0x55a28036ca50 2024-09-19T09:28:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.843+0000 7f4aced7f740 10 client.15642 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.843+0000 7f4aced7f740 10 client.15642 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:28:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.843+0000 7f4aced7f740 20 client.15642 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.843+0000 7f4aced7f740 1 --2- 172.21.15.16:0/1523103877 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a28037c500 0x55a28037e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.843+0000 7f4aced7f740 4 client.15642 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.843+0000 7f4aced7f740 1 -- 172.21.15.16:0/1523103877 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2802cbb90 con 0x55a28037c500 2024-09-19T09:28:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.844+0000 7f4ac659c640 1 --2- 172.21.15.16:0/1523103877 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a28037c500 0x55a28037e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.844+0000 7f4ac659c640 1 --2- 172.21.15.16:0/1523103877 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a28037c500 0x55a28037e8f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f58fe48f9dd899 server_cookie=879f07b214eaa61c in_seq=0 out_seq=0 2024-09-19T09:28:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.844+0000 7f4aadffb640 10 client.15642 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aadffb640 1 -- 172.21.15.16:0/1523103877 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2802cbb90 con 0x55a28037c500 2024-09-19T09:28:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aadffb640 10 client.15642 handle_command_reply: tid=0 2024-09-19T09:28:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aced7f740 1 client.15642 shutdown 2024-09-19T09:28:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aced7f740 1 -- 172.21.15.16:0/1523103877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28036ca50 msgr2=0x55a28036a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aced7f740 1 --2- 172.21.15.16:0/1523103877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28036ca50 0x55a28036a3e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006010 tx=0x7f4ac0004230 comp rx=0 tx=0).stop 2024-09-19T09:28:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aced7f740 1 -- 172.21.15.16:0/1523103877 shutdown_connections 2024-09-19T09:28:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aced7f740 1 --2- 172.21.15.16:0/1523103877 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a28037c500 0x55a28037e8f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aced7f740 1 --2- 172.21.15.16:0/1523103877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28036ca50 0x55a28036a3e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aced7f740 1 --2- 172.21.15.16:0/1523103877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280362190 0x55a280369ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aced7f740 1 --2- 172.21.15.16:0/1523103877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280361870 0x55a280369960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aced7f740 1 -- 172.21.15.16:0/1523103877 >> 172.21.15.16:0/1523103877 conn(0x55a28035de00 msgr2=0x55a280363b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aadffb640 0 client.15642 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aced7f740 1 -- 172.21.15.16:0/1523103877 shutdown_connections 2024-09-19T09:28:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aced7f740 1 -- 172.21.15.16:0/1523103877 wait complete. 2024-09-19T09:28:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.845+0000 7f4aced7f740 20 client.15642 trim_cache size 0 max 16384 2024-09-19T09:28:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.862+0000 7f4aced7f740 1 Processor -- start 2024-09-19T09:28:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.862+0000 7f4aced7f740 1 -- start start 2024-09-19T09:28:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.863+0000 7f4aced7f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280361870 0x55a280364370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.863+0000 7f4aced7f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280362190 0x55a280354640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.863+0000 7f4aced7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28036ca50 0x55a280366bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.863+0000 7f4ac659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280361870 0x55a280364370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.863+0000 7f4ac659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280361870 0x55a280364370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:33248/0 (socket says 172.21.15.16:33248) 2024-09-19T09:28:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.863+0000 7f4ac659c640 1 -- 172.21.15.16:0/521307404 learned_addr learned my addr 172.21.15.16:0/521307404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.863+0000 7f4aced7f740 1 -- 172.21.15.16:0/521307404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac806fd20 con 0x55a28036ca50 2024-09-19T09:28:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.863+0000 7f4ac6d9d640 1 --2- 172.21.15.16:0/521307404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28036ca50 0x55a280366bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.863+0000 7f4ac5d9b640 1 --2- 172.21.15.16:0/521307404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280362190 0x55a280354640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.863+0000 7f4aced7f740 1 -- 172.21.15.16:0/521307404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8071860 con 0x55a280361870 2024-09-19T09:28:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.863+0000 7f4aced7f740 1 -- 172.21.15.16:0/521307404 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac80703f0 con 0x55a280362190 2024-09-19T09:28:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.864+0000 7f4ac5d9b640 1 -- 172.21.15.16:0/521307404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280361870 msgr2=0x55a280364370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.864+0000 7f4ac5d9b640 1 --2- 172.21.15.16:0/521307404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280361870 0x55a280364370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.864+0000 7f4ac5d9b640 1 -- 172.21.15.16:0/521307404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28036ca50 msgr2=0x55a280366bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.864+0000 7f4ac5d9b640 1 --2- 172.21.15.16:0/521307404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28036ca50 0x55a280366bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.864+0000 7f4ac5d9b640 1 -- 172.21.15.16:0/521307404 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a28036f1a0 con 0x55a280362190 2024-09-19T09:28:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.864+0000 7f4ac6d9d640 1 --2- 172.21.15.16:0/521307404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28036ca50 0x55a280366bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.865+0000 7f4ac5d9b640 1 --2- 172.21.15.16:0/521307404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280362190 0x55a280354640 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4abc098e40 tx=0x7f4abc0d3850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=875264324718e2cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.865+0000 7f4aaffff640 1 -- 172.21.15.16:0/521307404 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc09bfb0 con 0x55a280362190 2024-09-19T09:28:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.865+0000 7f4aaffff640 1 -- 172.21.15.16:0/521307404 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4abc09cb70 con 0x55a280362190 2024-09-19T09:28:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.865+0000 7f4aaffff640 1 -- 172.21.15.16:0/521307404 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc09bdd0 con 0x55a280362190 2024-09-19T09:28:18.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.866+0000 7f4aced7f740 1 -- 172.21.15.16:0/521307404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280362190 msgr2=0x55a280354640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.866+0000 7f4aced7f740 1 --2- 172.21.15.16:0/521307404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280362190 0x55a280354640 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4abc098e40 tx=0x7f4abc0d3850 comp rx=0 tx=0).stop 2024-09-19T09:28:18.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.866+0000 7f4aced7f740 1 -- 172.21.15.16:0/521307404 shutdown_connections 2024-09-19T09:28:18.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.866+0000 7f4aced7f740 1 --2- 172.21.15.16:0/521307404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a280362190 0x55a280354640 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.866+0000 7f4aced7f740 1 --2- 172.21.15.16:0/521307404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28036ca50 0x55a280366bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.866+0000 7f4aced7f740 1 --2- 172.21.15.16:0/521307404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280361870 0x55a280364370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.866+0000 7f4aced7f740 1 -- 172.21.15.16:0/521307404 >> 172.21.15.16:0/521307404 conn(0x55a28035de00 msgr2=0x55a280353f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.866+0000 7f4aced7f740 1 -- 172.21.15.16:0/521307404 shutdown_connections 2024-09-19T09:28:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.866+0000 7f4aced7f740 1 -- 172.21.15.16:0/521307404 wait complete. 2024-09-19T09:28:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.867+0000 7f4aced7f740 1 -- start start 2024-09-19T09:28:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.867+0000 7f4aced7f740 1 Processor -- start 2024-09-19T09:28:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.867+0000 7f4aced7f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280361870 0x55a2803818e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.867+0000 7f4aced7f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a28036ca50 0x55a280381e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.867+0000 7f4aced7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28037fd50 0x55a2803801d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.867+0000 7f4aced7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8071190 con 0x55a28037fd50 2024-09-19T09:28:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.867+0000 7f4aced7f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8070ac0 con 0x55a280361870 2024-09-19T09:28:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.867+0000 7f4aced7f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac806dce0 con 0x55a28036ca50 2024-09-19T09:28:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.867+0000 7f4aced7f740 10 client.0 _ll_register_callbacks cb 0x7f4ac0005a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.867+0000 7f4ac659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280361870 0x55a2803818e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.867+0000 7f4ac659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280361870 0x55a2803818e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:33264/0 (socket says 172.21.15.16:33264) 2024-09-19T09:28:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.867+0000 7f4ac659c640 1 -- 172.21.15.16:0/395243813 learned_addr learned my addr 172.21.15.16:0/395243813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.867+0000 7f4ac6d9d640 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28037fd50 0x55a2803801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.868+0000 7f4ac5d9b640 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a28036ca50 0x55a280381e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.868+0000 7f4ac659c640 1 -- 172.21.15.16:0/395243813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a28036ca50 msgr2=0x55a280381e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.868+0000 7f4ac659c640 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a28036ca50 0x55a280381e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.868+0000 7f4ac659c640 1 -- 172.21.15.16:0/395243813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28037fd50 msgr2=0x55a2803801d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.868+0000 7f4ac659c640 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28037fd50 0x55a2803801d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.868+0000 7f4ac659c640 1 -- 172.21.15.16:0/395243813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2803809d0 con 0x55a280361870 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.868+0000 7f4ac5d9b640 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a28036ca50 0x55a280381e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.868+0000 7f4ac659c640 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280361870 0x55a2803818e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ab802aa00 tx=0x7f4ab8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f185c80946238d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.868+0000 7f4aadffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.868+0000 7f4aadffb640 1 -- 172.21.15.16:0/395243813 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8035050 con 0x55a280361870 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.868+0000 7f4aadffb640 1 -- 172.21.15.16:0/395243813 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4ab80046c0 con 0x55a280361870 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.868+0000 7f4aced7f740 1 -- 172.21.15.16:0/395243813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4abc09cb70 con 0x55a280361870 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.869+0000 7f4aadffb640 1 -- 172.21.15.16:0/395243813 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ab80048c0 con 0x55a280361870 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.869+0000 7f4aced7f740 10 client.25271 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.869+0000 7f4aced7f740 1 -- 172.21.15.16:0/395243813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a2803e7f40 con 0x55a280361870 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.869+0000 7f4aadffb640 1 -- 172.21.15.16:0/395243813 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab80325c0 con 0x55a280361870 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.869+0000 7f4aadffb640 1 -- 172.21.15.16:0/395243813 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4ab8057480 con 0x55a280361870 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.869+0000 7f4aced7f740 10 client.25271 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:18.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.869+0000 7f4aced7f740 10 client.25271 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:28:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.869+0000 7f4aced7f740 20 client.25271 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.869+0000 7f4aced7f740 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a2803e8610 0x55a2803eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.869+0000 7f4aced7f740 4 client.25271 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a08a4233", "release": true, "await": true}] 2024-09-19T09:28:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.869+0000 7f4aced7f740 1 -- 172.21.15.16:0/395243813 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a08a4233", "release": true, "await": true}) -- 0x7f4abc09cb70 con 0x55a2803e8610 2024-09-19T09:28:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.869+0000 7f4ac5d9b640 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a2803e8610 0x55a2803eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.872+0000 7f4ac5d9b640 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a2803e8610 0x55a2803eaa00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cef6733df061f4 server_cookie=4833dd206926dd8e in_seq=0 out_seq=0 2024-09-19T09:28:18.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.872+0000 7f4aadffb640 10 client.25271 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aadffb640 1 -- 172.21.15.16:0/395243813 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f4abc09cb70 con 0x55a2803e8610 2024-09-19T09:28:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aadffb640 10 client.25271 handle_command_reply: tid=0 2024-09-19T09:28:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aced7f740 1 client.25271 shutdown 2024-09-19T09:28:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aced7f740 1 -- 172.21.15.16:0/395243813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280361870 msgr2=0x55a2803818e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aced7f740 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280361870 0x55a2803818e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ab802aa00 tx=0x7f4ab8009260 comp rx=0 tx=0).stop 2024-09-19T09:28:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aced7f740 1 -- 172.21.15.16:0/395243813 shutdown_connections 2024-09-19T09:28:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aced7f740 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a2803e8610 0x55a2803eaa00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aadffb640 0 client.25271 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aced7f740 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a28036ca50 0x55a280381e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aced7f740 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a28037fd50 0x55a2803801d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aced7f740 1 --2- 172.21.15.16:0/395243813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a280361870 0x55a2803818e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aced7f740 1 -- 172.21.15.16:0/395243813 >> 172.21.15.16:0/395243813 conn(0x55a28035de00 msgr2=0x55a280354750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aced7f740 1 -- 172.21.15.16:0/395243813 shutdown_connections 2024-09-19T09:28:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aced7f740 1 -- 172.21.15.16:0/395243813 wait complete. 2024-09-19T09:28:18.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.874+0000 7f4aced7f740 20 client.25271 trim_cache size 0 max 16384 2024-09-19T09:28:18.877 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:28:18.877 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-19T09:28:18.877 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24500, 2024-09-19T09:28:18.877 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 4, 2024-09-19T09:28:18.877 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-19T09:28:18.877 INFO:teuthology.orchestra.run.smithi016.stdout: "a08a4233": { 2024-09-19T09:28:18.877 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 4, 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 131.2, 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-19T09:28:18.878 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:18.879 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:18.879 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:18.879 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:18.879 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:18.879 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:28:18.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.875+0000 7f4acc9a6640 1 -- 172.21.15.16:0/3400884160 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4ab00a3340 msgr2=0x7f4ab00a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.875+0000 7f4acc9a6640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4ab00a3340 0x7f4ab00a5800 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8006010 tx=0x7f4ab8000bc0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.875+0000 7f4acc9a6640 1 -- 172.21.15.16:0/3400884160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac8102790 msgr2=0x7f4ac8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.875+0000 7f4acc9a6640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac8102790 0x7f4ac8074a30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00ed60 tx=0x7f4abc00c6a0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.875+0000 7f4acc9a6640 1 -- 172.21.15.16:0/3400884160 shutdown_connections 2024-09-19T09:28:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.875+0000 7f4acc9a6640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4ab00a3340 0x7f4ab00a5800 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.875+0000 7f4acc9a6640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ac8106f10 0x7f4ac8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.875+0000 7f4acc9a6640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac8102790 0x7f4ac8074a30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.875+0000 7f4acc9a6640 1 --2- 172.21.15.16:0/3400884160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ac8101de0 0x7f4ac80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.875+0000 7f4acc9a6640 1 -- 172.21.15.16:0/3400884160 >> 172.21.15.16:0/3400884160 conn(0x7f4ac80fd8c0 msgr2=0x7f4ac80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.875+0000 7f4acc9a6640 1 -- 172.21.15.16:0/3400884160 shutdown_connections 2024-09-19T09:28:18.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:18.875+0000 7f4acc9a6640 1 -- 172.21.15.16:0/3400884160 wait complete. 2024-09-19T09:28:19.168 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:28:19.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:28:19.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:19 smithi142 ceph-mon[35032]: pgmap v359: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:28:19.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:19 smithi097 ceph-mon[34864]: pgmap v359: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:28:19.469 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:19 smithi016 ceph-mon[30979]: pgmap v359: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-19T09:28:19.608 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a08a4233', total seconds elapsed: 15.4 2024-09-19T09:28:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.219+0000 7effb5b1c640 1 -- 172.21.15.16:0/3210260173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb0103760 msgr2=0x7effb0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.219+0000 7effb5b1c640 1 --2- 172.21.15.16:0/3210260173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb0103760 0x7effb0107ba0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7eff98009930 tx=0x7eff9802a420 comp rx=0 tx=0).stop 2024-09-19T09:28:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.220+0000 7effb5b1c640 1 -- 172.21.15.16:0/3210260173 shutdown_connections 2024-09-19T09:28:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.220+0000 7effb5b1c640 1 --2- 172.21.15.16:0/3210260173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effb01082e0 0x7effb010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.220+0000 7effb5b1c640 1 --2- 172.21.15.16:0/3210260173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb0103760 0x7effb0107ba0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.220+0000 7effb5b1c640 1 --2- 172.21.15.16:0/3210260173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effb0102db0 0x7effb0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.220+0000 7effb5b1c640 1 -- 172.21.15.16:0/3210260173 >> 172.21.15.16:0/3210260173 conn(0x7effb00fc7d0 msgr2=0x7effb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:20.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.220+0000 7effb5b1c640 1 -- 172.21.15.16:0/3210260173 shutdown_connections 2024-09-19T09:28:20.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.220+0000 7effb5b1c640 1 -- 172.21.15.16:0/3210260173 wait complete. 2024-09-19T09:28:20.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.221+0000 7effb5b1c640 1 Processor -- start 2024-09-19T09:28:20.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.221+0000 7effb5b1c640 1 -- start start 2024-09-19T09:28:20.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.221+0000 7effb5b1c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effb0102db0 0x7effb0100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.221+0000 7effb5b1c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effb0103760 0x7effb0100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.221+0000 7effb5b1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb01082e0 0x7effb0101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.222+0000 7effa7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effb0103760 0x7effb0100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.222+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effb0102db0 0x7effb0100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.222+0000 7effa7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effb0103760 0x7effb0100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:33292/0 (socket says 172.21.15.16:33292) 2024-09-19T09:28:20.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.222+0000 7effa7fff640 1 -- 172.21.15.16:0/2180701525 learned_addr learned my addr 172.21.15.16:0/2180701525 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:20.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.222+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effb0102db0 0x7effb0100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:44120/0 (socket says 172.21.15.16:44120) 2024-09-19T09:28:20.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.222+0000 7effaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb01082e0 0x7effb0101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.222+0000 7effb5b1c640 1 -- 172.21.15.16:0/2180701525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7effb0112050 con 0x7effb01082e0 2024-09-19T09:28:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.222+0000 7effb5b1c640 1 -- 172.21.15.16:0/2180701525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7effb0111ed0 con 0x7effb0103760 2024-09-19T09:28:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.222+0000 7effb5b1c640 1 -- 172.21.15.16:0/2180701525 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7effb01121d0 con 0x7effb0102db0 2024-09-19T09:28:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.222+0000 7effa7fff640 1 -- 172.21.15.16:0/2180701525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effb0102db0 msgr2=0x7effb0100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.222+0000 7effa7fff640 1 --2- 172.21.15.16:0/2180701525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effb0102db0 0x7effb0100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.222+0000 7effa7fff640 1 -- 172.21.15.16:0/2180701525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb01082e0 msgr2=0x7effb0101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.223+0000 7effa7fff640 1 --2- 172.21.15.16:0/2180701525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb01082e0 0x7effb0101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.223+0000 7effa7fff640 1 -- 172.21.15.16:0/2180701525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb01a69f0 con 0x7effb0103760 2024-09-19T09:28:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.223+0000 7effaffff640 1 --2- 172.21.15.16:0/2180701525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb01082e0 0x7effb0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.223+0000 7effa7fff640 1 --2- 172.21.15.16:0/2180701525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effb0103760 0x7effb0100f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7eff98005940 tx=0x7eff98003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.223+0000 7effad7fa640 1 -- 172.21.15.16:0/2180701525 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9803f070 con 0x7effb0103760 2024-09-19T09:28:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.223+0000 7effb5b1c640 1 -- 172.21.15.16:0/2180701525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb01a6c80 con 0x7effb0103760 2024-09-19T09:28:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.223+0000 7effad7fa640 1 -- 172.21.15.16:0/2180701525 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff98031c30 con 0x7effb0103760 2024-09-19T09:28:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.223+0000 7effad7fa640 1 -- 172.21.15.16:0/2180701525 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98035030 con 0x7effb0103760 2024-09-19T09:28:20.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.223+0000 7effb5b1c640 1 -- 172.21.15.16:0/2180701525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb01a71b0 con 0x7effb0103760 2024-09-19T09:28:20.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.227+0000 7effad7fa640 1 -- 172.21.15.16:0/2180701525 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7eff98042070 con 0x7effb0103760 2024-09-19T09:28:20.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.227+0000 7effb7ef5740 1 Processor -- start 2024-09-19T09:28:20.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.227+0000 7effb7ef5740 1 -- start start 2024-09-19T09:28:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.227+0000 7effb7ef5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f1370 0x555ff89f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.227+0000 7effb7ef5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 0x555ff89fc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.227+0000 7effb7ef5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 0x555ff89fe940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.227+0000 7effb7ef5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555ff894e280 con 0x555ff89f1c90 2024-09-19T09:28:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.227+0000 7effb7ef5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555ff88f5b10 con 0x555ff89fc550 2024-09-19T09:28:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.227+0000 7effb7ef5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555ff894a240 con 0x555ff89f1370 2024-09-19T09:28:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.228+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 0x555ff89fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.228+0000 7effa7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 0x555ff89fe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.228+0000 7effaffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f1370 0x555ff89f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.228+0000 7effaffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f1370 0x555ff89f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:44128/0 (socket says 172.21.15.16:44128) 2024-09-19T09:28:20.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.228+0000 7effaffff640 1 -- 172.21.15.16:0/782800684 learned_addr learned my addr 172.21.15.16:0/782800684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:20.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.228+0000 7effad7fa640 1 --2- 172.21.15.16:0/2180701525 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7eff9c0a3240 0x7eff9c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.228+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 0x555ff89fc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55368/0 (socket says 172.21.15.16:55368) 2024-09-19T09:28:20.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.229+0000 7effaffff640 1 --2- 172.21.15.16:0/2180701525 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7eff9c0a3240 0x7eff9c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.229+0000 7effad7fa640 1 -- 172.21.15.16:0/2180701525 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7eff980f0000 con 0x7effb0103760 2024-09-19T09:28:20.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.229+0000 7effaffff640 1 -- 172.21.15.16:0/782800684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 msgr2=0x555ff89fe940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:20.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.229+0000 7effaffff640 1 --2- 172.21.15.16:0/782800684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 0x555ff89fe940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.229+0000 7effaffff640 1 -- 172.21.15.16:0/782800684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 msgr2=0x555ff89fc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:20.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.229+0000 7effaffff640 1 --2- 172.21.15.16:0/782800684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 0x555ff89fc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.229+0000 7effaffff640 1 -- 172.21.15.16:0/782800684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ff89021f0 con 0x555ff89f1370 2024-09-19T09:28:20.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.229+0000 7effaf7fe640 1 --2- 172.21.15.16:0/782800684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 0x555ff89fc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:20.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.229+0000 7effaffff640 1 --2- 172.21.15.16:0/782800684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f1370 0x555ff89f1750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7effa8013a00 tx=0x7effa8013ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c633c27fcb3fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.230+0000 7effa5ffb640 1 -- 172.21.15.16:0/782800684 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa801aea0 con 0x555ff89f1370 2024-09-19T09:28:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.230+0000 7effaffff640 1 --2- 172.21.15.16:0/2180701525 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7eff9c0a3240 0x7eff9c0a5700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7effa801f000 tx=0x7effa801a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.230+0000 7effa5ffb640 1 -- 172.21.15.16:0/782800684 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa8020070 con 0x555ff89f1370 2024-09-19T09:28:20.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.230+0000 7effa5ffb640 1 -- 172.21.15.16:0/782800684 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8024420 con 0x555ff89f1370 2024-09-19T09:28:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.230+0000 7effb7ef5740 1 -- 172.21.15.16:0/782800684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f1370 msgr2=0x555ff89f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.230+0000 7effb7ef5740 1 --2- 172.21.15.16:0/782800684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f1370 0x555ff89f1750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7effa8013a00 tx=0x7effa8013ec0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.231+0000 7effb7ef5740 1 -- 172.21.15.16:0/782800684 shutdown_connections 2024-09-19T09:28:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.231+0000 7effb7ef5740 1 --2- 172.21.15.16:0/782800684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 0x555ff89fe940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.231+0000 7effb7ef5740 1 --2- 172.21.15.16:0/782800684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 0x555ff89fc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.231+0000 7effb7ef5740 1 --2- 172.21.15.16:0/782800684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f1370 0x555ff89f1750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.231+0000 7effb7ef5740 1 -- 172.21.15.16:0/782800684 >> 172.21.15.16:0/782800684 conn(0x555ff89ed900 msgr2=0x555ff89edce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.231+0000 7effb7ef5740 1 -- 172.21.15.16:0/782800684 shutdown_connections 2024-09-19T09:28:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.231+0000 7effb7ef5740 1 -- 172.21.15.16:0/782800684 wait complete. 2024-09-19T09:28:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.231+0000 7effb7ef5740 1 -- start start 2024-09-19T09:28:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.231+0000 7effb7ef5740 1 Processor -- start 2024-09-19T09:28:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effb7ef5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1370 0x555ff89f9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effb7ef5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f1c90 0x555ff89f9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effb7ef5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 0x555ff89f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effb7ef5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555ff89ff120 con 0x555ff89f1370 2024-09-19T09:28:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effb7ef5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555ff8908f10 con 0x555ff89fc550 2024-09-19T09:28:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effb7ef5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555ff89ff2a0 con 0x555ff89f1c90 2024-09-19T09:28:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effb7ef5740 10 client.0 _ll_register_callbacks cb 0x555ff88d1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1370 0x555ff89f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1370 0x555ff89f9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55370/0 (socket says 172.21.15.16:55370) 2024-09-19T09:28:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effaf7fe640 1 -- 172.21.15.16:0/3252751531 learned_addr learned my addr 172.21.15.16:0/3252751531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effaffff640 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 0x555ff89f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effa7fff640 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f1c90 0x555ff89f9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effaf7fe640 1 -- 172.21.15.16:0/3252751531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f1c90 msgr2=0x555ff89f9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effaf7fe640 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f1c90 0x555ff89f9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effaf7fe640 1 -- 172.21.15.16:0/3252751531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 msgr2=0x555ff89f9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effaf7fe640 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 0x555ff89f9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effaf7fe640 1 -- 172.21.15.16:0/3252751531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ff86bf6a0 con 0x555ff89f1370 2024-09-19T09:28:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.232+0000 7effa7fff640 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f1c90 0x555ff89f9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.233+0000 7effaf7fe640 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1370 0x555ff89f9400 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7effa000baa0 tx=0x7effa000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98385c11d76a3ae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.233+0000 7eff8effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.233+0000 7eff8effd640 1 -- 172.21.15.16:0/3252751531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0011440 con 0x555ff89f1370 2024-09-19T09:28:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.233+0000 7eff8effd640 1 -- 172.21.15.16:0/3252751531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7effa00115e0 con 0x555ff89f1370 2024-09-19T09:28:20.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.233+0000 7effb7ef5740 1 -- 172.21.15.16:0/3252751531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa8020070 con 0x555ff89f1370 2024-09-19T09:28:20.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.234+0000 7eff8effd640 1 -- 172.21.15.16:0/3252751531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effa00117e0 con 0x555ff89f1370 2024-09-19T09:28:20.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.234+0000 7effb7ef5740 10 client.15660 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:20.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.234+0000 7effb7ef5740 1 -- 172.21.15.16:0/3252751531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ff8a0adb0 con 0x555ff89f1370 2024-09-19T09:28:20.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.234+0000 7eff8effd640 1 -- 172.21.15.16:0/3252751531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa8020070 con 0x555ff89f1370 2024-09-19T09:28:20.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.235+0000 7eff8effd640 1 -- 172.21.15.16:0/3252751531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7effa0026a40 con 0x555ff89f1370 2024-09-19T09:28:20.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.235+0000 7effb7ef5740 10 client.15660 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:20.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.235+0000 7effb7ef5740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:20.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.235+0000 7effb7ef5740 20 client.15660 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:20.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.235+0000 7effb7ef5740 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x555ff8a0c2c0 0x555ff8a0e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.235+0000 7effb7ef5740 4 client.15660 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:20.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.235+0000 7effb7ef5740 1 -- 172.21.15.16:0/3252751531 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ff895b7f0 con 0x555ff8a0c2c0 2024-09-19T09:28:20.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.235+0000 7effa7fff640 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x555ff8a0c2c0 0x555ff8a0e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.236+0000 7effa7fff640 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x555ff8a0c2c0 0x555ff8a0e6f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35e61a3c53f28de server_cookie=6c1a1be89bd8526 in_seq=0 out_seq=0 2024-09-19T09:28:20.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.236+0000 7eff8effd640 10 client.15660 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:20.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.237+0000 7eff8effd640 1 -- 172.21.15.16:0/3252751531 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ff895b7f0 con 0x555ff8a0c2c0 2024-09-19T09:28:20.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.237+0000 7eff8effd640 10 client.15660 handle_command_reply: tid=0 2024-09-19T09:28:20.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.237+0000 7effb7ef5740 1 client.15660 shutdown 2024-09-19T09:28:20.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.237+0000 7effb7ef5740 1 -- 172.21.15.16:0/3252751531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1370 msgr2=0x555ff89f9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:20.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.237+0000 7effb7ef5740 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1370 0x555ff89f9400 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7effa000baa0 tx=0x7effa000bf60 comp rx=0 tx=0).stop 2024-09-19T09:28:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.237+0000 7effb7ef5740 1 -- 172.21.15.16:0/3252751531 shutdown_connections 2024-09-19T09:28:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.237+0000 7effb7ef5740 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x555ff8a0c2c0 0x555ff8a0e6f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.237+0000 7effb7ef5740 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f1c90 0x555ff89f9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.237+0000 7effb7ef5740 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 0x555ff89f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.237+0000 7effb7ef5740 1 --2- 172.21.15.16:0/3252751531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1370 0x555ff89f9400 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.237+0000 7effb7ef5740 1 -- 172.21.15.16:0/3252751531 >> 172.21.15.16:0/3252751531 conn(0x555ff89ed900 msgr2=0x555ff89f3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:20.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.237+0000 7eff8effd640 0 client.15660 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:20.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.238+0000 7effb7ef5740 1 -- 172.21.15.16:0/3252751531 shutdown_connections 2024-09-19T09:28:20.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.238+0000 7effb7ef5740 1 -- 172.21.15.16:0/3252751531 wait complete. 2024-09-19T09:28:20.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.238+0000 7effb7ef5740 20 client.15660 trim_cache size 0 max 16384 2024-09-19T09:28:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.256+0000 7effb7ef5740 1 Processor -- start 2024-09-19T09:28:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.256+0000 7effb7ef5740 1 -- start start 2024-09-19T09:28:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.256+0000 7effb7ef5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 0x555ff89f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.256+0000 7effb7ef5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 0x555ff8a07c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.256+0000 7effb7ef5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f6680 0x555ff89f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.256+0000 7effb7ef5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7effb01ab330 con 0x555ff89f1c90 2024-09-19T09:28:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.256+0000 7effb7ef5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7effb01ace70 con 0x555ff89fc550 2024-09-19T09:28:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.256+0000 7effb7ef5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7effb01aba00 con 0x555ff89f6680 2024-09-19T09:28:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.256+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 0x555ff89f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.256+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 0x555ff89f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55380/0 (socket says 172.21.15.16:55380) 2024-09-19T09:28:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.256+0000 7effaf7fe640 1 -- 172.21.15.16:0/1099420268 learned_addr learned my addr 172.21.15.16:0/1099420268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.256+0000 7effaffff640 1 --2- 172.21.15.16:0/1099420268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f6680 0x555ff89f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.256+0000 7effa7fff640 1 --2- 172.21.15.16:0/1099420268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 0x555ff8a07c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.257+0000 7effaf7fe640 1 -- 172.21.15.16:0/1099420268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f6680 msgr2=0x555ff89f6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.257+0000 7effaf7fe640 1 --2- 172.21.15.16:0/1099420268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f6680 0x555ff89f6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.257+0000 7effaf7fe640 1 -- 172.21.15.16:0/1099420268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 msgr2=0x555ff8a07c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.257+0000 7effaf7fe640 1 --2- 172.21.15.16:0/1099420268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 0x555ff8a07c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.257+0000 7effaf7fe640 1 -- 172.21.15.16:0/1099420268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ff89ed730 con 0x555ff89f1c90 2024-09-19T09:28:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.257+0000 7effaf7fe640 1 --2- 172.21.15.16:0/1099420268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 0x555ff89f3e20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7effa0026e80 tx=0x7effa000b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9dd6ae607c4bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.257+0000 7effa57fa640 1 -- 172.21.15.16:0/1099420268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0011440 con 0x555ff89f1c90 2024-09-19T09:28:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.257+0000 7effa57fa640 1 -- 172.21.15.16:0/1099420268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa00115e0 con 0x555ff89f1c90 2024-09-19T09:28:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.257+0000 7effa57fa640 1 -- 172.21.15.16:0/1099420268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0005040 con 0x555ff89f1c90 2024-09-19T09:28:20.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.258+0000 7effb7ef5740 1 -- 172.21.15.16:0/1099420268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 msgr2=0x555ff89f3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:20.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.258+0000 7effb7ef5740 1 --2- 172.21.15.16:0/1099420268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 0x555ff89f3e20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7effa0026e80 tx=0x7effa000b400 comp rx=0 tx=0).stop 2024-09-19T09:28:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.258+0000 7effb7ef5740 1 -- 172.21.15.16:0/1099420268 shutdown_connections 2024-09-19T09:28:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.258+0000 7effb7ef5740 1 --2- 172.21.15.16:0/1099420268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f6680 0x555ff89f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.258+0000 7effb7ef5740 1 --2- 172.21.15.16:0/1099420268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89fc550 0x555ff8a07c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.258+0000 7effb7ef5740 1 --2- 172.21.15.16:0/1099420268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89f1c90 0x555ff89f3e20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.258+0000 7effb7ef5740 1 -- 172.21.15.16:0/1099420268 >> 172.21.15.16:0/1099420268 conn(0x555ff89ed900 msgr2=0x555ff8a07550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.258+0000 7effb7ef5740 1 -- 172.21.15.16:0/1099420268 shutdown_connections 2024-09-19T09:28:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.258+0000 7effb7ef5740 1 -- 172.21.15.16:0/1099420268 wait complete. 2024-09-19T09:28:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.259+0000 7effb7ef5740 1 -- start start 2024-09-19T09:28:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.259+0000 7effb7ef5740 1 Processor -- start 2024-09-19T09:28:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.259+0000 7effb7ef5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89f1c90 0x555ff8a11fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.259+0000 7effb7ef5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f6680 0x555ff8a124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.259+0000 7effb7ef5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89fc550 0x555ff8a75e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.259+0000 7effb7ef5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7effb01ac7a0 con 0x555ff89fc550 2024-09-19T09:28:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.259+0000 7effb7ef5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7effb01ac0d0 con 0x555ff89f1c90 2024-09-19T09:28:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.259+0000 7effb7ef5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7effb01a92f0 con 0x555ff89f6680 2024-09-19T09:28:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.259+0000 7effb7ef5740 10 client.0 _ll_register_callbacks cb 0x7effa00040c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.259+0000 7effaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89fc550 0x555ff8a75e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.259+0000 7effaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89fc550 0x555ff8a75e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55388/0 (socket says 172.21.15.16:55388) 2024-09-19T09:28:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.259+0000 7effaffff640 1 -- 172.21.15.16:0/711384893 learned_addr learned my addr 172.21.15.16:0/711384893 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.259+0000 7effaf7fe640 1 --2- 172.21.15.16:0/711384893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89f1c90 0x555ff8a11fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.260+0000 7effa7fff640 1 --2- 172.21.15.16:0/711384893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f6680 0x555ff8a124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.260+0000 7effaffff640 1 -- 172.21.15.16:0/711384893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f6680 msgr2=0x555ff8a124e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.260+0000 7effaffff640 1 --2- 172.21.15.16:0/711384893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f6680 0x555ff8a124e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.260+0000 7effaffff640 1 -- 172.21.15.16:0/711384893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89f1c90 msgr2=0x555ff8a11fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.260+0000 7effaffff640 1 --2- 172.21.15.16:0/711384893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89f1c90 0x555ff8a11fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.260+0000 7effaffff640 1 -- 172.21.15.16:0/711384893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ff8a12de0 con 0x555ff89fc550 2024-09-19T09:28:20.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.260+0000 7effaffff640 1 --2- 172.21.15.16:0/711384893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89fc550 0x555ff8a75e80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7effa80139d0 tx=0x7effa8010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80694bab002b74b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:20.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.260+0000 7eff8effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:20.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.260+0000 7eff8effd640 1 -- 172.21.15.16:0/711384893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8033060 con 0x555ff89fc550 2024-09-19T09:28:20.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.260+0000 7eff8effd640 1 -- 172.21.15.16:0/711384893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7effa8004b90 con 0x555ff89fc550 2024-09-19T09:28:20.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.260+0000 7effb7ef5740 1 -- 172.21.15.16:0/711384893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa00115e0 con 0x555ff89fc550 2024-09-19T09:28:20.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.261+0000 7eff8effd640 1 -- 172.21.15.16:0/711384893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effa801aea0 con 0x555ff89fc550 2024-09-19T09:28:20.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.261+0000 7effb7ef5740 10 client.15666 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.261+0000 7effb7ef5740 1 -- 172.21.15.16:0/711384893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ff8a77530 con 0x555ff89fc550 2024-09-19T09:28:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.261+0000 7eff8effd640 1 -- 172.21.15.16:0/711384893 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa8020040 con 0x555ff89fc550 2024-09-19T09:28:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.261+0000 7eff8effd640 1 -- 172.21.15.16:0/711384893 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7effa8038070 con 0x555ff89fc550 2024-09-19T09:28:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.261+0000 7effb7ef5740 10 client.15666 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.261+0000 7effb7ef5740 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.261+0000 7effb7ef5740 20 client.15666 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.261+0000 7effb7ef5740 1 --2- 172.21.15.16:0/711384893 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x555ff8a77ae0 0x555ff8a79f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.261+0000 7effb7ef5740 4 client.15666 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:28:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.261+0000 7effb7ef5740 1 -- 172.21.15.16:0/711384893 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7effa00115e0 con 0x555ff8a77ae0 2024-09-19T09:28:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.261+0000 7effaf7fe640 1 --2- 172.21.15.16:0/711384893 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x555ff8a77ae0 0x555ff8a79f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.262+0000 7eff8effd640 10 client.15666 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.262+0000 7effaf7fe640 1 --2- 172.21.15.16:0/711384893 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x555ff8a77ae0 0x555ff8a79f00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dbc21df1db8f253 server_cookie=97725a264b322985 in_seq=0 out_seq=0 2024-09-19T09:28:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.263+0000 7eff8effd640 1 -- 172.21.15.16:0/711384893 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7effa00115e0 con 0x555ff8a77ae0 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.263+0000 7eff8effd640 10 client.15666 handle_command_reply: tid=0 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.263+0000 7effb7ef5740 1 client.15666 shutdown 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.263+0000 7effb7ef5740 1 -- 172.21.15.16:0/711384893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89fc550 msgr2=0x555ff8a75e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.263+0000 7effb7ef5740 1 --2- 172.21.15.16:0/711384893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89fc550 0x555ff8a75e80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7effa80139d0 tx=0x7effa8010660 comp rx=0 tx=0).stop 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.263+0000 7effb7ef5740 1 -- 172.21.15.16:0/711384893 shutdown_connections 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.263+0000 7effb7ef5740 1 --2- 172.21.15.16:0/711384893 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x555ff8a77ae0 0x555ff8a79f00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.263+0000 7eff8effd640 0 client.15666 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.263+0000 7effb7ef5740 1 --2- 172.21.15.16:0/711384893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ff89f6680 0x555ff8a124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.263+0000 7effb7ef5740 1 --2- 172.21.15.16:0/711384893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ff89fc550 0x555ff8a75e80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.263+0000 7effb7ef5740 1 --2- 172.21.15.16:0/711384893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff89f1c90 0x555ff8a11fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.263+0000 7effb7ef5740 1 -- 172.21.15.16:0/711384893 >> 172.21.15.16:0/711384893 conn(0x555ff89ed900 msgr2=0x555ff89e3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.264+0000 7effb7ef5740 1 -- 172.21.15.16:0/711384893 shutdown_connections 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.264+0000 7effb7ef5740 1 -- 172.21.15.16:0/711384893 wait complete. 2024-09-19T09:28:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.264+0000 7effb7ef5740 20 client.15666 trim_cache size 0 max 16384 2024-09-19T09:28:20.268 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:28:20.268 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-19T09:28:20.268 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "ebb7b6ff-1fbe-4f15-ac5a-0ae5d02252a4", 2024-09-19T09:28:20.268 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-19T09:28:20.268 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:28:20.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.265+0000 7effb5b1c640 1 -- 172.21.15.16:0/2180701525 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7eff9c0a3240 msgr2=0x7eff9c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:20.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.265+0000 7effb5b1c640 1 --2- 172.21.15.16:0/2180701525 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7eff9c0a3240 0x7eff9c0a5700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7effa801f000 tx=0x7effa801a000 comp rx=0 tx=0).stop 2024-09-19T09:28:20.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.265+0000 7effb5b1c640 1 -- 172.21.15.16:0/2180701525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effb0103760 msgr2=0x7effb0100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:20.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.265+0000 7effb5b1c640 1 --2- 172.21.15.16:0/2180701525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effb0103760 0x7effb0100f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7eff98005940 tx=0x7eff98003c10 comp rx=0 tx=0).stop 2024-09-19T09:28:20.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.265+0000 7effb5b1c640 1 -- 172.21.15.16:0/2180701525 shutdown_connections 2024-09-19T09:28:20.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.265+0000 7effb5b1c640 1 --2- 172.21.15.16:0/2180701525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb01082e0 0x7effb0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.265+0000 7effb5b1c640 1 --2- 172.21.15.16:0/2180701525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effb0103760 0x7effb0100f50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.265+0000 7effb5b1c640 1 --2- 172.21.15.16:0/2180701525 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7eff9c0a3240 0x7eff9c0a5700 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.265+0000 7effb5b1c640 1 --2- 172.21.15.16:0/2180701525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effb0102db0 0x7effb0100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:20.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.265+0000 7effb5b1c640 1 -- 172.21.15.16:0/2180701525 >> 172.21.15.16:0/2180701525 conn(0x7effb00fc7d0 msgr2=0x7effb0105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:20.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.266+0000 7effb5b1c640 1 -- 172.21.15.16:0/2180701525 shutdown_connections 2024-09-19T09:28:20.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:20.266+0000 7effb5b1c640 1 -- 172.21.15.16:0/2180701525 wait complete. 2024-09-19T09:28:21.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:28:21.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:21 smithi016 ceph-mon[30979]: pgmap v360: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T09:28:21.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:21 smithi016 ceph-mon[30979]: scrub queued for path: / 2024-09-19T09:28:21.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:21 smithi016 ceph-mon[30979]: scrub summary: idle+waiting paths [/] 2024-09-19T09:28:21.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:21 smithi016 ceph-mon[30979]: scrub summary: active paths [/] 2024-09-19T09:28:21.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:21 smithi016 ceph-mon[30979]: scrub summary: idle+waiting paths [/] 2024-09-19T09:28:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:21.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:21 smithi142 ceph-mon[35032]: pgmap v360: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T09:28:21.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:21 smithi142 ceph-mon[35032]: scrub queued for path: / 2024-09-19T09:28:21.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:21 smithi142 ceph-mon[35032]: scrub summary: idle+waiting paths [/] 2024-09-19T09:28:21.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:21 smithi142 ceph-mon[35032]: scrub summary: active paths [/] 2024-09-19T09:28:21.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:21 smithi142 ceph-mon[35032]: scrub summary: idle+waiting paths [/] 2024-09-19T09:28:21.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:21 smithi097 ceph-mon[34864]: pgmap v360: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-19T09:28:21.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:21 smithi097 ceph-mon[34864]: scrub queued for path: / 2024-09-19T09:28:21.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:21 smithi097 ceph-mon[34864]: scrub summary: idle+waiting paths [/] 2024-09-19T09:28:21.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:21 smithi097 ceph-mon[34864]: scrub summary: active paths [/] 2024-09-19T09:28:21.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:21 smithi097 ceph-mon[34864]: scrub summary: idle+waiting paths [/] 2024-09-19T09:28:21.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.869+0000 7fa8a4f57640 1 -- 172.21.15.16:0/259406765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a0102d30 msgr2=0x7fa8a0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.869+0000 7fa8a4f57640 1 --2- 172.21.15.16:0/259406765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a0102d30 0x7fa8a0103110 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa890009930 tx=0x7fa89002a420 comp rx=0 tx=0).stop 2024-09-19T09:28:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.869+0000 7fa8a4f57640 1 -- 172.21.15.16:0/259406765 shutdown_connections 2024-09-19T09:28:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.869+0000 7fa8a4f57640 1 --2- 172.21.15.16:0/259406765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8a01036e0 0x7fa8a0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.869+0000 7fa8a4f57640 1 --2- 172.21.15.16:0/259406765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8a01082a0 0x7fa8a010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.869+0000 7fa8a4f57640 1 --2- 172.21.15.16:0/259406765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a0102d30 0x7fa8a0103110 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.869+0000 7fa8a4f57640 1 -- 172.21.15.16:0/259406765 >> 172.21.15.16:0/259406765 conn(0x7fa8a00fc7d0 msgr2=0x7fa8a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.869+0000 7fa8a4f57640 1 -- 172.21.15.16:0/259406765 shutdown_connections 2024-09-19T09:28:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.869+0000 7fa8a4f57640 1 -- 172.21.15.16:0/259406765 wait complete. 2024-09-19T09:28:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.870+0000 7fa8a4f57640 1 Processor -- start 2024-09-19T09:28:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.870+0000 7fa8a4f57640 1 -- start start 2024-09-19T09:28:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.871+0000 7fa8a4f57640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8a0102d30 0x7fa8a0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.871+0000 7fa8a4f57640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a01036e0 0x7fa8a0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.871+0000 7fa8a4f57640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8a01082a0 0x7fa8a0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.871+0000 7fa8a4f57640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a0112100 con 0x7fa8a01036e0 2024-09-19T09:28:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.871+0000 7fa89e59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8a0102d30 0x7fa8a0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.871+0000 7fa89e59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8a0102d30 0x7fa8a0100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:44172/0 (socket says 172.21.15.16:44172) 2024-09-19T09:28:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.871+0000 7fa89e59c640 1 -- 172.21.15.16:0/3155447314 learned_addr learned my addr 172.21.15.16:0/3155447314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.871+0000 7fa89dd9b640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a01036e0 0x7fa8a0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.871+0000 7fa8a4f57640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a0111f80 con 0x7fa8a01082a0 2024-09-19T09:28:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.871+0000 7fa89ed9d640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8a01082a0 0x7fa8a0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.871+0000 7fa8a4f57640 1 -- 172.21.15.16:0/3155447314 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a0112280 con 0x7fa8a0102d30 2024-09-19T09:28:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.872+0000 7fa89e59c640 1 -- 172.21.15.16:0/3155447314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8a01082a0 msgr2=0x7fa8a0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.872+0000 7fa89e59c640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8a01082a0 0x7fa8a0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.872+0000 7fa89e59c640 1 -- 172.21.15.16:0/3155447314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a01036e0 msgr2=0x7fa8a0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.872+0000 7fa89e59c640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a01036e0 0x7fa8a0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.872+0000 7fa89e59c640 1 -- 172.21.15.16:0/3155447314 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a01aaf70 con 0x7fa8a0102d30 2024-09-19T09:28:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.872+0000 7fa89ed9d640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8a01082a0 0x7fa8a0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.872+0000 7fa89dd9b640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a01036e0 0x7fa8a0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.872+0000 7fa89e59c640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8a0102d30 0x7fa8a0100990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa89002a940 tx=0x7fa890003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.872+0000 7fa88f7fe640 1 -- 172.21.15.16:0/3155447314 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89003f070 con 0x7fa8a0102d30 2024-09-19T09:28:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.873+0000 7fa8a4f57640 1 -- 172.21.15.16:0/3155447314 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a01ab140 con 0x7fa8a0102d30 2024-09-19T09:28:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.873+0000 7fa88f7fe640 1 -- 172.21.15.16:0/3155447314 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa890031c90 con 0x7fa8a0102d30 2024-09-19T09:28:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.873+0000 7fa88f7fe640 1 -- 172.21.15.16:0/3155447314 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890035030 con 0x7fa8a0102d30 2024-09-19T09:28:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.873+0000 7fa8a4f57640 1 -- 172.21.15.16:0/3155447314 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a01ab6d0 con 0x7fa8a0102d30 2024-09-19T09:28:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.876+0000 7fa88f7fe640 1 -- 172.21.15.16:0/3155447314 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa890042070 con 0x7fa8a0102d30 2024-09-19T09:28:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.876+0000 7fa8a7330740 1 Processor -- start 2024-09-19T09:28:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.877+0000 7fa8a7330740 1 -- start start 2024-09-19T09:28:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.877+0000 7fa88f7fe640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa8880a32c0 0x7fa8880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa88f7fe640 1 -- 172.21.15.16:0/3155447314 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa8900efc00 con 0x7fa8a0102d30 2024-09-19T09:28:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa89ed9d640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa8880a32c0 0x7fa8880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa89ed9d640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa8880a32c0 0x7fa8880a5780 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa898009fd0 tx=0x7fa898009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa8a7330740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62b4a0 0x56416e62b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa8a7330740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62bdc0 0x56416e636140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa8a7330740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 0x56416e638a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa8a7330740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56416e5912e0 con 0x56416e62bdc0 2024-09-19T09:28:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa8a7330740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56416e538ce0 con 0x56416e62b4a0 2024-09-19T09:28:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa8a7330740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56416e58d2a0 con 0x56416e636680 2024-09-19T09:28:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa89dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62bdc0 0x56416e636140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa89e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62b4a0 0x56416e62b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa89e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62b4a0 0x56416e62b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:33352/0 (socket says 172.21.15.16:33352) 2024-09-19T09:28:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa89e59c640 1 -- 172.21.15.16:0/1970580829 learned_addr learned my addr 172.21.15.16:0/1970580829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.878+0000 7fa89dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62bdc0 0x56416e636140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55410/0 (socket says 172.21.15.16:55410) 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.879+0000 7fa89ed9d640 1 --2- 172.21.15.16:0/1970580829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 0x56416e638a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.879+0000 7fa89e59c640 1 -- 172.21.15.16:0/1970580829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 msgr2=0x56416e638a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.879+0000 7fa89e59c640 1 --2- 172.21.15.16:0/1970580829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 0x56416e638a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.879+0000 7fa89e59c640 1 -- 172.21.15.16:0/1970580829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62bdc0 msgr2=0x56416e636140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.879+0000 7fa89e59c640 1 --2- 172.21.15.16:0/1970580829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62bdc0 0x56416e636140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.879+0000 7fa89e59c640 1 -- 172.21.15.16:0/1970580829 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56416e51d440 con 0x56416e62b4a0 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.879+0000 7fa89dd9b640 1 --2- 172.21.15.16:0/1970580829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62bdc0 0x56416e636140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.879+0000 7fa89e59c640 1 --2- 172.21.15.16:0/1970580829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62b4a0 0x56416e62b880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa890004650 tx=0x7fa89002a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f5a04a0c067814 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.879+0000 7fa88cff9640 1 -- 172.21.15.16:0/1970580829 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8900dae60 con 0x56416e62b4a0 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.879+0000 7fa88cff9640 1 -- 172.21.15.16:0/1970580829 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8900bc2d0 con 0x56416e62b4a0 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.879+0000 7fa88cff9640 1 -- 172.21.15.16:0/1970580829 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890049040 con 0x56416e62b4a0 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.880+0000 7fa8a7330740 1 -- 172.21.15.16:0/1970580829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62b4a0 msgr2=0x56416e62b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.880+0000 7fa8a7330740 1 --2- 172.21.15.16:0/1970580829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62b4a0 0x56416e62b880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa890004650 tx=0x7fa89002a420 comp rx=0 tx=0).stop 2024-09-19T09:28:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.880+0000 7fa8a7330740 1 -- 172.21.15.16:0/1970580829 shutdown_connections 2024-09-19T09:28:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.880+0000 7fa8a7330740 1 --2- 172.21.15.16:0/1970580829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 0x56416e638a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.880+0000 7fa8a7330740 1 --2- 172.21.15.16:0/1970580829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62bdc0 0x56416e636140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.880+0000 7fa8a7330740 1 --2- 172.21.15.16:0/1970580829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62b4a0 0x56416e62b880 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.880+0000 7fa8a7330740 1 -- 172.21.15.16:0/1970580829 >> 172.21.15.16:0/1970580829 conn(0x56416e627a30 msgr2=0x56416e627e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.881+0000 7fa8a7330740 1 -- 172.21.15.16:0/1970580829 shutdown_connections 2024-09-19T09:28:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.881+0000 7fa8a7330740 1 -- 172.21.15.16:0/1970580829 wait complete. 2024-09-19T09:28:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.881+0000 7fa8a7330740 1 -- start start 2024-09-19T09:28:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.881+0000 7fa8a7330740 1 Processor -- start 2024-09-19T09:28:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.881+0000 7fa8a7330740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 0x56416e635f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.881+0000 7fa8a7330740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e62bdc0 0x56416e633220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa89e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 0x56416e635f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa89e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 0x56416e635f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55414/0 (socket says 172.21.15.16:55414) 2024-09-19T09:28:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa89e59c640 1 -- 172.21.15.16:0/649480948 learned_addr learned my addr 172.21.15.16:0/649480948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa8a7330740 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e636680 0x56416e633760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa8a7330740 1 -- 172.21.15.16:0/649480948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56416e639010 con 0x56416e62b4a0 2024-09-19T09:28:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa8a7330740 1 -- 172.21.15.16:0/649480948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56416e54c050 con 0x56416e636680 2024-09-19T09:28:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa8a7330740 1 -- 172.21.15.16:0/649480948 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56416e639190 con 0x56416e62bdc0 2024-09-19T09:28:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa89ed9d640 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e636680 0x56416e633760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa89e59c640 1 -- 172.21.15.16:0/649480948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e62bdc0 msgr2=0x56416e633220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa89dd9b640 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e62bdc0 0x56416e633220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa89e59c640 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e62bdc0 0x56416e633220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa89e59c640 1 -- 172.21.15.16:0/649480948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e636680 msgr2=0x56416e633760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa89e59c640 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e636680 0x56416e633760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa89e59c640 1 -- 172.21.15.16:0/649480948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56416e3023c0 con 0x56416e62b4a0 2024-09-19T09:28:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa8a7330740 10 client.0 _ll_register_callbacks cb 0x56416e58c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa89ed9d640 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e636680 0x56416e633760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa89dd9b640 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e62bdc0 0x56416e633220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa89e59c640 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 0x56416e635f70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa8900c4960 tx=0x7fa890005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c98ed3b25b3136 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa8a7330740 1 -- 172.21.15.16:0/649480948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8900bc2d0 con 0x56416e62b4a0 2024-09-19T09:28:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.882+0000 7fa885ffb640 1 -- 172.21.15.16:0/649480948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8900d4070 con 0x56416e62b4a0 2024-09-19T09:28:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.883+0000 7fa885ffb640 1 -- 172.21.15.16:0/649480948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa8900dc0d0 con 0x56416e62b4a0 2024-09-19T09:28:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.883+0000 7fa885ffb640 1 -- 172.21.15.16:0/649480948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa8900dae60 con 0x56416e62b4a0 2024-09-19T09:28:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.883+0000 7fa8a7330740 10 client.15678 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.883+0000 7fa8a7330740 1 -- 172.21.15.16:0/649480948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56416e655620 con 0x56416e62b4a0 2024-09-19T09:28:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.883+0000 7fa885ffb640 1 -- 172.21.15.16:0/649480948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa89004c060 con 0x56416e62b4a0 2024-09-19T09:28:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.883+0000 7fa8a7330740 10 client.15678 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.883+0000 7fa8a7330740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.883+0000 7fa8a7330740 20 client.15678 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.883+0000 7fa8a7330740 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56416e6567f0 0x56416e658c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.883+0000 7fa8a7330740 4 client.15678 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.883+0000 7fa8a7330740 1 -- 172.21.15.16:0/649480948 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56416e59e850 con 0x56416e6567f0 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.884+0000 7fa89dd9b640 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56416e6567f0 0x56416e658c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.884+0000 7fa89dd9b640 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56416e6567f0 0x56416e658c60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c2f3750cef96cb server_cookie=5af7bb37106aa6ce in_seq=0 out_seq=0 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.884+0000 7fa885ffb640 10 client.15678 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa885ffb640 1 -- 172.21.15.16:0/649480948 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56416e59e850 con 0x56416e6567f0 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa885ffb640 10 client.15678 handle_command_reply: tid=0 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa8a7330740 1 client.15678 shutdown 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa8a7330740 1 -- 172.21.15.16:0/649480948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 msgr2=0x56416e635f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa8a7330740 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 0x56416e635f70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa8900c4960 tx=0x7fa890005be0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa8a7330740 1 -- 172.21.15.16:0/649480948 shutdown_connections 2024-09-19T09:28:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa8a7330740 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56416e6567f0 0x56416e658c60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa8a7330740 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e62bdc0 0x56416e633220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa885ffb640 0 client.15678 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa8a7330740 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e636680 0x56416e633760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa8a7330740 1 --2- 172.21.15.16:0/649480948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 0x56416e635f70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa8a7330740 1 -- 172.21.15.16:0/649480948 >> 172.21.15.16:0/649480948 conn(0x56416e627a30 msgr2=0x56416e62d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa8a7330740 1 -- 172.21.15.16:0/649480948 shutdown_connections 2024-09-19T09:28:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa8a7330740 1 -- 172.21.15.16:0/649480948 wait complete. 2024-09-19T09:28:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.885+0000 7fa8a7330740 20 client.15678 trim_cache size 0 max 16384 2024-09-19T09:28:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.919+0000 7fa8a7330740 1 Processor -- start 2024-09-19T09:28:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.919+0000 7fa8a7330740 1 -- start start 2024-09-19T09:28:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa8a7330740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 0x56416e6335b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa8a7330740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62bdc0 0x56416e64f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa8a7330740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 0x56416e64f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa8a7330740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a01a4ad0 con 0x56416e62b4a0 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa89e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 0x56416e6335b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa8a7330740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a006d1d0 con 0x56416e62bdc0 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa8a7330740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a006bd60 con 0x56416e636680 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa89e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 0x56416e6335b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55430/0 (socket says 172.21.15.16:55430) 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa89e59c640 1 -- 172.21.15.16:0/3139331135 learned_addr learned my addr 172.21.15.16:0/3139331135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa89ed9d640 1 --2- 172.21.15.16:0/3139331135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 0x56416e64f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa89e59c640 1 -- 172.21.15.16:0/3139331135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 msgr2=0x56416e64f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa89e59c640 1 --2- 172.21.15.16:0/3139331135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 0x56416e64f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa89e59c640 1 -- 172.21.15.16:0/3139331135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62bdc0 msgr2=0x56416e64f3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa89e59c640 1 --2- 172.21.15.16:0/3139331135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62bdc0 0x56416e64f3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.920+0000 7fa89e59c640 1 -- 172.21.15.16:0/3139331135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56416e6362e0 con 0x56416e62b4a0 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.921+0000 7fa89e59c640 1 --2- 172.21.15.16:0/3139331135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 0x56416e6335b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa890004650 tx=0x7fa8900bf000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744946b71c5fd56c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.921+0000 7fa887fff640 1 -- 172.21.15.16:0/3139331135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8900d4040 con 0x56416e62b4a0 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.921+0000 7fa887fff640 1 -- 172.21.15.16:0/3139331135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8900bc2d0 con 0x56416e62b4a0 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.921+0000 7fa887fff640 1 -- 172.21.15.16:0/3139331135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890049040 con 0x56416e62b4a0 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.922+0000 7fa8a7330740 1 -- 172.21.15.16:0/3139331135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 msgr2=0x56416e6335b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.922+0000 7fa8a7330740 1 --2- 172.21.15.16:0/3139331135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 0x56416e6335b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa890004650 tx=0x7fa8900bf000 comp rx=0 tx=0).stop 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.922+0000 7fa8a7330740 1 -- 172.21.15.16:0/3139331135 shutdown_connections 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.922+0000 7fa8a7330740 1 --2- 172.21.15.16:0/3139331135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 0x56416e64f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.922+0000 7fa8a7330740 1 --2- 172.21.15.16:0/3139331135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62bdc0 0x56416e64f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.922+0000 7fa8a7330740 1 --2- 172.21.15.16:0/3139331135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e62b4a0 0x56416e6335b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.922+0000 7fa8a7330740 1 -- 172.21.15.16:0/3139331135 >> 172.21.15.16:0/3139331135 conn(0x56416e627a30 msgr2=0x56416e6352a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.922+0000 7fa8a7330740 1 -- 172.21.15.16:0/3139331135 shutdown_connections 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.922+0000 7fa8a7330740 1 -- 172.21.15.16:0/3139331135 wait complete. 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.923+0000 7fa8a7330740 1 -- start start 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.923+0000 7fa8a7330740 1 Processor -- start 2024-09-19T09:28:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.923+0000 7fa8a7330740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62bdc0 0x56416e64d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.923+0000 7fa8a7330740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 0x56416e64de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.923+0000 7fa8a7330740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e6ba4b0 0x56416e6ba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.923+0000 7fa8a7330740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a006cb00 con 0x56416e6ba4b0 2024-09-19T09:28:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.923+0000 7fa8a7330740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a006c430 con 0x56416e62bdc0 2024-09-19T09:28:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.923+0000 7fa8a7330740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a01a2a90 con 0x56416e636680 2024-09-19T09:28:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.923+0000 7fa8a7330740 10 client.0 _ll_register_callbacks cb 0x7fa8900c0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa89ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e6ba4b0 0x56416e6ba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa89e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62bdc0 0x56416e64d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa89ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e6ba4b0 0x56416e6ba930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55434/0 (socket says 172.21.15.16:55434) 2024-09-19T09:28:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa89ed9d640 1 -- 172.21.15.16:0/3160577069 learned_addr learned my addr 172.21.15.16:0/3160577069 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa89dd9b640 1 --2- 172.21.15.16:0/3160577069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 0x56416e64de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa89e59c640 1 -- 172.21.15.16:0/3160577069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 msgr2=0x56416e64de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa89e59c640 1 --2- 172.21.15.16:0/3160577069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 0x56416e64de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa89e59c640 1 -- 172.21.15.16:0/3160577069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e6ba4b0 msgr2=0x56416e6ba930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa89e59c640 1 --2- 172.21.15.16:0/3160577069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e6ba4b0 0x56416e6ba930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa89e59c640 1 -- 172.21.15.16:0/3160577069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56416e6baea0 con 0x56416e62bdc0 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa89e59c640 1 --2- 172.21.15.16:0/3160577069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62bdc0 0x56416e64d900 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8900be000 tx=0x7fa89002a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1811813dea327043 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa885ffb640 1 -- 172.21.15.16:0/3160577069 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890049050 con 0x56416e62bdc0 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.924+0000 7fa885ffb640 1 -- 172.21.15.16:0/3160577069 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa8900eddc0 con 0x56416e62bdc0 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.925+0000 7fa8a7330740 1 -- 172.21.15.16:0/3160577069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8900bc2d0 con 0x56416e62bdc0 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.925+0000 7fa885ffb640 1 -- 172.21.15.16:0/3160577069 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa8900d4070 con 0x56416e62bdc0 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.925+0000 7fa8a7330740 10 client.25301 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.925+0000 7fa8a7330740 1 -- 172.21.15.16:0/3160577069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56416e6baea0 con 0x56416e62bdc0 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.925+0000 7fa885ffb640 1 -- 172.21.15.16:0/3160577069 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8900bc2d0 con 0x56416e62bdc0 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.926+0000 7fa885ffb640 1 -- 172.21.15.16:0/3160577069 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa8900bb720 con 0x56416e62bdc0 2024-09-19T09:28:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.926+0000 7fa8a7330740 10 client.25301 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.926+0000 7fa8a7330740 10 client.25301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.926+0000 7fa8a7330740 20 client.25301 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.926+0000 7fa8a7330740 1 --2- 172.21.15.16:0/3160577069 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56416e6c2470 0x56416e6c4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.926+0000 7fa8a7330740 4 client.25301 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.926+0000 7fa8a7330740 1 -- 172.21.15.16:0/3160577069 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56416e598d60 con 0x56416e6c2470 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.926+0000 7fa89dd9b640 1 --2- 172.21.15.16:0/3160577069 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56416e6c2470 0x56416e6c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.927+0000 7fa885ffb640 10 client.25301 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.927+0000 7fa89dd9b640 1 --2- 172.21.15.16:0/3160577069 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56416e6c2470 0x56416e6c4860 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=172512622920115e server_cookie=884354bcd7d54b4d in_seq=0 out_seq=0 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.927+0000 7fa885ffb640 1 -- 172.21.15.16:0/3160577069 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56416e598d60 con 0x56416e6c2470 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.927+0000 7fa885ffb640 10 client.25301 handle_command_reply: tid=0 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.927+0000 7fa8a7330740 1 client.25301 shutdown 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.927+0000 7fa8a7330740 1 -- 172.21.15.16:0/3160577069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62bdc0 msgr2=0x56416e64d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.927+0000 7fa8a7330740 1 --2- 172.21.15.16:0/3160577069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62bdc0 0x56416e64d900 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8900be000 tx=0x7fa89002a420 comp rx=0 tx=0).stop 2024-09-19T09:28:22.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.928+0000 7fa8a7330740 1 -- 172.21.15.16:0/3160577069 shutdown_connections 2024-09-19T09:28:22.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.928+0000 7fa8a7330740 1 --2- 172.21.15.16:0/3160577069 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56416e6c2470 0x56416e6c4860 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.928+0000 7fa8a7330740 1 --2- 172.21.15.16:0/3160577069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56416e636680 0x56416e64de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.446 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:28:22.446 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:28:22.446 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-19T09:28:22.446 INFO:teuthology.orchestra.run.smithi016.stdout: "ebb7b6ff-1fbe-4f15-ac5a-0ae5d02252a4": { 2024-09-19T09:28:22.447 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-19T09:28:22.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "ebb7b6ff-1fbe-4f15-ac5a-0ae5d02252a4", 2024-09-19T09:28:22.447 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-19T09:28:22.447 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:22.447 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:22.447 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:28:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.928+0000 7fa8a7330740 1 --2- 172.21.15.16:0/3160577069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56416e6ba4b0 0x56416e6ba930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.928+0000 7fa8a7330740 1 --2- 172.21.15.16:0/3160577069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56416e62bdc0 0x56416e64d900 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.928+0000 7fa8a7330740 1 -- 172.21.15.16:0/3160577069 >> 172.21.15.16:0/3160577069 conn(0x56416e627a30 msgr2=0x56416e652260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.928+0000 7fa885ffb640 0 client.25301 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:22.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.928+0000 7fa8a7330740 1 -- 172.21.15.16:0/3160577069 shutdown_connections 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.928+0000 7fa8a7330740 1 -- 172.21.15.16:0/3160577069 wait complete. 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.928+0000 7fa8a7330740 20 client.25301 trim_cache size 0 max 16384 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.929+0000 7fa8a4f57640 1 -- 172.21.15.16:0/3155447314 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa8880a32c0 msgr2=0x7fa8880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.929+0000 7fa8a4f57640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa8880a32c0 0x7fa8880a5780 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa898009fd0 tx=0x7fa898009600 comp rx=0 tx=0).stop 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.929+0000 7fa8a4f57640 1 -- 172.21.15.16:0/3155447314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8a0102d30 msgr2=0x7fa8a0100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.929+0000 7fa8a4f57640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8a0102d30 0x7fa8a0100990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa89002a940 tx=0x7fa890003b90 comp rx=0 tx=0).stop 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.929+0000 7fa8a4f57640 1 -- 172.21.15.16:0/3155447314 shutdown_connections 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.929+0000 7fa8a4f57640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8a01082a0 0x7fa8a0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.929+0000 7fa8a4f57640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a01036e0 0x7fa8a0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.929+0000 7fa8a4f57640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa8880a32c0 0x7fa8880a5780 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.929+0000 7fa8a4f57640 1 --2- 172.21.15.16:0/3155447314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8a0102d30 0x7fa8a0100990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.929+0000 7fa8a4f57640 1 -- 172.21.15.16:0/3155447314 >> 172.21.15.16:0/3155447314 conn(0x7fa8a00fc7d0 msgr2=0x7fa8a0106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.929+0000 7fa8a4f57640 1 -- 172.21.15.16:0/3155447314 shutdown_connections 2024-09-19T09:28:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:21.929+0000 7fa8a4f57640 1 -- 172.21.15.16:0/3155447314 wait complete. 2024-09-19T09:28:22.801 INFO:tasks.cephfs.filesystem:scrub status for tag:ebb7b6ff-1fbe-4f15-ac5a-0ae5d02252a4 - {'path': '/', 'tag': 'ebb7b6ff-1fbe-4f15-ac5a-0ae5d02252a4', 'options': 'recursive,force'} 2024-09-19T09:28:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:23 smithi142 ceph-mon[35032]: pgmap v361: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-19T09:28:23.379 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:23 smithi016 ceph-mon[30979]: pgmap v361: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-19T09:28:23.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:23 smithi097 ceph-mon[34864]: pgmap v361: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-19T09:28:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:25 smithi142 ceph-mon[35032]: pgmap v362: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-09-19T09:28:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:25 smithi142 ceph-mon[35032]: scrub summary: idle 2024-09-19T09:28:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:25 smithi097 ceph-mon[34864]: pgmap v362: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-09-19T09:28:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:25 smithi097 ceph-mon[34864]: scrub summary: idle 2024-09-19T09:28:25.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:25 smithi016 ceph-mon[30979]: pgmap v362: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-09-19T09:28:25.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:25 smithi016 ceph-mon[30979]: scrub summary: idle 2024-09-19T09:28:27.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:27 smithi142 ceph-mon[35032]: pgmap v363: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T09:28:27.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:27 smithi097 ceph-mon[34864]: pgmap v363: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T09:28:27.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:27 smithi016 ceph-mon[30979]: pgmap v363: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T09:28:29.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:29 smithi142 ceph-mon[35032]: pgmap v364: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 14 op/s 2024-09-19T09:28:29.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:29 smithi097 ceph-mon[34864]: pgmap v364: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 14 op/s 2024-09-19T09:28:29.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:29 smithi016 ceph-mon[30979]: pgmap v364: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 14 op/s 2024-09-19T09:28:31.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:31 smithi142 ceph-mon[35032]: pgmap v365: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T09:28:31.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:31 smithi097 ceph-mon[34864]: pgmap v365: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T09:28:31.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:31 smithi016 ceph-mon[30979]: pgmap v365: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-19T09:28:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.935+0000 7f174412a640 1 -- 172.21.15.16:0/4129379237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c102640 msgr2=0x7f173c104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.935+0000 7f174412a640 1 --2- 172.21.15.16:0/4129379237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c102640 0x7f173c104a30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1738009900 tx=0x7f173802a410 comp rx=0 tx=0).stop 2024-09-19T09:28:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.936+0000 7f174412a640 1 -- 172.21.15.16:0/4129379237 shutdown_connections 2024-09-19T09:28:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.936+0000 7f174412a640 1 --2- 172.21.15.16:0/4129379237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f173c1078a0 0x7f173c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.936+0000 7f174412a640 1 --2- 172.21.15.16:0/4129379237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c104f70 0x7f173c107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.936+0000 7f174412a640 1 --2- 172.21.15.16:0/4129379237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c102640 0x7f173c104a30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.936+0000 7f174412a640 1 -- 172.21.15.16:0/4129379237 >> 172.21.15.16:0/4129379237 conn(0x7f173c0fc1e0 msgr2=0x7f173c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.936+0000 7f174412a640 1 -- 172.21.15.16:0/4129379237 shutdown_connections 2024-09-19T09:28:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.936+0000 7f174412a640 1 -- 172.21.15.16:0/4129379237 wait complete. 2024-09-19T09:28:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.936+0000 7f174412a640 1 Processor -- start 2024-09-19T09:28:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.937+0000 7f174412a640 1 -- start start 2024-09-19T09:28:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.937+0000 7f174412a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c102640 0x7f173c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.937+0000 7f174412a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c104f70 0x7f173c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.937+0000 7f174412a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f173c1078a0 0x7f173c19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.937+0000 7f174412a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f173c10d830 con 0x7f173c104f70 2024-09-19T09:28:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.937+0000 7f174412a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f173c10d6b0 con 0x7f173c102640 2024-09-19T09:28:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.937+0000 7f174412a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f173c10d9b0 con 0x7f173c1078a0 2024-09-19T09:28:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.937+0000 7f17416c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c104f70 0x7f173c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.937+0000 7f1741ec6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c102640 0x7f173c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.937+0000 7f17416c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c104f70 0x7f173c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58348/0 (socket says 172.21.15.16:58348) 2024-09-19T09:28:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.937+0000 7f17416c5640 1 -- 172.21.15.16:0/1180317624 learned_addr learned my addr 172.21.15.16:0/1180317624 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.937+0000 7f17426c7640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f173c1078a0 0x7f173c19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.938+0000 7f17416c5640 1 -- 172.21.15.16:0/1180317624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f173c1078a0 msgr2=0x7f173c19f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.938+0000 7f17416c5640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f173c1078a0 0x7f173c19f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.938+0000 7f17416c5640 1 -- 172.21.15.16:0/1180317624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c102640 msgr2=0x7f173c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.938+0000 7f17416c5640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c102640 0x7f173c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.938+0000 7f17416c5640 1 -- 172.21.15.16:0/1180317624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173c100430 con 0x7f173c104f70 2024-09-19T09:28:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.938+0000 7f17426c7640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f173c1078a0 0x7f173c19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.938+0000 7f1741ec6640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c102640 0x7f173c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.938+0000 7f17416c5640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c104f70 0x7f173c198370 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f172c00d9d0 tx=0x7f172c00de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.938+0000 7f1732ffd640 1 -- 172.21.15.16:0/1180317624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c017070 con 0x7f173c104f70 2024-09-19T09:28:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.938+0000 7f1732ffd640 1 -- 172.21.15.16:0/1180317624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c009b40 con 0x7f173c104f70 2024-09-19T09:28:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.938+0000 7f1732ffd640 1 -- 172.21.15.16:0/1180317624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c00f6b0 con 0x7f173c104f70 2024-09-19T09:28:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.938+0000 7f174412a640 1 -- 172.21.15.16:0/1180317624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173c100720 con 0x7f173c104f70 2024-09-19T09:28:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.938+0000 7f174412a640 1 -- 172.21.15.16:0/1180317624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f173c100c20 con 0x7f173c104f70 2024-09-19T09:28:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.942+0000 7f1746ec4740 1 Processor -- start 2024-09-19T09:28:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.942+0000 7f1746ec4740 1 -- start start 2024-09-19T09:28:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.942+0000 7f1732ffd640 1 -- 172.21.15.16:0/1180317624 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f172c01a070 con 0x7f173c104f70 2024-09-19T09:28:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.942+0000 7f1746ec4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 0x56440ce565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.942+0000 7f1746ec4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce56af0 0x56440ce5b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.942+0000 7f1746ec4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce5bb80 0x56440ce5e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.942+0000 7f1746ec4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56440cdb12f0 con 0x56440ce561b0 2024-09-19T09:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.942+0000 7f1746ec4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56440cd59270 con 0x56440ce5bb80 2024-09-19T09:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.942+0000 7f1746ec4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56440cdad2b0 con 0x56440ce56af0 2024-09-19T09:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.943+0000 7f17426c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 0x56440ce565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.943+0000 7f17426c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 0x56440ce565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58354/0 (socket says 172.21.15.16:58354) 2024-09-19T09:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.943+0000 7f17426c7640 1 -- 172.21.15.16:0/1795579560 learned_addr learned my addr 172.21.15.16:0/1795579560 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.943+0000 7f1741ec6640 1 --2- 172.21.15.16:0/1795579560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce56af0 0x56440ce5b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.943+0000 7f17416c5640 1 --2- 172.21.15.16:0/1795579560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce5bb80 0x56440ce5e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.943+0000 7f17426c7640 1 -- 172.21.15.16:0/1795579560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce56af0 msgr2=0x56440ce5b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.943+0000 7f17426c7640 1 --2- 172.21.15.16:0/1795579560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce56af0 0x56440ce5b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.943+0000 7f17426c7640 1 -- 172.21.15.16:0/1795579560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce5bb80 msgr2=0x56440ce5e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.943+0000 7f17426c7640 1 --2- 172.21.15.16:0/1795579560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce5bb80 0x56440ce5e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.943+0000 7f17426c7640 1 -- 172.21.15.16:0/1795579560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56440cd58340 con 0x56440ce561b0 2024-09-19T09:28:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.944+0000 7f17426c7640 1 --2- 172.21.15.16:0/1795579560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 0x56440ce565b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1734009f60 tx=0x7f1734009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3527d84e41356c04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.944+0000 7f1727fff640 1 -- 172.21.15.16:0/1795579560 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734016070 con 0x56440ce561b0 2024-09-19T09:28:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.944+0000 7f1732ffd640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f17280a3370 0x7f17280a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.944+0000 7f1727fff640 1 -- 172.21.15.16:0/1795579560 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f173401bb40 con 0x56440ce561b0 2024-09-19T09:28:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.944+0000 7f1727fff640 1 -- 172.21.15.16:0/1795579560 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734012c40 con 0x56440ce561b0 2024-09-19T09:28:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.944+0000 7f1732ffd640 1 -- 172.21.15.16:0/1180317624 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f172c0cf270 con 0x7f173c104f70 2024-09-19T09:28:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.944+0000 7f1741ec6640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f17280a3370 0x7f17280a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.945+0000 7f1746ec4740 1 -- 172.21.15.16:0/1795579560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 msgr2=0x56440ce565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.945+0000 7f1741ec6640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f17280a3370 0x7f17280a5830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f17380314e0 tx=0x7f1738005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.945+0000 7f1746ec4740 1 --2- 172.21.15.16:0/1795579560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 0x56440ce565b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1734009f60 tx=0x7f1734009f90 comp rx=0 tx=0).stop 2024-09-19T09:28:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.945+0000 7f1746ec4740 1 -- 172.21.15.16:0/1795579560 shutdown_connections 2024-09-19T09:28:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.945+0000 7f1746ec4740 1 --2- 172.21.15.16:0/1795579560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce56af0 0x56440ce5b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.945+0000 7f1746ec4740 1 --2- 172.21.15.16:0/1795579560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce5bb80 0x56440ce5e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.945+0000 7f1746ec4740 1 --2- 172.21.15.16:0/1795579560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 0x56440ce565b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.945+0000 7f1746ec4740 1 -- 172.21.15.16:0/1795579560 >> 172.21.15.16:0/1795579560 conn(0x56440ce51820 msgr2=0x56440ce51c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.945+0000 7f1746ec4740 1 -- 172.21.15.16:0/1795579560 shutdown_connections 2024-09-19T09:28:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.945+0000 7f1746ec4740 1 -- 172.21.15.16:0/1795579560 wait complete. 2024-09-19T09:28:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.946+0000 7f1746ec4740 1 -- start start 2024-09-19T09:28:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.946+0000 7f1746ec4740 1 Processor -- start 2024-09-19T09:28:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.946+0000 7f1746ec4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 0x56440ce60460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.946+0000 7f1746ec4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce56af0 0x56440ce609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.946+0000 7f1746ec4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce5bb80 0x56440ce679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.946+0000 7f1746ec4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56440ce5eb50 con 0x56440ce561b0 2024-09-19T09:28:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.946+0000 7f1746ec4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56440cd3a290 con 0x56440ce56af0 2024-09-19T09:28:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.946+0000 7f1746ec4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56440ce5ecd0 con 0x56440ce5bb80 2024-09-19T09:28:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.946+0000 7f1746ec4740 10 client.0 _ll_register_callbacks cb 0x56440cd6b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.946+0000 7f1741ec6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 0x56440ce60460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.946+0000 7f1741ec6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 0x56440ce60460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58366/0 (socket says 172.21.15.16:58366) 2024-09-19T09:28:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.947+0000 7f1741ec6640 1 -- 172.21.15.16:0/1345476176 learned_addr learned my addr 172.21.15.16:0/1345476176 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.947+0000 7f17426c7640 1 --2- 172.21.15.16:0/1345476176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce5bb80 0x56440ce679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.947+0000 7f17416c5640 1 --2- 172.21.15.16:0/1345476176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce56af0 0x56440ce609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.947+0000 7f1741ec6640 1 -- 172.21.15.16:0/1345476176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce5bb80 msgr2=0x56440ce679e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.947+0000 7f1741ec6640 1 --2- 172.21.15.16:0/1345476176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce5bb80 0x56440ce679e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.947+0000 7f1741ec6640 1 -- 172.21.15.16:0/1345476176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce56af0 msgr2=0x56440ce609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.947+0000 7f1741ec6640 1 --2- 172.21.15.16:0/1345476176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce56af0 0x56440ce609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.947+0000 7f1741ec6640 1 -- 172.21.15.16:0/1345476176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56440cac4210 con 0x56440ce561b0 2024-09-19T09:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.947+0000 7f1741ec6640 1 --2- 172.21.15.16:0/1345476176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 0x56440ce60460 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1738031610 tx=0x7f1738009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94e830e5dc5acac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.947+0000 7f17257fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.947+0000 7f17257fa640 1 -- 172.21.15.16:0/1345476176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173804d030 con 0x56440ce561b0 2024-09-19T09:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.948+0000 7f17257fa640 1 -- 172.21.15.16:0/1345476176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f1738004970 con 0x56440ce561b0 2024-09-19T09:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.948+0000 7f1746ec4740 1 -- 172.21.15.16:0/1345476176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f173401bb40 con 0x56440ce561b0 2024-09-19T09:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.948+0000 7f17257fa640 1 -- 172.21.15.16:0/1345476176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1738035040 con 0x56440ce561b0 2024-09-19T09:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.948+0000 7f1746ec4740 10 client.15696 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.948+0000 7f1746ec4740 1 -- 172.21.15.16:0/1345476176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56440cb90620 con 0x56440ce561b0 2024-09-19T09:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.948+0000 7f17257fa640 1 -- 172.21.15.16:0/1345476176 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f173401bb40 con 0x56440ce561b0 2024-09-19T09:28:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.948+0000 7f17257fa640 1 -- 172.21.15.16:0/1345476176 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f173803dc90 con 0x56440ce561b0 2024-09-19T09:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.948+0000 7f1746ec4740 10 client.15696 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.948+0000 7f1746ec4740 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.948+0000 7f1746ec4740 20 client.15696 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.948+0000 7f1746ec4740 1 --2- 172.21.15.16:0/1345476176 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56440ce6c120 0x56440ce6e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.948+0000 7f1746ec4740 4 client.15696 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.949+0000 7f1746ec4740 1 -- 172.21.15.16:0/1345476176 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56440cdbea20 con 0x56440ce6c120 2024-09-19T09:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.949+0000 7f17416c5640 1 --2- 172.21.15.16:0/1345476176 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56440ce6c120 0x56440ce6e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.949+0000 7f17416c5640 1 --2- 172.21.15.16:0/1345476176 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56440ce6c120 0x56440ce6e570 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec9b32f1d0188a7 server_cookie=fa39de8df1a0d1da in_seq=0 out_seq=0 2024-09-19T09:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.949+0000 7f17257fa640 10 client.15696 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.950+0000 7f17257fa640 1 -- 172.21.15.16:0/1345476176 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56440cdbea20 con 0x56440ce6c120 2024-09-19T09:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.950+0000 7f17257fa640 10 client.15696 handle_command_reply: tid=0 2024-09-19T09:28:31.959 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.950+0000 7f1746ec4740 1 client.15696 shutdown 2024-09-19T09:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.950+0000 7f1746ec4740 1 -- 172.21.15.16:0/1345476176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 msgr2=0x56440ce60460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.950+0000 7f1746ec4740 1 --2- 172.21.15.16:0/1345476176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 0x56440ce60460 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1738031610 tx=0x7f1738009260 comp rx=0 tx=0).stop 2024-09-19T09:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.950+0000 7f1746ec4740 1 -- 172.21.15.16:0/1345476176 shutdown_connections 2024-09-19T09:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.950+0000 7f1746ec4740 1 --2- 172.21.15.16:0/1345476176 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56440ce6c120 0x56440ce6e570 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.951+0000 7f17257fa640 0 client.15696 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.951+0000 7f1746ec4740 1 --2- 172.21.15.16:0/1345476176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce5bb80 0x56440ce679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.951+0000 7f1746ec4740 1 --2- 172.21.15.16:0/1345476176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce56af0 0x56440ce609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.951+0000 7f1746ec4740 1 --2- 172.21.15.16:0/1345476176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce561b0 0x56440ce60460 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.951+0000 7f1746ec4740 1 -- 172.21.15.16:0/1345476176 >> 172.21.15.16:0/1345476176 conn(0x56440ce51820 msgr2=0x56440ce53030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.951+0000 7f1746ec4740 1 -- 172.21.15.16:0/1345476176 shutdown_connections 2024-09-19T09:28:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.951+0000 7f1746ec4740 1 -- 172.21.15.16:0/1345476176 wait complete. 2024-09-19T09:28:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.951+0000 7f1746ec4740 20 client.15696 trim_cache size 0 max 16384 2024-09-19T09:28:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.980+0000 7f1746ec4740 1 Processor -- start 2024-09-19T09:28:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.980+0000 7f1746ec4740 1 -- start start 2024-09-19T09:28:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.980+0000 7f1746ec4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce561b0 0x56440ce60a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.980+0000 7f1746ec4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce56af0 0x56440ce56590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.980+0000 7f1746ec4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce5bb80 0x56440ce679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.980+0000 7f1746ec4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1a6db0 con 0x56440ce56af0 2024-09-19T09:28:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.980+0000 7f1746ec4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1a88f0 con 0x56440ce5bb80 2024-09-19T09:28:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.980+0000 7f1746ec4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1a7480 con 0x56440ce561b0 2024-09-19T09:28:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.980+0000 7f17416c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce56af0 0x56440ce56590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.980+0000 7f17416c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce56af0 0x56440ce56590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58372/0 (socket says 172.21.15.16:58372) 2024-09-19T09:28:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.980+0000 7f17416c5640 1 -- 172.21.15.16:0/344805761 learned_addr learned my addr 172.21.15.16:0/344805761 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.980+0000 7f1741ec6640 1 --2- 172.21.15.16:0/344805761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce561b0 0x56440ce60a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.981+0000 7f17426c7640 1 --2- 172.21.15.16:0/344805761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce5bb80 0x56440ce679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.981+0000 7f17416c5640 1 -- 172.21.15.16:0/344805761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce561b0 msgr2=0x56440ce60a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.981+0000 7f17416c5640 1 --2- 172.21.15.16:0/344805761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce561b0 0x56440ce60a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.981+0000 7f17416c5640 1 -- 172.21.15.16:0/344805761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce5bb80 msgr2=0x56440ce679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.981+0000 7f17416c5640 1 --2- 172.21.15.16:0/344805761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce5bb80 0x56440ce679a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.981+0000 7f17416c5640 1 -- 172.21.15.16:0/344805761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56440ce5df70 con 0x56440ce56af0 2024-09-19T09:28:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.981+0000 7f17416c5640 1 --2- 172.21.15.16:0/344805761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce56af0 0x56440ce56590 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f172c098260 tx=0x7f172c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd73d3d848ff59a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.981+0000 7f17277fe640 1 -- 172.21.15.16:0/344805761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c0a4500 con 0x56440ce56af0 2024-09-19T09:28:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.981+0000 7f17277fe640 1 -- 172.21.15.16:0/344805761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c015040 con 0x56440ce56af0 2024-09-19T09:28:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.981+0000 7f17277fe640 1 -- 172.21.15.16:0/344805761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c0a4070 con 0x56440ce56af0 2024-09-19T09:28:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.982+0000 7f1746ec4740 1 -- 172.21.15.16:0/344805761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce56af0 msgr2=0x56440ce56590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.982+0000 7f1746ec4740 1 --2- 172.21.15.16:0/344805761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce56af0 0x56440ce56590 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f172c098260 tx=0x7f172c000bc0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.982+0000 7f1746ec4740 1 -- 172.21.15.16:0/344805761 shutdown_connections 2024-09-19T09:28:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.982+0000 7f1746ec4740 1 --2- 172.21.15.16:0/344805761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce5bb80 0x56440ce679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.982+0000 7f1746ec4740 1 --2- 172.21.15.16:0/344805761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce56af0 0x56440ce56590 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.982+0000 7f1746ec4740 1 --2- 172.21.15.16:0/344805761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce561b0 0x56440ce60a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.982+0000 7f1746ec4740 1 -- 172.21.15.16:0/344805761 >> 172.21.15.16:0/344805761 conn(0x56440ce51820 msgr2=0x56440ce479a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.982+0000 7f1746ec4740 1 -- 172.21.15.16:0/344805761 shutdown_connections 2024-09-19T09:28:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.982+0000 7f1746ec4740 1 -- 172.21.15.16:0/344805761 wait complete. 2024-09-19T09:28:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.982+0000 7f1746ec4740 1 -- start start 2024-09-19T09:28:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.983+0000 7f1746ec4740 1 Processor -- start 2024-09-19T09:28:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.983+0000 7f1746ec4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce561b0 0x56440ce659e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.983+0000 7f1746ec4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce56af0 0x56440ce65f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.983+0000 7f1746ec4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce5bb80 0x56440ce66ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.983+0000 7f1746ec4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1a8220 con 0x56440ce5bb80 2024-09-19T09:28:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.983+0000 7f1746ec4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1a7b50 con 0x56440ce56af0 2024-09-19T09:28:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.983+0000 7f1746ec4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1a4d70 con 0x56440ce561b0 2024-09-19T09:28:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.983+0000 7f1746ec4740 10 client.0 _ll_register_callbacks cb 0x7f1738031a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.983+0000 7f17426c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce5bb80 0x56440ce66ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.983+0000 7f17426c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce5bb80 0x56440ce66ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58374/0 (socket says 172.21.15.16:58374) 2024-09-19T09:28:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.983+0000 7f17426c7640 1 -- 172.21.15.16:0/2818012741 learned_addr learned my addr 172.21.15.16:0/2818012741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.983+0000 7f1741ec6640 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce561b0 0x56440ce659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.984+0000 7f17416c5640 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce56af0 0x56440ce65f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.984+0000 7f17426c7640 1 -- 172.21.15.16:0/2818012741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce561b0 msgr2=0x56440ce659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.984+0000 7f17426c7640 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce561b0 0x56440ce659e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.984+0000 7f17426c7640 1 -- 172.21.15.16:0/2818012741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce56af0 msgr2=0x56440ce65f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.984+0000 7f17426c7640 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce56af0 0x56440ce65f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.984+0000 7f17426c7640 1 -- 172.21.15.16:0/2818012741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56440ce674f0 con 0x56440ce5bb80 2024-09-19T09:28:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.984+0000 7f1741ec6640 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce561b0 0x56440ce659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.984+0000 7f17426c7640 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce5bb80 0x56440ce66ee0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1734010c90 tx=0x7f1734004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acf463750da164a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.984+0000 7f17257fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.984+0000 7f17257fa640 1 -- 172.21.15.16:0/2818012741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734016040 con 0x56440ce5bb80 2024-09-19T09:28:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.984+0000 7f17257fa640 1 -- 172.21.15.16:0/2818012741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f173401b660 con 0x56440ce5bb80 2024-09-19T09:28:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.984+0000 7f1746ec4740 1 -- 172.21.15.16:0/2818012741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f172c015040 con 0x56440ce5bb80 2024-09-19T09:28:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.985+0000 7f17257fa640 1 -- 172.21.15.16:0/2818012741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f173401b860 con 0x56440ce5bb80 2024-09-19T09:28:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.985+0000 7f1746ec4740 10 client.15702 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.985+0000 7f1746ec4740 1 -- 172.21.15.16:0/2818012741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56440ced76d0 con 0x56440ce5bb80 2024-09-19T09:28:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.985+0000 7f17257fa640 1 -- 172.21.15.16:0/2818012741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c015040 con 0x56440ce5bb80 2024-09-19T09:28:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.985+0000 7f17257fa640 1 -- 172.21.15.16:0/2818012741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f173400f040 con 0x56440ce5bb80 2024-09-19T09:28:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.985+0000 7f1746ec4740 10 client.15702 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.985+0000 7f1746ec4740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.985+0000 7f1746ec4740 20 client.15702 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.985+0000 7f1746ec4740 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56440ce6c120 0x56440ced9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.985+0000 7f1746ec4740 4 client.15702 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.985+0000 7f1746ec4740 1 -- 172.21.15.16:0/2818012741 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56440cdb8d70 con 0x56440ce6c120 2024-09-19T09:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.985+0000 7f1741ec6640 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56440ce6c120 0x56440ced9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.986+0000 7f17257fa640 10 client.15702 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.986+0000 7f1741ec6640 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56440ce6c120 0x56440ced9e60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17df6aae3a2cff7 server_cookie=651bbc96eb8736f5 in_seq=0 out_seq=0 2024-09-19T09:28:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f17257fa640 1 -- 172.21.15.16:0/2818012741 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56440cdb8d70 con 0x56440ce6c120 2024-09-19T09:28:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f17257fa640 10 client.15702 handle_command_reply: tid=0 2024-09-19T09:28:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f1746ec4740 1 client.15702 shutdown 2024-09-19T09:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f1746ec4740 1 -- 172.21.15.16:0/2818012741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce5bb80 msgr2=0x56440ce66ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f1746ec4740 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce5bb80 0x56440ce66ee0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1734010c90 tx=0x7f1734004c30 comp rx=0 tx=0).stop 2024-09-19T09:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f1746ec4740 1 -- 172.21.15.16:0/2818012741 shutdown_connections 2024-09-19T09:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f1746ec4740 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56440ce6c120 0x56440ced9e60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f1746ec4740 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56440ce5bb80 0x56440ce66ee0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f1746ec4740 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56440ce56af0 0x56440ce65f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f1746ec4740 1 --2- 172.21.15.16:0/2818012741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56440ce561b0 0x56440ce659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f1746ec4740 1 -- 172.21.15.16:0/2818012741 >> 172.21.15.16:0/2818012741 conn(0x56440ce51820 msgr2=0x56440ce481c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f17257fa640 0 client.15702 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f1746ec4740 1 -- 172.21.15.16:0/2818012741 shutdown_connections 2024-09-19T09:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f1746ec4740 1 -- 172.21.15.16:0/2818012741 wait complete. 2024-09-19T09:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.990+0000 7f1746ec4740 20 client.15702 trim_cache size 0 max 16384 2024-09-19T09:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-19T09:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-19T09:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-19T09:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-19T09:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.991+0000 7f174412a640 1 -- 172.21.15.16:0/1180317624 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f17280a3370 msgr2=0x7f17280a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.991+0000 7f174412a640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f17280a3370 0x7f17280a5830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f17380314e0 tx=0x7f1738005c00 comp rx=0 tx=0).stop 2024-09-19T09:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.991+0000 7f174412a640 1 -- 172.21.15.16:0/1180317624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c104f70 msgr2=0x7f173c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.991+0000 7f174412a640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c104f70 0x7f173c198370 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f172c00d9d0 tx=0x7f172c00de90 comp rx=0 tx=0).stop 2024-09-19T09:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.991+0000 7f174412a640 1 -- 172.21.15.16:0/1180317624 shutdown_connections 2024-09-19T09:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.991+0000 7f174412a640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f17280a3370 0x7f17280a5830 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.991+0000 7f174412a640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f173c1078a0 0x7f173c19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.991+0000 7f174412a640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c104f70 0x7f173c198370 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.991+0000 7f174412a640 1 --2- 172.21.15.16:0/1180317624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c102640 0x7f173c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.991+0000 7f174412a640 1 -- 172.21.15.16:0/1180317624 >> 172.21.15.16:0/1180317624 conn(0x7f173c0fc1e0 msgr2=0x7f173c102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.991+0000 7f174412a640 1 -- 172.21.15.16:0/1180317624 shutdown_connections 2024-09-19T09:28:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:31.991+0000 7f174412a640 1 -- 172.21.15.16:0/1180317624 wait complete. 2024-09-19T09:28:33.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:33 smithi142 ceph-mon[35032]: pgmap v366: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T09:28:33.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:33 smithi097 ceph-mon[34864]: pgmap v366: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T09:28:33.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:33 smithi016 ceph-mon[30979]: pgmap v366: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-19T09:28:35.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:35 smithi142 ceph-mon[35032]: pgmap v367: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 853 B/s wr, 15 op/s 2024-09-19T09:28:35.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:35 smithi097 ceph-mon[34864]: pgmap v367: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 853 B/s wr, 15 op/s 2024-09-19T09:28:35.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:35 smithi016 ceph-mon[30979]: pgmap v367: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 853 B/s wr, 15 op/s 2024-09-19T09:28:37.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:37 smithi142 ceph-mon[35032]: pgmap v368: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-19T09:28:37.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:37 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:37.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:37 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:28:37.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:37 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:28:37.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:37 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:37.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:37 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:37.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:37 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:28:37.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:37 smithi097 ceph-mon[34864]: pgmap v368: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-19T09:28:37.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:37 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:37.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:37 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:28:37.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:37 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:28:37.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:37 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:37.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:37 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:37.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:37 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:28:37.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:37 smithi016 ceph-mon[30979]: pgmap v368: 97 pgs: 97 active+clean; 666 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-19T09:28:37.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:37 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:28:37.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:37 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:28:37.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:37 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:28:37.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:37 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:28:37.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:37 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:28:37.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:37 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:28:39.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:39 smithi142 ceph-mon[35032]: pgmap v369: 97 pgs: 97 active+clean; 667 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T09:28:39.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:39 smithi097 ceph-mon[34864]: pgmap v369: 97 pgs: 97 active+clean; 667 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T09:28:39.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:39 smithi016 ceph-mon[30979]: pgmap v369: 97 pgs: 97 active+clean; 667 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T09:28:39.627 INFO:tasks.workunit.client.0.smithi016.stderr:Updating files: 94% (12226/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T09:28:40.409 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T09:28:40.409 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T09:28:40.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:28:40.479 INFO:tasks.workunit.client.0.smithi016.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T09:28:40.480 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-19T09:28:40.480 INFO:tasks.workunit.client.0.smithi016.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T09:28:40.480 INFO:tasks.workunit.client.0.smithi016.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T09:28:40.480 INFO:tasks.workunit.client.0.smithi016.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T09:28:40.480 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-19T09:28:40.480 INFO:tasks.workunit.client.0.smithi016.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T09:28:40.480 INFO:tasks.workunit.client.0.smithi016.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T09:28:40.480 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-19T09:28:40.480 INFO:tasks.workunit.client.0.smithi016.stderr: git switch -c 2024-09-19T09:28:40.480 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-19T09:28:40.480 INFO:tasks.workunit.client.0.smithi016.stderr:Or undo this operation with: 2024-09-19T09:28:40.480 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-19T09:28:40.480 INFO:tasks.workunit.client.0.smithi016.stderr: git switch - 2024-09-19T09:28:40.481 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-19T09:28:40.481 INFO:tasks.workunit.client.0.smithi016.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T09:28:40.481 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-19T09:28:40.481 INFO:tasks.workunit.client.0.smithi016.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T09:28:40.485 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T09:28:40.585 INFO:tasks.workunit.client.0.smithi016.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T09:28:40.587 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T09:28:40.587 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T09:28:40.669 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:41.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:41 smithi016 ceph-mon[30979]: pgmap v370: 97 pgs: 97 active+clean; 667 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:28:41.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.336+0000 7f8380ed0640 1 -- 172.21.15.16:0/991058081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c1082a0 msgr2=0x7f837c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:41.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.336+0000 7f8380ed0640 1 --2- 172.21.15.16:0/991058081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c1082a0 0x7f837c10ec40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8374009cd0 tx=0x7f837402a8e0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.336+0000 7f8380ed0640 1 -- 172.21.15.16:0/991058081 shutdown_connections 2024-09-19T09:28:41.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.336+0000 7f8380ed0640 1 --2- 172.21.15.16:0/991058081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f837c1036e0 0x7f837c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.336+0000 7f8380ed0640 1 --2- 172.21.15.16:0/991058081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c1082a0 0x7f837c10ec40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.336+0000 7f8380ed0640 1 --2- 172.21.15.16:0/991058081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f837c102d30 0x7f837c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.336+0000 7f8380ed0640 1 -- 172.21.15.16:0/991058081 >> 172.21.15.16:0/991058081 conn(0x7f837c0fc7d0 msgr2=0x7f837c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:41.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.336+0000 7f8380ed0640 1 -- 172.21.15.16:0/991058081 shutdown_connections 2024-09-19T09:28:41.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.336+0000 7f8380ed0640 1 -- 172.21.15.16:0/991058081 wait complete. 2024-09-19T09:28:41.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.337+0000 7f8380ed0640 1 Processor -- start 2024-09-19T09:28:41.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.337+0000 7f8380ed0640 1 -- start start 2024-09-19T09:28:41.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.337+0000 7f8380ed0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f837c102d30 0x7f837c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:41.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.337+0000 7f8380ed0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c1036e0 0x7f837c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:41.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.337+0000 7f8380ed0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f837c1082a0 0x7f837c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:41.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.337+0000 7f8380ed0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1120a0 con 0x7f837c1036e0 2024-09-19T09:28:41.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.337+0000 7f8380ed0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f837c111f20 con 0x7f837c1082a0 2024-09-19T09:28:41.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.337+0000 7f8380ed0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f837c112220 con 0x7f837c102d30 2024-09-19T09:28:41.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.337+0000 7f8371d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c1036e0 0x7f837c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:41.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f8371d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c1036e0 0x7f837c198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52902/0 (socket says 172.21.15.16:52902) 2024-09-19T09:28:41.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f8371d9b640 1 -- 172.21.15.16:0/111547002 learned_addr learned my addr 172.21.15.16:0/111547002 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:41.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f837ad9d640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f837c1082a0 0x7f837c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:41.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f837a59c640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f837c102d30 0x7f837c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:41.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f8371d9b640 1 -- 172.21.15.16:0/111547002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f837c102d30 msgr2=0x7f837c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:41.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f8371d9b640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f837c102d30 0x7f837c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f8371d9b640 1 -- 172.21.15.16:0/111547002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f837c1082a0 msgr2=0x7f837c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:41.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f8371d9b640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f837c1082a0 0x7f837c19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f8371d9b640 1 -- 172.21.15.16:0/111547002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837c19d4f0 con 0x7f837c1036e0 2024-09-19T09:28:41.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f8371d9b640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c1036e0 0x7f837c198a80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f836400e9f0 tx=0x7f836400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:41.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f8373fff640 1 -- 172.21.15.16:0/111547002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836400ccc0 con 0x7f837c1036e0 2024-09-19T09:28:41.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f8380ed0640 1 -- 172.21.15.16:0/111547002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837c19d7e0 con 0x7f837c1036e0 2024-09-19T09:28:41.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f8373fff640 1 -- 172.21.15.16:0/111547002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8364010910 con 0x7f837c1036e0 2024-09-19T09:28:41.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f8373fff640 1 -- 172.21.15.16:0/111547002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364018920 con 0x7f837c1036e0 2024-09-19T09:28:41.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.338+0000 7f8380ed0640 1 -- 172.21.15.16:0/111547002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f837c1a50c0 con 0x7f837c1036e0 2024-09-19T09:28:41.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.339+0000 7f8380ed0640 1 -- 172.21.15.16:0/111547002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f837c103110 con 0x7f837c1036e0 2024-09-19T09:28:41.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.341+0000 7f8373fff640 1 -- 172.21.15.16:0/111547002 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8364010ac0 con 0x7f837c1036e0 2024-09-19T09:28:41.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.341+0000 7f8373fff640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f83680a3240 0x7f83680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:41.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.341+0000 7f837a59c640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f83680a3240 0x7f83680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:41.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.341+0000 7f8373fff640 1 -- 172.21.15.16:0/111547002 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f83640cffd0 con 0x7f837c1036e0 2024-09-19T09:28:41.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.342+0000 7f837a59c640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f83680a3240 0x7f83680a5700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f836c005d00 tx=0x7f836c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:41.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.345+0000 7f8373fff640 1 -- 172.21.15.16:0/111547002 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f836409c6a0 con 0x7f837c1036e0 2024-09-19T09:28:41.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:41 smithi142 ceph-mon[35032]: pgmap v370: 97 pgs: 97 active+clean; 667 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:28:41.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:41 smithi097 ceph-mon[34864]: pgmap v370: 97 pgs: 97 active+clean; 667 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T09:28:41.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.557+0000 7f8380ed0640 1 -- 172.21.15.16:0/111547002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f837c1a52c0 con 0x7f837c1036e0 2024-09-19T09:28:41.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.558+0000 7f8373fff640 1 -- 172.21.15.16:0/111547002 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f83640a09a0 con 0x7f837c1036e0 2024-09-19T09:28:41.559 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:28:41.559 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:28:41.560 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:28:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.560+0000 7f8380ed0640 1 -- 172.21.15.16:0/111547002 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f83680a3240 msgr2=0x7f83680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.560+0000 7f8380ed0640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f83680a3240 0x7f83680a5700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f836c005d00 tx=0x7f836c00b410 comp rx=0 tx=0).stop 2024-09-19T09:28:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.560+0000 7f8380ed0640 1 -- 172.21.15.16:0/111547002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c1036e0 msgr2=0x7f837c198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.560+0000 7f8380ed0640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c1036e0 0x7f837c198a80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f836400e9f0 tx=0x7f836400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.560+0000 7f8380ed0640 1 -- 172.21.15.16:0/111547002 shutdown_connections 2024-09-19T09:28:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.560+0000 7f8380ed0640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f837c1082a0 0x7f837c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.560+0000 7f8380ed0640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c1036e0 0x7f837c198a80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.560+0000 7f8380ed0640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f83680a3240 0x7f83680a5700 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.560+0000 7f8380ed0640 1 --2- 172.21.15.16:0/111547002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f837c102d30 0x7f837c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.560+0000 7f8380ed0640 1 -- 172.21.15.16:0/111547002 >> 172.21.15.16:0/111547002 conn(0x7f837c0fc7d0 msgr2=0x7f837c105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.560+0000 7f8380ed0640 1 -- 172.21.15.16:0/111547002 shutdown_connections 2024-09-19T09:28:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:41.560+0000 7f8380ed0640 1 -- 172.21.15.16:0/111547002 wait complete. 2024-09-19T09:28:42.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:42 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/111547002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:42.287 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T09:28:42.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:42 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/111547002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:42.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:42 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/111547002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:42.410 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T09:28:42.452 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T09:28:42.454 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T09:28:42.454 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T09:28:42.494 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T09:28:42.498 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:28:42.498 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T09:28:42.515 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-09-19T09:28:42.516 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-09-19T09:28:42.516 DEBUG:teuthology.orchestra.run.smithi016:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-09-19T09:28:42.578 INFO:tasks.workunit.client.0.smithi016.stdout:test large (16MB) dio write 2024-09-19T09:28:42.579 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'test large (16MB) dio write' 2024-09-19T09:28:42.579 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-09-19T09:28:42.632 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T09:28:42.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T09:28:42.677 INFO:tasks.workunit.client.0.smithi016.stderr:1+0 records in 2024-09-19T09:28:42.678 INFO:tasks.workunit.client.0.smithi016.stderr:1+0 records out 2024-09-19T09:28:42.678 INFO:tasks.workunit.client.0.smithi016.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.0966987 s, 173 MB/s 2024-09-19T09:28:42.678 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo OK 2024-09-19T09:28:42.679 INFO:tasks.workunit.client.0.smithi016.stdout:OK 2024-09-19T09:28:42.679 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T09:28:42.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T09:28:42.756 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-09-19T09:28:42.757 DEBUG:teuthology.orchestra.run.smithi016:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-09-19T09:28:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:43.216 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:43 smithi016 ceph-mon[30979]: pgmap v371: 97 pgs: 97 active+clean; 667 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T09:28:43.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:43 smithi142 ceph-mon[35032]: pgmap v371: 97 pgs: 97 active+clean; 667 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T09:28:43.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:43 smithi097 ceph-mon[34864]: pgmap v371: 97 pgs: 97 active+clean; 667 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T09:28:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.443+0000 7feef4859640 1 -- 172.21.15.16:0/1750830456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef0102d30 msgr2=0x7feef0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.443+0000 7feef4859640 1 --2- 172.21.15.16:0/1750830456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef0102d30 0x7feef0103110 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7feee0009930 tx=0x7feee002a420 comp rx=0 tx=0).stop 2024-09-19T09:28:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.443+0000 7feef4859640 1 -- 172.21.15.16:0/1750830456 shutdown_connections 2024-09-19T09:28:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.443+0000 7feef4859640 1 --2- 172.21.15.16:0/1750830456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feef01036e0 0x7feef0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.443+0000 7feef4859640 1 --2- 172.21.15.16:0/1750830456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feef01082a0 0x7feef010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.443+0000 7feef4859640 1 --2- 172.21.15.16:0/1750830456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef0102d30 0x7feef0103110 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.443+0000 7feef4859640 1 -- 172.21.15.16:0/1750830456 >> 172.21.15.16:0/1750830456 conn(0x7feef00fc7d0 msgr2=0x7feef00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.443+0000 7feef4859640 1 -- 172.21.15.16:0/1750830456 shutdown_connections 2024-09-19T09:28:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.443+0000 7feef4859640 1 -- 172.21.15.16:0/1750830456 wait complete. 2024-09-19T09:28:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feef4859640 1 Processor -- start 2024-09-19T09:28:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feef4859640 1 -- start start 2024-09-19T09:28:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feef4859640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef0102d30 0x7feef0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feef4859640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feef01036e0 0x7feef0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feef4859640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feef01082a0 0x7feef0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feef4859640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feef01120e0 con 0x7feef0102d30 2024-09-19T09:28:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feef4859640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feef0111f60 con 0x7feef01036e0 2024-09-19T09:28:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feef4859640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feef0112260 con 0x7feef01082a0 2024-09-19T09:28:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feeee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef0102d30 0x7feef0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feeee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef0102d30 0x7feef0100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52920/0 (socket says 172.21.15.16:52920) 2024-09-19T09:28:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feeee59c640 1 -- 172.21.15.16:0/3201549881 learned_addr learned my addr 172.21.15.16:0/3201549881 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feeeed9d640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feef01082a0 0x7feef0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feeedd9b640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feef01036e0 0x7feef0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feeee59c640 1 -- 172.21.15.16:0/3201549881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feef01082a0 msgr2=0x7feef0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feeee59c640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feef01082a0 0x7feef0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feeee59c640 1 -- 172.21.15.16:0/3201549881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feef01036e0 msgr2=0x7feef0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feeee59c640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feef01036e0 0x7feef0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.444+0000 7feeee59c640 1 -- 172.21.15.16:0/3201549881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef01a69e0 con 0x7feef0102d30 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.445+0000 7feeee59c640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef0102d30 0x7feef0100990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7feee002a940 tx=0x7feee0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.445+0000 7feedf7fe640 1 -- 172.21.15.16:0/3201549881 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee003f070 con 0x7feef0102d30 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.445+0000 7feedf7fe640 1 -- 172.21.15.16:0/3201549881 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee0031c90 con 0x7feef0102d30 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.445+0000 7feedf7fe640 1 -- 172.21.15.16:0/3201549881 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0035030 con 0x7feef0102d30 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.445+0000 7feef4859640 1 -- 172.21.15.16:0/3201549881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef01a6c70 con 0x7feef0102d30 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.445+0000 7feef4859640 1 -- 172.21.15.16:0/3201549881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef01a71a0 con 0x7feef0102d30 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.447+0000 7feedf7fe640 1 -- 172.21.15.16:0/3201549881 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7feee0042070 con 0x7feef0102d30 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feef6c32740 1 Processor -- start 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feef6c32740 1 -- start start 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feef6c32740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f0850 0x562f9c7f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feef6c32740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7f1170 0x562f9c7fb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feef6c32740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7fba30 0x562f9c7fde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feedf7fe640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7feed80a32c0 0x7feed80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feef6c32740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f9c755160 con 0x562f9c7fba30 2024-09-19T09:28:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feef6c32740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562f9c6fc720 con 0x562f9c7f0850 2024-09-19T09:28:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feef6c32740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562f9c751120 con 0x562f9c7f1170 2024-09-19T09:28:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feeee59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7f1170 0x562f9c7fb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feeedd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7fba30 0x562f9c7fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feeeed9d640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7feed80a32c0 0x7feed80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feeedd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7fba30 0x562f9c7fde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52936/0 (socket says 172.21.15.16:52936) 2024-09-19T09:28:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feeee59c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7f1170 0x562f9c7fb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:51562/0 (socket says 172.21.15.16:51562) 2024-09-19T09:28:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feeedd9b640 1 -- 172.21.15.16:0/2496772118 learned_addr learned my addr 172.21.15.16:0/2496772118 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.448+0000 7feedf7fe640 1 -- 172.21.15.16:0/3201549881 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7feee00eff80 con 0x7feef0102d30 2024-09-19T09:28:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.449+0000 7feeeed9d640 1 --2- 172.21.15.16:0/2496772118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f0850 0x562f9c7f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.449+0000 7feeee59c640 1 -- 172.21.15.16:0/2496772118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f0850 msgr2=0x562f9c7f0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.449+0000 7feeee59c640 1 --2- 172.21.15.16:0/2496772118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f0850 0x562f9c7f0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.449+0000 7feeee59c640 1 -- 172.21.15.16:0/2496772118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7fba30 msgr2=0x562f9c7fde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.449+0000 7feeee59c640 1 --2- 172.21.15.16:0/2496772118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7fba30 0x562f9c7fde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.449+0000 7feeee59c640 1 -- 172.21.15.16:0/2496772118 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f9c4c6ba0 con 0x562f9c7f1170 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.449+0000 7feeeed9d640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7feed80a32c0 0x7feed80a5780 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feee8007fd0 tx=0x7feee8007d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.449+0000 7feeee59c640 1 --2- 172.21.15.16:0/2496772118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7f1170 0x562f9c7fb4f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feee00bc350 tx=0x7feee0049e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1dcaab9f5567df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.449+0000 7feedcff9640 1 -- 172.21.15.16:0/2496772118 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0049070 con 0x562f9c7f1170 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.449+0000 7feedcff9640 1 -- 172.21.15.16:0/2496772118 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee00bc650 con 0x562f9c7f1170 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.449+0000 7feedcff9640 1 -- 172.21.15.16:0/2496772118 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00b8cc0 con 0x562f9c7f1170 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.450+0000 7feef6c32740 1 -- 172.21.15.16:0/2496772118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7f1170 msgr2=0x562f9c7fb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.450+0000 7feef6c32740 1 --2- 172.21.15.16:0/2496772118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7f1170 0x562f9c7fb4f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feee00bc350 tx=0x7feee0049e10 comp rx=0 tx=0).stop 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.450+0000 7feef6c32740 1 -- 172.21.15.16:0/2496772118 shutdown_connections 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.450+0000 7feef6c32740 1 --2- 172.21.15.16:0/2496772118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7f1170 0x562f9c7fb4f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.450+0000 7feef6c32740 1 --2- 172.21.15.16:0/2496772118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7fba30 0x562f9c7fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.450+0000 7feef6c32740 1 --2- 172.21.15.16:0/2496772118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f0850 0x562f9c7f0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.450+0000 7feef6c32740 1 -- 172.21.15.16:0/2496772118 >> 172.21.15.16:0/2496772118 conn(0x562f9c7ecde0 msgr2=0x562f9c7ed1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.450+0000 7feef6c32740 1 -- 172.21.15.16:0/2496772118 shutdown_connections 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.450+0000 7feef6c32740 1 -- 172.21.15.16:0/2496772118 wait complete. 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.450+0000 7feef6c32740 1 -- start start 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.450+0000 7feef6c32740 1 Processor -- start 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feef6c32740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c7f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feef6c32740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7f1170 0x562f9c7f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feef6c32740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7fba30 0x562f9c7f9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feef6c32740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f9c7fe3c0 con 0x562f9c7f0850 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feef6c32740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562f9c6dd900 con 0x562f9c7fba30 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feef6c32740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562f9c7fe540 con 0x562f9c7f1170 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feef6c32740 10 client.0 _ll_register_callbacks cb 0x562f9c6db780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:43.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feeee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c7f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feeedd9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7f1170 0x562f9c7f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feeee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c7f88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52950/0 (socket says 172.21.15.16:52950) 2024-09-19T09:28:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feeee59c640 1 -- 172.21.15.16:0/1340293826 learned_addr learned my addr 172.21.15.16:0/1340293826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feeee59c640 1 -- 172.21.15.16:0/1340293826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7f1170 msgr2=0x562f9c7f8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feeee59c640 1 --2- 172.21.15.16:0/1340293826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7f1170 0x562f9c7f8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feeee59c640 1 -- 172.21.15.16:0/1340293826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7fba30 msgr2=0x562f9c7f9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feeee59c640 1 --2- 172.21.15.16:0/1340293826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7fba30 0x562f9c7f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feeee59c640 1 -- 172.21.15.16:0/1340293826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f9c71dc50 con 0x562f9c7f0850 2024-09-19T09:28:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feeee59c640 1 --2- 172.21.15.16:0/1340293826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c7f88e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7feee00c4ce0 tx=0x7feee002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f30d0c33ef12721 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.451+0000 7feed5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feed5ffb640 1 -- 172.21.15.16:0/1340293826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0049070 con 0x562f9c7f0850 2024-09-19T09:28:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feed5ffb640 1 -- 172.21.15.16:0/1340293826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7feee003b5f0 con 0x562f9c7f0850 2024-09-19T09:28:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feef6c32740 1 -- 172.21.15.16:0/1340293826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee00bc650 con 0x562f9c7f0850 2024-09-19T09:28:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feed5ffb640 1 -- 172.21.15.16:0/1340293826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feee00c0950 con 0x562f9c7f0850 2024-09-19T09:28:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feef6c32740 10 client.15720 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feef6c32740 1 -- 172.21.15.16:0/1340293826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f9c7fe3c0 con 0x562f9c7f0850 2024-09-19T09:28:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feed5ffb640 1 -- 172.21.15.16:0/1340293826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee00bc650 con 0x562f9c7f0850 2024-09-19T09:28:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feed5ffb640 1 -- 172.21.15.16:0/1340293826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feee00c4d80 con 0x562f9c7f0850 2024-09-19T09:28:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feef6c32740 10 client.15720 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feef6c32740 10 client.15720 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:28:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feef6c32740 20 client.15720 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feef6c32740 1 --2- 172.21.15.16:0/1340293826 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f9c80b1f0 0x562f9c80d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feef6c32740 4 client.15720 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.452+0000 7feef6c32740 1 -- 172.21.15.16:0/1340293826 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f9c762890 con 0x562f9c80b1f0 2024-09-19T09:28:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.453+0000 7feeedd9b640 1 --2- 172.21.15.16:0/1340293826 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f9c80b1f0 0x562f9c80d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.453+0000 7feeedd9b640 1 --2- 172.21.15.16:0/1340293826 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f9c80b1f0 0x562f9c80d640 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f02b86adb287164 server_cookie=5f0e267337c59ffd in_seq=0 out_seq=0 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.453+0000 7feed5ffb640 10 client.15720 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feed5ffb640 1 -- 172.21.15.16:0/1340293826 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f9c762890 con 0x562f9c80b1f0 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feed5ffb640 10 client.15720 handle_command_reply: tid=0 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feef6c32740 1 client.15720 shutdown 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feef6c32740 1 -- 172.21.15.16:0/1340293826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 msgr2=0x562f9c7f88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feef6c32740 1 --2- 172.21.15.16:0/1340293826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c7f88e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7feee00c4ce0 tx=0x7feee002a420 comp rx=0 tx=0).stop 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feef6c32740 1 -- 172.21.15.16:0/1340293826 shutdown_connections 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feef6c32740 1 --2- 172.21.15.16:0/1340293826 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f9c80b1f0 0x562f9c80d640 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feef6c32740 1 --2- 172.21.15.16:0/1340293826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7f1170 0x562f9c7f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feed5ffb640 0 client.15720 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feef6c32740 1 --2- 172.21.15.16:0/1340293826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7fba30 0x562f9c7f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feef6c32740 1 --2- 172.21.15.16:0/1340293826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c7f88e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feef6c32740 1 -- 172.21.15.16:0/1340293826 >> 172.21.15.16:0/1340293826 conn(0x562f9c7ecde0 msgr2=0x562f9c7f2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:43.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feef6c32740 1 -- 172.21.15.16:0/1340293826 shutdown_connections 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feef6c32740 1 -- 172.21.15.16:0/1340293826 wait complete. 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.454+0000 7feef6c32740 20 client.15720 trim_cache size 0 max 16384 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feef6c32740 1 Processor -- start 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feef6c32740 1 -- start start 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feef6c32740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c7f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feef6c32740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f1170 0x562f9c7f7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feef6c32740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7fba30 0x562f9c7f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feef6c32740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feef01af6f0 con 0x562f9c7f0850 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feef6c32740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feef01b1230 con 0x562f9c7f1170 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feef6c32740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feef01afdc0 con 0x562f9c7fba30 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feeee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c7f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feeee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c7f3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52958/0 (socket says 172.21.15.16:52958) 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feeee59c640 1 -- 172.21.15.16:0/3548221290 learned_addr learned my addr 172.21.15.16:0/3548221290 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feeeed9d640 1 --2- 172.21.15.16:0/3548221290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7fba30 0x562f9c7f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feeee59c640 1 -- 172.21.15.16:0/3548221290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7fba30 msgr2=0x562f9c7f32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feeedd9b640 1 --2- 172.21.15.16:0/3548221290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f1170 0x562f9c7f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feeee59c640 1 --2- 172.21.15.16:0/3548221290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7fba30 0x562f9c7f32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feeee59c640 1 -- 172.21.15.16:0/3548221290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f1170 msgr2=0x562f9c7f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.471+0000 7feeee59c640 1 --2- 172.21.15.16:0/3548221290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f1170 0x562f9c7f7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.472+0000 7feeee59c640 1 -- 172.21.15.16:0/3548221290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f9c7ecc10 con 0x562f9c7f0850 2024-09-19T09:28:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.472+0000 7feeee59c640 1 --2- 172.21.15.16:0/3548221290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c7f3e00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7feee00ba900 tx=0x7feee00f0200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205b7e79ac2b601c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.472+0000 7feed7fff640 1 -- 172.21.15.16:0/3548221290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0049030 con 0x562f9c7f0850 2024-09-19T09:28:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.472+0000 7feed7fff640 1 -- 172.21.15.16:0/3548221290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee00ba950 con 0x562f9c7f0850 2024-09-19T09:28:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.472+0000 7feed7fff640 1 -- 172.21.15.16:0/3548221290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00bc650 con 0x562f9c7f0850 2024-09-19T09:28:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.472+0000 7feef6c32740 1 -- 172.21.15.16:0/3548221290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 msgr2=0x562f9c7f3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.472+0000 7feef6c32740 1 --2- 172.21.15.16:0/3548221290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c7f3e00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7feee00ba900 tx=0x7feee00f0200 comp rx=0 tx=0).stop 2024-09-19T09:28:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 -- 172.21.15.16:0/3548221290 shutdown_connections 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 --2- 172.21.15.16:0/3548221290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7fba30 0x562f9c7f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 --2- 172.21.15.16:0/3548221290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f1170 0x562f9c7f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 --2- 172.21.15.16:0/3548221290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c7f3e00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 -- 172.21.15.16:0/3548221290 >> 172.21.15.16:0/3548221290 conn(0x562f9c7ecde0 msgr2=0x562f9c7f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 -- 172.21.15.16:0/3548221290 shutdown_connections 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 -- 172.21.15.16:0/3548221290 wait complete. 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 -- start start 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 Processor -- start 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c803f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f1170 0x562f9c7f5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7fba30 0x562f9c7f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feef01b0b60 con 0x562f9c7f0850 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feef01b0490 con 0x562f9c7f1170 2024-09-19T09:28:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feef01ad6b0 con 0x562f9c7fba30 2024-09-19T09:28:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feef6c32740 10 client.0 _ll_register_callbacks cb 0x7feee400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feeee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c803f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feeee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c803f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52960/0 (socket says 172.21.15.16:52960) 2024-09-19T09:28:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feeee59c640 1 -- 172.21.15.16:0/1108304500 learned_addr learned my addr 172.21.15.16:0/1108304500 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.473+0000 7feeeed9d640 1 --2- 172.21.15.16:0/1108304500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7fba30 0x562f9c7f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feeedd9b640 1 --2- 172.21.15.16:0/1108304500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f1170 0x562f9c7f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feeee59c640 1 -- 172.21.15.16:0/1108304500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7fba30 msgr2=0x562f9c7f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feeee59c640 1 --2- 172.21.15.16:0/1108304500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7fba30 0x562f9c7f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feeee59c640 1 -- 172.21.15.16:0/1108304500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f1170 msgr2=0x562f9c7f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feeee59c640 1 --2- 172.21.15.16:0/1108304500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f1170 0x562f9c7f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feeee59c640 1 -- 172.21.15.16:0/1108304500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f9c7f65f0 con 0x562f9c7f0850 2024-09-19T09:28:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feeee59c640 1 --2- 172.21.15.16:0/1108304500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c803f70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7feee0004650 tx=0x7feee00f0270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d523dff077fa3823 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feed5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feed5ffb640 1 -- 172.21.15.16:0/1108304500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00da060 con 0x562f9c7f0850 2024-09-19T09:28:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feed5ffb640 1 -- 172.21.15.16:0/1108304500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7feee00bbaa0 con 0x562f9c7f0850 2024-09-19T09:28:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feef6c32740 1 -- 172.21.15.16:0/1108304500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562f9c75cbe0 con 0x562f9c7f0850 2024-09-19T09:28:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feed5ffb640 1 -- 172.21.15.16:0/1108304500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feee0049070 con 0x562f9c7f0850 2024-09-19T09:28:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feef6c32740 10 client.15726 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.474+0000 7feef6c32740 1 -- 172.21.15.16:0/1108304500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f9c7f65f0 con 0x562f9c7f0850 2024-09-19T09:28:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.475+0000 7feed5ffb640 1 -- 172.21.15.16:0/1108304500 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562f9c75cbe0 con 0x562f9c7f0850 2024-09-19T09:28:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.475+0000 7feed5ffb640 1 -- 172.21.15.16:0/1108304500 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feee00b83a0 con 0x562f9c7f0850 2024-09-19T09:28:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.475+0000 7feef6c32740 10 client.15726 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.475+0000 7feef6c32740 10 client.15726 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:28:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.475+0000 7feef6c32740 20 client.15726 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.475+0000 7feef6c32740 1 --2- 172.21.15.16:0/1108304500 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f9c8762e0 0x562f9c8786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.475+0000 7feef6c32740 4 client.15726 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T09:28:43.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.475+0000 7feef6c32740 1 -- 172.21.15.16:0/1108304500 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x562f9c6d8f40 con 0x562f9c8762e0 2024-09-19T09:28:43.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.475+0000 7feeedd9b640 1 --2- 172.21.15.16:0/1108304500 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f9c8762e0 0x562f9c8786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:43.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.475+0000 7feeedd9b640 1 --2- 172.21.15.16:0/1108304500 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f9c8762e0 0x562f9c8786d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c4f7c8f5bb7ceac server_cookie=9309ea8e93e71b0f in_seq=0 out_seq=0 2024-09-19T09:28:43.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:43.475+0000 7feed5ffb640 10 client.15726 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feed5ffb640 1 -- 172.21.15.16:0/1108304500 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562f9c6d8f40 con 0x562f9c8762e0 2024-09-19T09:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feed5ffb640 10 client.15726 handle_command_reply: tid=0 2024-09-19T09:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feef6c32740 1 client.15726 shutdown 2024-09-19T09:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feef6c32740 1 -- 172.21.15.16:0/1108304500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 msgr2=0x562f9c803f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feef6c32740 1 --2- 172.21.15.16:0/1108304500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c803f70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7feee0004650 tx=0x7feee00f0270 comp rx=0 tx=0).stop 2024-09-19T09:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feef6c32740 1 -- 172.21.15.16:0/1108304500 shutdown_connections 2024-09-19T09:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feef6c32740 1 --2- 172.21.15.16:0/1108304500 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562f9c8762e0 0x562f9c8786d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feef6c32740 1 --2- 172.21.15.16:0/1108304500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f9c7fba30 0x562f9c7f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feef6c32740 1 --2- 172.21.15.16:0/1108304500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f9c7f1170 0x562f9c7f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feef6c32740 1 --2- 172.21.15.16:0/1108304500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f9c7f0850 0x562f9c803f70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feef6c32740 1 -- 172.21.15.16:0/1108304500 >> 172.21.15.16:0/1108304500 conn(0x562f9c7ecde0 msgr2=0x562f9c806580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feed5ffb640 0 client.15726 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feef6c32740 1 -- 172.21.15.16:0/1108304500 shutdown_connections 2024-09-19T09:28:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feef6c32740 1 -- 172.21.15.16:0/1108304500 wait complete. 2024-09-19T09:28:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.348+0000 7feef6c32740 20 client.15726 trim_cache size 0 max 16384 2024-09-19T09:28:44.351 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:28:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-19T09:28:44.351 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24500, 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 6, 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "9f439ef8": { 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 6, 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.3, 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-19T09:28:44.352 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-19T09:28:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-19T09:28:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-19T09:28:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-19T09:28:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:44.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:28:44.353 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:28:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.349+0000 7feef4859640 1 -- 172.21.15.16:0/3201549881 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7feed80a32c0 msgr2=0x7feed80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.349+0000 7feef4859640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7feed80a32c0 0x7feed80a5780 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feee8007fd0 tx=0x7feee8007d60 comp rx=0 tx=0).stop 2024-09-19T09:28:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.349+0000 7feef4859640 1 -- 172.21.15.16:0/3201549881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef0102d30 msgr2=0x7feef0100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.349+0000 7feef4859640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef0102d30 0x7feef0100990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7feee002a940 tx=0x7feee0003b90 comp rx=0 tx=0).stop 2024-09-19T09:28:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.349+0000 7feef4859640 1 -- 172.21.15.16:0/3201549881 shutdown_connections 2024-09-19T09:28:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.349+0000 7feef4859640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7feed80a32c0 0x7feed80a5780 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.349+0000 7feef4859640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feef01082a0 0x7feef0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.349+0000 7feef4859640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feef01036e0 0x7feef0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.349+0000 7feef4859640 1 --2- 172.21.15.16:0/3201549881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef0102d30 0x7feef0100990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.349+0000 7feef4859640 1 -- 172.21.15.16:0/3201549881 >> 172.21.15.16:0/3201549881 conn(0x7feef00fc7d0 msgr2=0x7feef0105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.349+0000 7feef4859640 1 -- 172.21.15.16:0/3201549881 shutdown_connections 2024-09-19T09:28:44.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:44.349+0000 7feef4859640 1 -- 172.21.15.16:0/3201549881 wait complete. 2024-09-19T09:28:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.083+0000 7f932e52c640 1 -- 172.21.15.16:0/60762381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9328101c30 msgr2=0x7f93281020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.083+0000 7f932e52c640 1 --2- 172.21.15.16:0/60762381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9328101c30 0x7f93281020b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f931c009860 tx=0x7f931c02a340 comp rx=0 tx=0).stop 2024-09-19T09:28:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.084+0000 7f932e52c640 1 -- 172.21.15.16:0/60762381 shutdown_connections 2024-09-19T09:28:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.084+0000 7f932e52c640 1 --2- 172.21.15.16:0/60762381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9328101c30 0x7f93281020b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.084+0000 7f932e52c640 1 --2- 172.21.15.16:0/60762381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93281025f0 0x7f9328108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.084+0000 7f932e52c640 1 --2- 172.21.15.16:0/60762381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9328100a40 0x7f9328100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.084+0000 7f932e52c640 1 -- 172.21.15.16:0/60762381 >> 172.21.15.16:0/60762381 conn(0x7f93280fc1e0 msgr2=0x7f93280fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.084+0000 7f932e52c640 1 -- 172.21.15.16:0/60762381 shutdown_connections 2024-09-19T09:28:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.084+0000 7f932e52c640 1 -- 172.21.15.16:0/60762381 wait complete. 2024-09-19T09:28:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.084+0000 7f932e52c640 1 Processor -- start 2024-09-19T09:28:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.084+0000 7f932e52c640 1 -- start start 2024-09-19T09:28:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f932e52c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9328100a40 0x7f9328197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f932e52c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9328101c30 0x7f9328198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f932e52c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93281025f0 0x7f932819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f932e52c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f932810d810 con 0x7f9328100a40 2024-09-19T09:28:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f932e52c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f932810d690 con 0x7f93281025f0 2024-09-19T09:28:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f932e52c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f932810d990 con 0x7f9328101c30 2024-09-19T09:28:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f9327fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9328100a40 0x7f9328197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f9327fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9328100a40 0x7f9328197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52964/0 (socket says 172.21.15.16:52964) 2024-09-19T09:28:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f9327fff640 1 -- 172.21.15.16:0/1968789694 learned_addr learned my addr 172.21.15.16:0/1968789694 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f93277fe640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9328101c30 0x7f9328198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f932cac9640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93281025f0 0x7f932819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f9327fff640 1 -- 172.21.15.16:0/1968789694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9328101c30 msgr2=0x7f9328198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f9327fff640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9328101c30 0x7f9328198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f9327fff640 1 -- 172.21.15.16:0/1968789694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93281025f0 msgr2=0x7f932819f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f9327fff640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93281025f0 0x7f932819f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f9327fff640 1 -- 172.21.15.16:0/1968789694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932819f9d0 con 0x7f9328100a40 2024-09-19T09:28:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.085+0000 7f9327fff640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9328100a40 0x7f9328197dc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f931800e9a0 tx=0x7f931800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.086+0000 7f93257fa640 1 -- 172.21.15.16:0/1968789694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931800cd80 con 0x7f9328100a40 2024-09-19T09:28:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.086+0000 7f93257fa640 1 -- 172.21.15.16:0/1968789694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9318010910 con 0x7f9328100a40 2024-09-19T09:28:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.086+0000 7f93257fa640 1 -- 172.21.15.16:0/1968789694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318018a30 con 0x7f9328100a40 2024-09-19T09:28:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.086+0000 7f932e52c640 1 -- 172.21.15.16:0/1968789694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932819fcc0 con 0x7f9328100a40 2024-09-19T09:28:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.086+0000 7f932e52c640 1 -- 172.21.15.16:0/1968789694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93281a01c0 con 0x7f9328100a40 2024-09-19T09:28:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.088+0000 7f93257fa640 1 -- 172.21.15.16:0/1968789694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9318010ac0 con 0x7f9328100a40 2024-09-19T09:28:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.088+0000 7f93312bd740 1 Processor -- start 2024-09-19T09:28:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.088+0000 7f93312bd740 1 -- start start 2024-09-19T09:28:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93312bd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d519a0 0x557ed9d51da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93312bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d522e0 0x557ed9d56da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93312bd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d57370 0x557ed9d59830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93312bd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557ed9cb52f0 con 0x557ed9d57370 2024-09-19T09:28:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93312bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557ed9c5d270 con 0x557ed9d522e0 2024-09-19T09:28:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93312bd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557ed9cb12b0 con 0x557ed9d519a0 2024-09-19T09:28:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93257fa640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f93100a32f0 0x7f93100a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f932cac9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d519a0 0x557ed9d51da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d57370 0x557ed9d59830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f9327fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d522e0 0x557ed9d56da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d57370 0x557ed9d59830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52974/0 (socket says 172.21.15.16:52974) 2024-09-19T09:28:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93277fe640 1 -- 172.21.15.16:0/2377349735 learned_addr learned my addr 172.21.15.16:0/2377349735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f932cac9640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f93100a32f0 0x7f93100a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93257fa640 1 -- 172.21.15.16:0/1968789694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f93180cfd60 con 0x7f9328100a40 2024-09-19T09:28:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93277fe640 1 -- 172.21.15.16:0/2377349735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d519a0 msgr2=0x557ed9d51da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93277fe640 1 --2- 172.21.15.16:0/2377349735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d519a0 0x557ed9d51da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93277fe640 1 -- 172.21.15.16:0/2377349735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d522e0 msgr2=0x557ed9d56da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93277fe640 1 --2- 172.21.15.16:0/2377349735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d522e0 0x557ed9d56da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.089+0000 7f93277fe640 1 -- 172.21.15.16:0/2377349735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed9c5c340 con 0x557ed9d57370 2024-09-19T09:28:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.090+0000 7f93277fe640 1 --2- 172.21.15.16:0/2377349735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d57370 0x557ed9d59830 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f931c02a860 tx=0x7f931c0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16f304b0fde100e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.090+0000 7f932cac9640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f93100a32f0 0x7f93100a57b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9320005c40 tx=0x7f9320009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.090+0000 7f930e7fc640 1 -- 172.21.15.16:0/2377349735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c042070 con 0x557ed9d57370 2024-09-19T09:28:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.090+0000 7f930e7fc640 1 -- 172.21.15.16:0/2377349735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f931c030e50 con 0x557ed9d57370 2024-09-19T09:28:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.090+0000 7f930e7fc640 1 -- 172.21.15.16:0/2377349735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c037040 con 0x557ed9d57370 2024-09-19T09:28:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.091+0000 7f93312bd740 1 -- 172.21.15.16:0/2377349735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d57370 msgr2=0x557ed9d59830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.091+0000 7f93312bd740 1 --2- 172.21.15.16:0/2377349735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d57370 0x557ed9d59830 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f931c02a860 tx=0x7f931c0095a0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.091+0000 7f93312bd740 1 -- 172.21.15.16:0/2377349735 shutdown_connections 2024-09-19T09:28:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.091+0000 7f93312bd740 1 --2- 172.21.15.16:0/2377349735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d57370 0x557ed9d59830 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.091+0000 7f93312bd740 1 --2- 172.21.15.16:0/2377349735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d522e0 0x557ed9d56da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.091+0000 7f93312bd740 1 --2- 172.21.15.16:0/2377349735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d519a0 0x557ed9d51da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.091+0000 7f93312bd740 1 -- 172.21.15.16:0/2377349735 >> 172.21.15.16:0/2377349735 conn(0x557ed9d4d010 msgr2=0x557ed9d4d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.091+0000 7f93312bd740 1 -- 172.21.15.16:0/2377349735 shutdown_connections 2024-09-19T09:28:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.091+0000 7f93312bd740 1 -- 172.21.15.16:0/2377349735 wait complete. 2024-09-19T09:28:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.091+0000 7f93312bd740 1 -- start start 2024-09-19T09:28:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.091+0000 7f93312bd740 1 Processor -- start 2024-09-19T09:28:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f93312bd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d519a0 0x557ed9d5bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f93312bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d522e0 0x557ed9d5c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f93312bd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d57370 0x557ed9d630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f9327fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d519a0 0x557ed9d5bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f9327fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d519a0 0x557ed9d5bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:51628/0 (socket says 172.21.15.16:51628) 2024-09-19T09:28:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f93277fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d522e0 0x557ed9d5c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f93277fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d522e0 0x557ed9d5c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:40904/0 (socket says 172.21.15.16:40904) 2024-09-19T09:28:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f932cac9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d57370 0x557ed9d630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f932cac9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d57370 0x557ed9d630b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52984/0 (socket says 172.21.15.16:52984) 2024-09-19T09:28:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f93312bd740 1 -- 172.21.15.16:0/372144570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557ed9d5a2e0 con 0x557ed9d57370 2024-09-19T09:28:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f9327fff640 1 -- 172.21.15.16:0/372144570 learned_addr learned my addr 172.21.15.16:0/372144570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f93312bd740 1 -- 172.21.15.16:0/372144570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557ed9c3e290 con 0x557ed9d522e0 2024-09-19T09:28:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f93312bd740 1 -- 172.21.15.16:0/372144570 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557ed9d5a460 con 0x557ed9d519a0 2024-09-19T09:28:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f93312bd740 10 client.0 _ll_register_callbacks cb 0x557ed9c6f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f9327fff640 1 -- 172.21.15.16:0/372144570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d522e0 msgr2=0x557ed9d5c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f9327fff640 1 --2- 172.21.15.16:0/372144570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d522e0 0x557ed9d5c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f9327fff640 1 -- 172.21.15.16:0/372144570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d57370 msgr2=0x557ed9d630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f9327fff640 1 --2- 172.21.15.16:0/372144570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d57370 0x557ed9d630b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.092+0000 7f9327fff640 1 -- 172.21.15.16:0/372144570 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed99c8210 con 0x557ed9d519a0 2024-09-19T09:28:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.093+0000 7f93312bd740 1 -- 172.21.15.16:0/372144570 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f931c030e50 con 0x557ed9d519a0 2024-09-19T09:28:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.093+0000 7f9327fff640 1 --2- 172.21.15.16:0/372144570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d519a0 0x557ed9d5bb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f93180058c0 tx=0x7f931806bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b90d7adb63c6a0f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.093+0000 7f93077fe640 10 client.25294 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-19T09:28:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.094+0000 7f93077fe640 1 -- 172.21.15.16:0/372144570 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931809bca0 con 0x557ed9d519a0 2024-09-19T09:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.095+0000 7f93077fe640 1 -- 172.21.15.16:0/372144570 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f931809b990 con 0x557ed9d519a0 2024-09-19T09:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.095+0000 7f93077fe640 1 -- 172.21.15.16:0/372144570 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f931809a480 con 0x557ed9d519a0 2024-09-19T09:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.095+0000 7f93312bd740 10 client.25294 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.095+0000 7f93312bd740 1 -- 172.21.15.16:0/372144570 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=28}) -- 0x557ed9a94620 con 0x557ed9d519a0 2024-09-19T09:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.095+0000 7f93077fe640 1 -- 172.21.15.16:0/372144570 <== mon.1 v2:172.21.15.142:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f93180cdce0 con 0x557ed9d519a0 2024-09-19T09:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.095+0000 7f93312bd740 10 client.25294 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.095+0000 7f93312bd740 10 client.25294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.095+0000 7f93312bd740 20 client.25294 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.095+0000 7f93312bd740 1 --2- 172.21.15.16:0/372144570 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x557ed9d67610 0x557ed9d69a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.095+0000 7f93312bd740 4 client.25294 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.095+0000 7f93312bd740 1 -- 172.21.15.16:0/372144570 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ed9cc2a20 con 0x557ed9d67610 2024-09-19T09:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.095+0000 7f93277fe640 1 --2- 172.21.15.16:0/372144570 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x557ed9d67610 0x557ed9d69a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.096+0000 7f93277fe640 1 --2- 172.21.15.16:0/372144570 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x557ed9d67610 0x557ed9d69a40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcfb0724153b93af server_cookie=3dffdd59da3bcce9 in_seq=0 out_seq=0 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.096+0000 7f93077fe640 10 client.25294 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.096+0000 7f93077fe640 1 -- 172.21.15.16:0/372144570 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ed9cc2a20 con 0x557ed9d67610 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.096+0000 7f93077fe640 10 client.25294 handle_command_reply: tid=0 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.097+0000 7f93312bd740 1 client.25294 shutdown 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.097+0000 7f93312bd740 1 -- 172.21.15.16:0/372144570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d519a0 msgr2=0x557ed9d5bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.097+0000 7f93312bd740 1 --2- 172.21.15.16:0/372144570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d519a0 0x557ed9d5bb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f93180058c0 tx=0x7f931806bf20 comp rx=0 tx=0).stop 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.097+0000 7f93312bd740 1 -- 172.21.15.16:0/372144570 shutdown_connections 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.097+0000 7f93312bd740 1 --2- 172.21.15.16:0/372144570 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x557ed9d67610 0x557ed9d69a40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.097+0000 7f93312bd740 1 --2- 172.21.15.16:0/372144570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d57370 0x557ed9d630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.097+0000 7f93312bd740 1 --2- 172.21.15.16:0/372144570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d522e0 0x557ed9d5c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.097+0000 7f93077fe640 0 client.25294 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.097+0000 7f93312bd740 1 --2- 172.21.15.16:0/372144570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d519a0 0x557ed9d5bb30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.097+0000 7f93312bd740 1 -- 172.21.15.16:0/372144570 >> 172.21.15.16:0/372144570 conn(0x557ed9d4d010 msgr2=0x557ed9d4e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.097+0000 7f93312bd740 1 -- 172.21.15.16:0/372144570 shutdown_connections 2024-09-19T09:28:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.097+0000 7f93312bd740 1 -- 172.21.15.16:0/372144570 wait complete. 2024-09-19T09:28:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.097+0000 7f93312bd740 20 client.25294 trim_cache size 0 max 16384 2024-09-19T09:28:45.104 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9f439ef8', quiesce took 4.7 seconds. Will release after: 15.650000000000002 2024-09-19T09:28:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93312bd740 1 Processor -- start 2024-09-19T09:28:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93312bd740 1 -- start start 2024-09-19T09:28:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93312bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d519a0 0x557ed9d6e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93312bd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d522e0 0x557ed9d6e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93312bd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d57370 0x557ed9d6ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93312bd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93281a8fb0 con 0x557ed9d522e0 2024-09-19T09:28:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93312bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f93281aaaf0 con 0x557ed9d519a0 2024-09-19T09:28:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93312bd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f93281a9680 con 0x557ed9d57370 2024-09-19T09:28:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d522e0 0x557ed9d6e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d522e0 0x557ed9d6e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52998/0 (socket says 172.21.15.16:52998) 2024-09-19T09:28:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93277fe640 1 -- 172.21.15.16:0/2749286320 learned_addr learned my addr 172.21.15.16:0/2749286320 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f932cac9640 1 --2- 172.21.15.16:0/2749286320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d57370 0x557ed9d6ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f9327fff640 1 --2- 172.21.15.16:0/2749286320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d519a0 0x557ed9d6e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93277fe640 1 -- 172.21.15.16:0/2749286320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d57370 msgr2=0x557ed9d6ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93277fe640 1 --2- 172.21.15.16:0/2749286320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d57370 0x557ed9d6ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93277fe640 1 -- 172.21.15.16:0/2749286320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d519a0 msgr2=0x557ed9d6e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93277fe640 1 --2- 172.21.15.16:0/2749286320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d519a0 0x557ed9d6e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93277fe640 1 -- 172.21.15.16:0/2749286320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed9d527c0 con 0x557ed9d522e0 2024-09-19T09:28:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.114+0000 7f93277fe640 1 --2- 172.21.15.16:0/2749286320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d522e0 0x557ed9d6e8d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f931c02a930 tx=0x7f931c02a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a0eb53aef21439 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.115+0000 7f930dffb640 1 -- 172.21.15.16:0/2749286320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c042040 con 0x557ed9d522e0 2024-09-19T09:28:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.115+0000 7f930dffb640 1 -- 172.21.15.16:0/2749286320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f931c043040 con 0x557ed9d522e0 2024-09-19T09:28:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.115+0000 7f930dffb640 1 -- 172.21.15.16:0/2749286320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c037040 con 0x557ed9d522e0 2024-09-19T09:28:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.115+0000 7f93312bd740 1 -- 172.21.15.16:0/2749286320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d522e0 msgr2=0x557ed9d6e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.115+0000 7f93312bd740 1 --2- 172.21.15.16:0/2749286320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d522e0 0x557ed9d6e8d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f931c02a930 tx=0x7f931c02a340 comp rx=0 tx=0).stop 2024-09-19T09:28:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.115+0000 7f93312bd740 1 -- 172.21.15.16:0/2749286320 shutdown_connections 2024-09-19T09:28:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.115+0000 7f93312bd740 1 --2- 172.21.15.16:0/2749286320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d57370 0x557ed9d6ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.115+0000 7f93312bd740 1 --2- 172.21.15.16:0/2749286320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d522e0 0x557ed9d6e8d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.115+0000 7f93312bd740 1 --2- 172.21.15.16:0/2749286320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d519a0 0x557ed9d6e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.115+0000 7f93312bd740 1 -- 172.21.15.16:0/2749286320 >> 172.21.15.16:0/2749286320 conn(0x557ed9d4d010 msgr2=0x557ed9d4c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.115+0000 7f93312bd740 1 -- 172.21.15.16:0/2749286320 shutdown_connections 2024-09-19T09:28:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.115+0000 7f93312bd740 1 -- 172.21.15.16:0/2749286320 wait complete. 2024-09-19T09:28:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f93312bd740 1 -- start start 2024-09-19T09:28:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f93312bd740 1 Processor -- start 2024-09-19T09:28:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f93312bd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d519a0 0x557ed9dce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f93312bd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d522e0 0x557ed9dce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f93312bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d57370 0x557ed9dd5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f93312bd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93281aa420 con 0x557ed9d519a0 2024-09-19T09:28:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f93312bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f93281a9d50 con 0x557ed9d57370 2024-09-19T09:28:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f93312bd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f93281a6f70 con 0x557ed9d522e0 2024-09-19T09:28:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f93312bd740 10 client.0 _ll_register_callbacks cb 0x7f9318018420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f9327fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d519a0 0x557ed9dce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f9327fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d519a0 0x557ed9dce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53012/0 (socket says 172.21.15.16:53012) 2024-09-19T09:28:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f9327fff640 1 -- 172.21.15.16:0/2649558177 learned_addr learned my addr 172.21.15.16:0/2649558177 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f93277fe640 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d522e0 0x557ed9dce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.116+0000 7f932cac9640 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d57370 0x557ed9dd5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f9327fff640 1 -- 172.21.15.16:0/2649558177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d522e0 msgr2=0x557ed9dce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f9327fff640 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d522e0 0x557ed9dce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f9327fff640 1 -- 172.21.15.16:0/2649558177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d57370 msgr2=0x557ed9dd5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f9327fff640 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d57370 0x557ed9dd5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f9327fff640 1 -- 172.21.15.16:0/2649558177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed9dd5e80 con 0x557ed9d519a0 2024-09-19T09:28:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f93277fe640 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d522e0 0x557ed9dce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:28:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f9327fff640 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d519a0 0x557ed9dce2f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9318098c30 tx=0x7f93180cd6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ec2d666f82eb1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f93077fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f93077fe640 1 -- 172.21.15.16:0/2649558177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931809bca0 con 0x557ed9d519a0 2024-09-19T09:28:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f93312bd740 1 -- 172.21.15.16:0/2649558177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f931c043040 con 0x557ed9d519a0 2024-09-19T09:28:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f93077fe640 1 -- 172.21.15.16:0/2649558177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9318098000 con 0x557ed9d519a0 2024-09-19T09:28:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f93077fe640 1 -- 172.21.15.16:0/2649558177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f931809ca40 con 0x557ed9d519a0 2024-09-19T09:28:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f93312bd740 10 client.15744 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f93312bd740 1 -- 172.21.15.16:0/2649558177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ed9dd7200 con 0x557ed9d519a0 2024-09-19T09:28:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f93077fe640 1 -- 172.21.15.16:0/2649558177 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f931c043040 con 0x557ed9d519a0 2024-09-19T09:28:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.117+0000 7f93077fe640 1 -- 172.21.15.16:0/2649558177 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9318098200 con 0x557ed9d519a0 2024-09-19T09:28:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.118+0000 7f93312bd740 10 client.15744 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.118+0000 7f93312bd740 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.118+0000 7f93312bd740 20 client.15744 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.118+0000 7f93312bd740 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x557ed9d67610 0x557ed9dd9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.118+0000 7f93312bd740 4 client.15744 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:28:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.118+0000 7f93312bd740 1 -- 172.21.15.16:0/2649558177 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ed9cbcd70 con 0x557ed9d67610 2024-09-19T09:28:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.118+0000 7f93277fe640 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x557ed9d67610 0x557ed9dd9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.118+0000 7f93277fe640 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x557ed9d67610 0x557ed9dd9bc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b343facf9b142bf server_cookie=4181a3acc63bdf27 in_seq=0 out_seq=0 2024-09-19T09:28:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.118+0000 7f93077fe640 10 client.15744 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93077fe640 1 -- 172.21.15.16:0/2649558177 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ed9cbcd70 con 0x557ed9d67610 2024-09-19T09:28:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93077fe640 10 client.15744 handle_command_reply: tid=0 2024-09-19T09:28:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93312bd740 1 client.15744 shutdown 2024-09-19T09:28:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93312bd740 1 -- 172.21.15.16:0/2649558177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d519a0 msgr2=0x557ed9dce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93312bd740 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d519a0 0x557ed9dce2f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9318098c30 tx=0x7f93180cd6f0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93312bd740 1 -- 172.21.15.16:0/2649558177 shutdown_connections 2024-09-19T09:28:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93312bd740 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x557ed9d67610 0x557ed9dd9bc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93312bd740 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed9d522e0 0x557ed9dce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93077fe640 0 client.15744 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93312bd740 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ed9d57370 0x557ed9dd5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93312bd740 1 --2- 172.21.15.16:0/2649558177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ed9d519a0 0x557ed9dce2f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93312bd740 1 -- 172.21.15.16:0/2649558177 >> 172.21.15.16:0/2649558177 conn(0x557ed9d4d010 msgr2=0x557ed9d4c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93312bd740 1 -- 172.21.15.16:0/2649558177 shutdown_connections 2024-09-19T09:28:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93312bd740 1 -- 172.21.15.16:0/2649558177 wait complete. 2024-09-19T09:28:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.130+0000 7f93312bd740 20 client.15744 trim_cache size 0 max 16384 2024-09-19T09:28:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-19T09:28:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-19T09:28:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-19T09:28:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-19T09:28:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.131+0000 7f932e52c640 1 -- 172.21.15.16:0/1968789694 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f93100a32f0 msgr2=0x7f93100a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.131+0000 7f932e52c640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f93100a32f0 0x7f93100a57b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9320005c40 tx=0x7f9320009500 comp rx=0 tx=0).stop 2024-09-19T09:28:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.131+0000 7f932e52c640 1 -- 172.21.15.16:0/1968789694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9328100a40 msgr2=0x7f9328197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.131+0000 7f932e52c640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9328100a40 0x7f9328197dc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f931800e9a0 tx=0x7f931800ee60 comp rx=0 tx=0).stop 2024-09-19T09:28:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.131+0000 7f932e52c640 1 -- 172.21.15.16:0/1968789694 shutdown_connections 2024-09-19T09:28:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.131+0000 7f932e52c640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f93100a32f0 0x7f93100a57b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.131+0000 7f932e52c640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9328101c30 0x7f9328198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.131+0000 7f932e52c640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93281025f0 0x7f932819f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.131+0000 7f932e52c640 1 --2- 172.21.15.16:0/1968789694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9328100a40 0x7f9328197dc0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.131+0000 7f932e52c640 1 -- 172.21.15.16:0/1968789694 >> 172.21.15.16:0/1968789694 conn(0x7f93280fc1e0 msgr2=0x7f9328104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.131+0000 7f932e52c640 1 -- 172.21.15.16:0/1968789694 shutdown_connections 2024-09-19T09:28:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:45.131+0000 7f932e52c640 1 -- 172.21.15.16:0/1968789694 wait complete. 2024-09-19T09:28:45.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:45 smithi142 ceph-mon[35032]: pgmap v372: 97 pgs: 97 active+clean; 9.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 716 KiB/s wr, 88 op/s 2024-09-19T09:28:45.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:45 smithi097 ceph-mon[34864]: pgmap v372: 97 pgs: 97 active+clean; 9.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 716 KiB/s wr, 88 op/s 2024-09-19T09:28:45.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:45 smithi016 ceph-mon[30979]: pgmap v372: 97 pgs: 97 active+clean; 9.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 716 KiB/s wr, 88 op/s 2024-09-19T09:28:47.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:47 smithi142 ceph-mon[35032]: pgmap v373: 97 pgs: 97 active+clean; 9.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 716 KiB/s wr, 84 op/s 2024-09-19T09:28:47.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:47 smithi097 ceph-mon[34864]: pgmap v373: 97 pgs: 97 active+clean; 9.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 716 KiB/s wr, 84 op/s 2024-09-19T09:28:47.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:47 smithi016 ceph-mon[30979]: pgmap v373: 97 pgs: 97 active+clean; 9.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 716 KiB/s wr, 84 op/s 2024-09-19T09:28:49.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:49 smithi142 ceph-mon[35032]: pgmap v374: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-09-19T09:28:49.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:49 smithi097 ceph-mon[34864]: pgmap v374: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-09-19T09:28:49.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:49 smithi016 ceph-mon[30979]: pgmap v374: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-09-19T09:28:51.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:51 smithi142 ceph-mon[35032]: pgmap v375: 97 pgs: 97 active+clean; 9.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.4 MiB/s wr, 1.78k op/s 2024-09-19T09:28:51.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:51 smithi097 ceph-mon[34864]: pgmap v375: 97 pgs: 97 active+clean; 9.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.4 MiB/s wr, 1.78k op/s 2024-09-19T09:28:51.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:51 smithi016 ceph-mon[30979]: pgmap v375: 97 pgs: 97 active+clean; 9.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.4 MiB/s wr, 1.78k op/s 2024-09-19T09:28:52.001 INFO:tasks.workunit.client.0.smithi016.stdout:.......... 2024-09-19T09:28:52.001 INFO:tasks.workunit.client.0.smithi016.stdout:do_o_direct_reads: SUCCESS 2024-09-19T09:28:52.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:28:53.079 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:53.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:53 smithi142 ceph-mon[35032]: pgmap v376: 97 pgs: 97 active+clean; 9.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.4 MiB/s wr, 1.78k op/s 2024-09-19T09:28:53.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:53 smithi097 ceph-mon[34864]: pgmap v376: 97 pgs: 97 active+clean; 9.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.4 MiB/s wr, 1.78k op/s 2024-09-19T09:28:53.440 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:53 smithi016 ceph-mon[30979]: pgmap v376: 97 pgs: 97 active+clean; 9.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.4 MiB/s wr, 1.78k op/s 2024-09-19T09:28:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.522+0000 7f0699619640 1 -- 172.21.15.16:0/469563406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f069410c170 msgr2=0x7f069410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.522+0000 7f0699619640 1 --2- 172.21.15.16:0/469563406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f069410c170 0x7f069410e610 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f068c009ca0 tx=0x7f068c02a950 comp rx=0 tx=0).stop 2024-09-19T09:28:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.522+0000 7f0699619640 1 -- 172.21.15.16:0/469563406 shutdown_connections 2024-09-19T09:28:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.522+0000 7f0699619640 1 --2- 172.21.15.16:0/469563406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0694103660 0x7f069410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.522+0000 7f0699619640 1 --2- 172.21.15.16:0/469563406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f069410c170 0x7f069410e610 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.522+0000 7f0699619640 1 --2- 172.21.15.16:0/469563406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0694102d40 0x7f0694103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.522+0000 7f0699619640 1 -- 172.21.15.16:0/469563406 >> 172.21.15.16:0/469563406 conn(0x7f06940fc7a0 msgr2=0x7f06940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.522+0000 7f0699619640 1 -- 172.21.15.16:0/469563406 shutdown_connections 2024-09-19T09:28:53.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.522+0000 7f0699619640 1 -- 172.21.15.16:0/469563406 wait complete. 2024-09-19T09:28:53.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.523+0000 7f0699619640 1 Processor -- start 2024-09-19T09:28:53.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.523+0000 7f0699619640 1 -- start start 2024-09-19T09:28:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.523+0000 7f0699619640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0694102d40 0x7f069419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.523+0000 7f0699619640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0694103660 0x7f069419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.523+0000 7f0699619640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f069410c170 0x7f069419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.523+0000 7f0699619640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0694111fb0 con 0x7f0694102d40 2024-09-19T09:28:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.523+0000 7f0699619640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0694111e30 con 0x7f0694103660 2024-09-19T09:28:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.523+0000 7f0699619640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0694112130 con 0x7f069410c170 2024-09-19T09:28:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.523+0000 7f0692ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0694102d40 0x7f069419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.523+0000 7f0692ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0694102d40 0x7f069419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33724/0 (socket says 172.21.15.16:33724) 2024-09-19T09:28:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.523+0000 7f0692ffd640 1 -- 172.21.15.16:0/842650996 learned_addr learned my addr 172.21.15.16:0/842650996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.524+0000 7f0692ffd640 1 -- 172.21.15.16:0/842650996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f069410c170 msgr2=0x7f069419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:53.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.524+0000 7f0692ffd640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f069410c170 0x7f069419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.524+0000 7f0692ffd640 1 -- 172.21.15.16:0/842650996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0694103660 msgr2=0x7f069419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:53.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.524+0000 7f068bfff640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0694103660 0x7f069419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.524+0000 7f0692ffd640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0694103660 0x7f069419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.524+0000 7f0692ffd640 1 -- 172.21.15.16:0/842650996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069419f5d0 con 0x7f0694102d40 2024-09-19T09:28:53.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.524+0000 7f068bfff640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0694103660 0x7f069419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:53.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.524+0000 7f0692ffd640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0694102d40 0x7f069419a620 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f068400d950 tx=0x7f068400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:53.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.524+0000 7f0690ff9640 1 -- 172.21.15.16:0/842650996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0684017070 con 0x7f0694102d40 2024-09-19T09:28:53.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.524+0000 7f0699619640 1 -- 172.21.15.16:0/842650996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069419f8c0 con 0x7f0694102d40 2024-09-19T09:28:53.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.524+0000 7f0690ff9640 1 -- 172.21.15.16:0/842650996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0684009c30 con 0x7f0694102d40 2024-09-19T09:28:53.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.524+0000 7f0690ff9640 1 -- 172.21.15.16:0/842650996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068400f6b0 con 0x7f0694102d40 2024-09-19T09:28:53.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.524+0000 7f0699619640 1 -- 172.21.15.16:0/842650996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06941a71a0 con 0x7f0694102d40 2024-09-19T09:28:53.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.527+0000 7f0690ff9640 1 -- 172.21.15.16:0/842650996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f06840109d0 con 0x7f0694102d40 2024-09-19T09:28:53.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.527+0000 7f069b9f2740 1 Processor -- start 2024-09-19T09:28:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.527+0000 7f069b9f2740 1 -- start start 2024-09-19T09:28:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.527+0000 7f069b9f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5f4a0 0x5570faf5f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.527+0000 7f069b9f2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5fdc0 0x5570faf6a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.527+0000 7f069b9f2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf6a680 0x5570faf6ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.527+0000 7f069b9f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570faec52e0 con 0x5570faf5f4a0 2024-09-19T09:28:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.527+0000 7f069b9f2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5570fae6cce0 con 0x5570faf6a680 2024-09-19T09:28:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.527+0000 7f069b9f2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5570faec12a0 con 0x5570faf5fdc0 2024-09-19T09:28:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.527+0000 7f06937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5f4a0 0x5570faf5f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.527+0000 7f06937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5f4a0 0x5570faf5f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33740/0 (socket says 172.21.15.16:33740) 2024-09-19T09:28:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.527+0000 7f06937fe640 1 -- 172.21.15.16:0/1879021049 learned_addr learned my addr 172.21.15.16:0/1879021049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f0692ffd640 1 --2- 172.21.15.16:0/1879021049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5fdc0 0x5570faf6a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f068bfff640 1 --2- 172.21.15.16:0/1879021049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf6a680 0x5570faf6ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f06937fe640 1 -- 172.21.15.16:0/1879021049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5fdc0 msgr2=0x5570faf6a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f06937fe640 1 --2- 172.21.15.16:0/1879021049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5fdc0 0x5570faf6a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f06937fe640 1 -- 172.21.15.16:0/1879021049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf6a680 msgr2=0x5570faf6ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f06937fe640 1 --2- 172.21.15.16:0/1879021049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf6a680 0x5570faf6ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f06937fe640 1 -- 172.21.15.16:0/1879021049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570fae51440 con 0x5570faf5f4a0 2024-09-19T09:28:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f0690ff9640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f06800a3240 0x7f06800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f06937fe640 1 --2- 172.21.15.16:0/1879021049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5f4a0 0x5570faf5f880 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f068c005e30 tx=0x7f068c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d452621388244fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f0692ffd640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f06800a3240 0x7f06800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f0690ff9640 1 -- 172.21.15.16:0/842650996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f06840cf650 con 0x7f0694102d40 2024-09-19T09:28:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f06897fa640 1 -- 172.21.15.16:0/1879021049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c034040 con 0x5570faf5f4a0 2024-09-19T09:28:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f06897fa640 1 -- 172.21.15.16:0/1879021049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f068c02fb40 con 0x5570faf5f4a0 2024-09-19T09:28:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.528+0000 7f06897fa640 1 -- 172.21.15.16:0/1879021049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c004430 con 0x5570faf5f4a0 2024-09-19T09:28:53.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.529+0000 7f069b9f2740 1 -- 172.21.15.16:0/1879021049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5f4a0 msgr2=0x5570faf5f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.529+0000 7f0692ffd640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f06800a3240 0x7f06800a5700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0684009610 tx=0x7f0684005880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.529+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1879021049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5f4a0 0x5570faf5f880 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f068c005e30 tx=0x7f068c005fd0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.529+0000 7f069b9f2740 1 -- 172.21.15.16:0/1879021049 shutdown_connections 2024-09-19T09:28:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.529+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1879021049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5fdc0 0x5570faf6a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.529+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1879021049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf6a680 0x5570faf6ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.529+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1879021049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5f4a0 0x5570faf5f880 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.529+0000 7f069b9f2740 1 -- 172.21.15.16:0/1879021049 >> 172.21.15.16:0/1879021049 conn(0x5570faf5ba30 msgr2=0x5570faf5be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.529+0000 7f069b9f2740 1 -- 172.21.15.16:0/1879021049 shutdown_connections 2024-09-19T09:28:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.529+0000 7f069b9f2740 1 -- 172.21.15.16:0/1879021049 wait complete. 2024-09-19T09:28:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f069b9f2740 1 -- start start 2024-09-19T09:28:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f069b9f2740 1 Processor -- start 2024-09-19T09:28:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f069b9f2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 0x5570faf69fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f069b9f2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5fdc0 0x5570faf67250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f069b9f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf6a680 0x5570faf67790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f069b9f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570faf6d250 con 0x5570faf6a680 2024-09-19T09:28:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f069b9f2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5570fae80050 con 0x5570faf5fdc0 2024-09-19T09:28:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f069b9f2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5570faf6d3d0 con 0x5570faf5f4a0 2024-09-19T09:28:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f069b9f2740 10 client.0 _ll_register_callbacks cb 0x5570faec0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f06937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf6a680 0x5570faf67790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f06937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf6a680 0x5570faf67790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33750/0 (socket says 172.21.15.16:33750) 2024-09-19T09:28:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f06937fe640 1 -- 172.21.15.16:0/1352235393 learned_addr learned my addr 172.21.15.16:0/1352235393 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f06937fe640 1 --2- 172.21.15.16:0/1352235393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 0x5570faf69fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f068bfff640 1 --2- 172.21.15.16:0/1352235393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5fdc0 0x5570faf67250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f06937fe640 1 -- 172.21.15.16:0/1352235393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 msgr2=0x5570faf69fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f06937fe640 1 --2- 172.21.15.16:0/1352235393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 0x5570faf69fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f06937fe640 1 -- 172.21.15.16:0/1352235393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5fdc0 msgr2=0x5570faf67250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f06937fe640 1 --2- 172.21.15.16:0/1352235393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5fdc0 0x5570faf67250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.530+0000 7f06937fe640 1 -- 172.21.15.16:0/1352235393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570fac363c0 con 0x5570faf6a680 2024-09-19T09:28:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f06937fe640 1 --2- 172.21.15.16:0/1352235393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf6a680 0x5570faf67790 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f068c003ba0 tx=0x7f068c03c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25e9426952c2d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f067a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f067a7fc640 1 -- 172.21.15.16:0/1352235393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c057060 con 0x5570faf6a680 2024-09-19T09:28:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f067a7fc640 1 -- 172.21.15.16:0/1352235393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f068c0041b0 con 0x5570faf6a680 2024-09-19T09:28:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f069b9f2740 1 -- 172.21.15.16:0/1352235393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f068c02fb40 con 0x5570faf6a680 2024-09-19T09:28:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f067a7fc640 1 -- 172.21.15.16:0/1352235393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f068c034030 con 0x5570faf6a680 2024-09-19T09:28:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f069b9f2740 10 client.15756 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f069b9f2740 1 -- 172.21.15.16:0/1352235393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570faf78bd0 con 0x5570faf6a680 2024-09-19T09:28:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f067a7fc640 1 -- 172.21.15.16:0/1352235393 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f068c02fb40 con 0x5570faf6a680 2024-09-19T09:28:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f067a7fc640 1 -- 172.21.15.16:0/1352235393 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f068c039800 con 0x5570faf6a680 2024-09-19T09:28:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f069b9f2740 10 client.15756 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f069b9f2740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f069b9f2740 20 client.15756 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1352235393 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5570faf79f20 0x5570faf7c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f069b9f2740 4 client.15756 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.531+0000 7f069b9f2740 1 -- 172.21.15.16:0/1352235393 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570faed2850 con 0x5570faf79f20 2024-09-19T09:28:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.532+0000 7f068bfff640 1 --2- 172.21.15.16:0/1352235393 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5570faf79f20 0x5570faf7c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.532+0000 7f068bfff640 1 --2- 172.21.15.16:0/1352235393 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5570faf79f20 0x5570faf7c310 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967a49de93289bb6 server_cookie=4419b778d852fec8 in_seq=0 out_seq=0 2024-09-19T09:28:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.532+0000 7f067a7fc640 10 client.15756 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f067a7fc640 1 -- 172.21.15.16:0/1352235393 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570faed2850 con 0x5570faf79f20 2024-09-19T09:28:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f067a7fc640 10 client.15756 handle_command_reply: tid=0 2024-09-19T09:28:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f069b9f2740 1 client.15756 shutdown 2024-09-19T09:28:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f069b9f2740 1 -- 172.21.15.16:0/1352235393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf6a680 msgr2=0x5570faf67790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1352235393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf6a680 0x5570faf67790 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f068c003ba0 tx=0x7f068c03c040 comp rx=0 tx=0).stop 2024-09-19T09:28:53.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f069b9f2740 1 -- 172.21.15.16:0/1352235393 shutdown_connections 2024-09-19T09:28:53.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1352235393 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5570faf79f20 0x5570faf7c310 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1352235393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf6a680 0x5570faf67790 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f067a7fc640 0 client.15756 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:53.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1352235393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5fdc0 0x5570faf67250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1352235393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 0x5570faf69fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f069b9f2740 1 -- 172.21.15.16:0/1352235393 >> 172.21.15.16:0/1352235393 conn(0x5570faf5ba30 msgr2=0x5570faf61710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:53.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f069b9f2740 1 -- 172.21.15.16:0/1352235393 shutdown_connections 2024-09-19T09:28:53.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f069b9f2740 1 -- 172.21.15.16:0/1352235393 wait complete. 2024-09-19T09:28:53.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.533+0000 7f069b9f2740 20 client.15756 trim_cache size 0 max 16384 2024-09-19T09:28:53.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.550+0000 7f069b9f2740 1 Processor -- start 2024-09-19T09:28:53.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.550+0000 7f069b9f2740 1 -- start start 2024-09-19T09:28:53.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f069b9f2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5ba30 0x5570faf691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f069b9f2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 0x5570faf7f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f069b9f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5fdc0 0x5570faf7f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f069b9f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06941af8d0 con 0x5570faf5fdc0 2024-09-19T09:28:53.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f069b9f2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f06941b1410 con 0x5570faf5ba30 2024-09-19T09:28:53.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f069b9f2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f06941affa0 con 0x5570faf5f4a0 2024-09-19T09:28:53.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f06937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5fdc0 0x5570faf7f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f06937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5fdc0 0x5570faf7f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33762/0 (socket says 172.21.15.16:33762) 2024-09-19T09:28:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f06937fe640 1 -- 172.21.15.16:0/1169254193 learned_addr learned my addr 172.21.15.16:0/1169254193 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f06937fe640 1 --2- 172.21.15.16:0/1169254193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5ba30 0x5570faf691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f068bfff640 1 --2- 172.21.15.16:0/1169254193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 0x5570faf7f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f06937fe640 1 -- 172.21.15.16:0/1169254193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 msgr2=0x5570faf7f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f06937fe640 1 --2- 172.21.15.16:0/1169254193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 0x5570faf7f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f06937fe640 1 -- 172.21.15.16:0/1169254193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5ba30 msgr2=0x5570faf691c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f06937fe640 1 --2- 172.21.15.16:0/1169254193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5ba30 0x5570faf691c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f06937fe640 1 -- 172.21.15.16:0/1169254193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570faf5fbf0 con 0x5570faf5fdc0 2024-09-19T09:28:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.551+0000 7f068bfff640 1 --2- 172.21.15.16:0/1169254193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 0x5570faf7f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.552+0000 7f06937fe640 1 --2- 172.21.15.16:0/1169254193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5fdc0 0x5570faf7f880 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f068c006010 tx=0x7f068c02ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ff00f25935c998 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.552+0000 7f0688ff9640 1 -- 172.21.15.16:0/1169254193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c061040 con 0x5570faf5fdc0 2024-09-19T09:28:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.552+0000 7f0688ff9640 1 -- 172.21.15.16:0/1169254193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f068c049040 con 0x5570faf5fdc0 2024-09-19T09:28:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.552+0000 7f0688ff9640 1 -- 172.21.15.16:0/1169254193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c057040 con 0x5570faf5fdc0 2024-09-19T09:28:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.552+0000 7f069b9f2740 1 -- 172.21.15.16:0/1169254193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5fdc0 msgr2=0x5570faf7f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.552+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1169254193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5fdc0 0x5570faf7f880 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f068c006010 tx=0x7f068c02ff00 comp rx=0 tx=0).stop 2024-09-19T09:28:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.552+0000 7f069b9f2740 1 -- 172.21.15.16:0/1169254193 shutdown_connections 2024-09-19T09:28:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.552+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1169254193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 0x5570faf7f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.552+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1169254193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5fdc0 0x5570faf7f880 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.552+0000 7f069b9f2740 1 --2- 172.21.15.16:0/1169254193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5ba30 0x5570faf691c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.552+0000 7f069b9f2740 1 -- 172.21.15.16:0/1169254193 >> 172.21.15.16:0/1169254193 conn(0x5570faf678f0 msgr2=0x5570faf68ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f069b9f2740 1 -- 172.21.15.16:0/1169254193 shutdown_connections 2024-09-19T09:28:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f069b9f2740 1 -- 172.21.15.16:0/1169254193 wait complete. 2024-09-19T09:28:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f069b9f2740 1 -- start start 2024-09-19T09:28:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f069b9f2740 1 Processor -- start 2024-09-19T09:28:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f069b9f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5ba30 0x5570fafdbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f069b9f2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 0x5570fafdc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f069b9f2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5fdc0 0x5570faf7eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f069b9f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06941b0d40 con 0x5570faf5ba30 2024-09-19T09:28:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f069b9f2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f06941b0670 con 0x5570faf5fdc0 2024-09-19T09:28:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f069b9f2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f06941ad890 con 0x5570faf5f4a0 2024-09-19T09:28:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f069b9f2740 10 client.0 _ll_register_callbacks cb 0x7f068c009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f06937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5ba30 0x5570fafdbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f06937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5ba30 0x5570fafdbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33778/0 (socket says 172.21.15.16:33778) 2024-09-19T09:28:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f06937fe640 1 -- 172.21.15.16:0/3690338817 learned_addr learned my addr 172.21.15.16:0/3690338817 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.553+0000 7f068bfff640 1 --2- 172.21.15.16:0/3690338817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 0x5570fafdc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f06937fe640 1 -- 172.21.15.16:0/3690338817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 msgr2=0x5570fafdc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f06937fe640 1 --2- 172.21.15.16:0/3690338817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 0x5570fafdc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f06937fe640 1 -- 172.21.15.16:0/3690338817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5fdc0 msgr2=0x5570faf7eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f06937fe640 1 --2- 172.21.15.16:0/3690338817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5fdc0 0x5570faf7eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f06937fe640 1 -- 172.21.15.16:0/3690338817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570faf7f2d0 con 0x5570faf5ba30 2024-09-19T09:28:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f06937fe640 1 --2- 172.21.15.16:0/3690338817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5ba30 0x5570fafdbc20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f068c02ff70 tx=0x7f068c030440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb3c2484862a825 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f067a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f067a7fc640 1 -- 172.21.15.16:0/3690338817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c057030 con 0x5570faf5ba30 2024-09-19T09:28:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f067a7fc640 1 -- 172.21.15.16:0/3690338817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f068c0399a0 con 0x5570faf5ba30 2024-09-19T09:28:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f069b9f2740 1 -- 172.21.15.16:0/3690338817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f068c049040 con 0x5570faf5ba30 2024-09-19T09:28:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f067a7fc640 1 -- 172.21.15.16:0/3690338817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f068c061040 con 0x5570faf5ba30 2024-09-19T09:28:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f069b9f2740 10 client.15762 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f069b9f2740 1 -- 172.21.15.16:0/3690338817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570fafe44a0 con 0x5570faf5ba30 2024-09-19T09:28:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f067a7fc640 1 -- 172.21.15.16:0/3690338817 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f068c049040 con 0x5570faf5ba30 2024-09-19T09:28:53.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f067a7fc640 1 -- 172.21.15.16:0/3690338817 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f068c004300 con 0x5570faf5ba30 2024-09-19T09:28:53.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.554+0000 7f069b9f2740 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:53.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.555+0000 7f069b9f2740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:53.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.555+0000 7f069b9f2740 20 client.15762 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:53.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.555+0000 7f069b9f2740 1 --2- 172.21.15.16:0/3690338817 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5570faf6a680 0x5570fafe6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:53.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.555+0000 7f069b9f2740 4 client.15762 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:28:53.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.555+0000 7f069b9f2740 1 -- 172.21.15.16:0/3690338817 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5570faeccd60 con 0x5570faf6a680 2024-09-19T09:28:53.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.555+0000 7f068bfff640 1 --2- 172.21.15.16:0/3690338817 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5570faf6a680 0x5570fafe6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:53.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.555+0000 7f068bfff640 1 --2- 172.21.15.16:0/3690338817 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5570faf6a680 0x5570fafe6930 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=798e1e01dc717782 server_cookie=d19572cc0ef4773 in_seq=0 out_seq=0 2024-09-19T09:28:53.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.555+0000 7f067a7fc640 10 client.15762 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:53.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.555+0000 7f067a7fc640 1 -- 172.21.15.16:0/3690338817 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5570faeccd60 con 0x5570faf6a680 2024-09-19T09:28:53.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.555+0000 7f067a7fc640 10 client.15762 handle_command_reply: tid=0 2024-09-19T09:28:53.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.555+0000 7f069b9f2740 1 client.15762 shutdown 2024-09-19T09:28:53.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.555+0000 7f069b9f2740 1 -- 172.21.15.16:0/3690338817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5ba30 msgr2=0x5570fafdbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.555+0000 7f069b9f2740 1 --2- 172.21.15.16:0/3690338817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5ba30 0x5570fafdbc20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f068c02ff70 tx=0x7f068c030440 comp rx=0 tx=0).stop 2024-09-19T09:28:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f069b9f2740 1 -- 172.21.15.16:0/3690338817 shutdown_connections 2024-09-19T09:28:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f069b9f2740 1 --2- 172.21.15.16:0/3690338817 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5570faf6a680 0x5570fafe6930 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f069b9f2740 1 --2- 172.21.15.16:0/3690338817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570faf5f4a0 0x5570fafdc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f069b9f2740 1 --2- 172.21.15.16:0/3690338817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5570faf5fdc0 0x5570faf7eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f069b9f2740 1 --2- 172.21.15.16:0/3690338817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570faf5ba30 0x5570fafdbc20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f069b9f2740 1 -- 172.21.15.16:0/3690338817 >> 172.21.15.16:0/3690338817 conn(0x5570faf678f0 msgr2=0x5570faf692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f067a7fc640 0 client.15762 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f069b9f2740 1 -- 172.21.15.16:0/3690338817 shutdown_connections 2024-09-19T09:28:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f069b9f2740 1 -- 172.21.15.16:0/3690338817 wait complete. 2024-09-19T09:28:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f069b9f2740 20 client.15762 trim_cache size 0 max 16384 2024-09-19T09:28:53.573 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:28:53.573 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-19T09:28:53.573 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-19T09:28:53.573 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:28:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f0699619640 1 -- 172.21.15.16:0/842650996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f06800a3240 msgr2=0x7f06800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f0699619640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f06800a3240 0x7f06800a5700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0684009610 tx=0x7f0684005880 comp rx=0 tx=0).stop 2024-09-19T09:28:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f0699619640 1 -- 172.21.15.16:0/842650996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0694102d40 msgr2=0x7f069419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f0699619640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0694102d40 0x7f069419a620 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f068400d950 tx=0x7f068400de10 comp rx=0 tx=0).stop 2024-09-19T09:28:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f0699619640 1 -- 172.21.15.16:0/842650996 shutdown_connections 2024-09-19T09:28:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f0699619640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f06800a3240 0x7f06800a5700 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f0699619640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f069410c170 0x7f069419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f0699619640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0694103660 0x7f069419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f0699619640 1 --2- 172.21.15.16:0/842650996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0694102d40 0x7f069419a620 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.556+0000 7f0699619640 1 -- 172.21.15.16:0/842650996 >> 172.21.15.16:0/842650996 conn(0x7f06940fc7a0 msgr2=0x7f06940fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:53.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.557+0000 7f0699619640 1 -- 172.21.15.16:0/842650996 shutdown_connections 2024-09-19T09:28:53.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:53.557+0000 7f0699619640 1 -- 172.21.15.16:0/842650996 wait complete. 2024-09-19T09:28:54.344 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:28:54.345 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:28:54.609 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.071+0000 7f4549f99640 1 -- 172.21.15.16:0/1038706482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4544106f10 msgr2=0x7f454410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.071+0000 7f4549f99640 1 --2- 172.21.15.16:0/1038706482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4544106f10 0x7f454410ea90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f453c009ca0 tx=0x7f453c02a8c0 comp rx=0 tx=0).stop 2024-09-19T09:28:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.071+0000 7f4549f99640 1 -- 172.21.15.16:0/1038706482 shutdown_connections 2024-09-19T09:28:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.071+0000 7f4549f99640 1 --2- 172.21.15.16:0/1038706482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4544102790 0x7f45441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.071+0000 7f4549f99640 1 --2- 172.21.15.16:0/1038706482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4544106f10 0x7f454410ea90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.071+0000 7f4549f99640 1 --2- 172.21.15.16:0/1038706482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4544101de0 0x7f45441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.071+0000 7f4549f99640 1 -- 172.21.15.16:0/1038706482 >> 172.21.15.16:0/1038706482 conn(0x7f45440fd8c0 msgr2=0x7f45440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.071+0000 7f4549f99640 1 -- 172.21.15.16:0/1038706482 shutdown_connections 2024-09-19T09:28:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.071+0000 7f4549f99640 1 -- 172.21.15.16:0/1038706482 wait complete. 2024-09-19T09:28:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.072+0000 7f4549f99640 1 Processor -- start 2024-09-19T09:28:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.072+0000 7f4549f99640 1 -- start start 2024-09-19T09:28:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f4549f99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4544101de0 0x7f4544198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f4549f99640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4544102790 0x7f4544198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f4549f99640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4544106f10 0x7f454419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f4549f99640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4544111fc0 con 0x7f4544101de0 2024-09-19T09:28:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f4549f99640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4544111e40 con 0x7f4544106f10 2024-09-19T09:28:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f4549f99640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4544112140 con 0x7f4544102790 2024-09-19T09:28:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f45437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4544101de0 0x7f4544198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f45437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4544101de0 0x7f4544198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33786/0 (socket says 172.21.15.16:33786) 2024-09-19T09:28:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f45437fe640 1 -- 172.21.15.16:0/1521368105 learned_addr learned my addr 172.21.15.16:0/1521368105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f4543fff640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4544106f10 0x7f454419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f4542ffd640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4544102790 0x7f4544198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f45437fe640 1 -- 172.21.15.16:0/1521368105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4544102790 msgr2=0x7f4544198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f45437fe640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4544102790 0x7f4544198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f45437fe640 1 -- 172.21.15.16:0/1521368105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4544106f10 msgr2=0x7f454419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f45437fe640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4544106f10 0x7f454419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f45437fe640 1 -- 172.21.15.16:0/1521368105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454419d2b0 con 0x7f4544101de0 2024-09-19T09:28:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f4542ffd640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4544102790 0x7f4544198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.073+0000 7f45437fe640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4544101de0 0x7f4544198300 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f453400c920 tx=0x7f453400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.074+0000 7f4540ff9640 1 -- 172.21.15.16:0/1521368105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534007c70 con 0x7f4544101de0 2024-09-19T09:28:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.074+0000 7f4540ff9640 1 -- 172.21.15.16:0/1521368105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4534007e10 con 0x7f4544101de0 2024-09-19T09:28:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.074+0000 7f4549f99640 1 -- 172.21.15.16:0/1521368105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454419d5a0 con 0x7f4544101de0 2024-09-19T09:28:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.074+0000 7f4540ff9640 1 -- 172.21.15.16:0/1521368105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534018890 con 0x7f4544101de0 2024-09-19T09:28:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.074+0000 7f4549f99640 1 -- 172.21.15.16:0/1521368105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45441a4e80 con 0x7f4544101de0 2024-09-19T09:28:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.075+0000 7f4549f99640 1 -- 172.21.15.16:0/1521368105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45441021c0 con 0x7f4544101de0 2024-09-19T09:28:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.076+0000 7f4540ff9640 1 -- 172.21.15.16:0/1521368105 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4534018a70 con 0x7f4544101de0 2024-09-19T09:28:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.077+0000 7f4540ff9640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f452c0a32c0 0x7f452c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.077+0000 7f4542ffd640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f452c0a32c0 0x7f452c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.077+0000 7f4540ff9640 1 -- 172.21.15.16:0/1521368105 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f45340cfa00 con 0x7f4544101de0 2024-09-19T09:28:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.077+0000 7f4542ffd640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f452c0a32c0 0x7f452c0a5780 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4538005e10 tx=0x7f4538005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.081+0000 7f4540ff9640 1 -- 172.21.15.16:0/1521368105 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4534014030 con 0x7f4544101de0 2024-09-19T09:28:55.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:55 smithi016 ceph-mon[30979]: pgmap v377: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.11k op/s 2024-09-19T09:28:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.294+0000 7f4549f99640 1 -- 172.21.15.16:0/1521368105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45441a5180 con 0x7f4544101de0 2024-09-19T09:28:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.295+0000 7f4540ff9640 1 -- 172.21.15.16:0/1521368105 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f453409c720 con 0x7f4544101de0 2024-09-19T09:28:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:28:55.297 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:28:55.297 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:28:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.297+0000 7f4549f99640 1 -- 172.21.15.16:0/1521368105 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f452c0a32c0 msgr2=0x7f452c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:55.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.297+0000 7f4549f99640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f452c0a32c0 0x7f452c0a5780 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4538005e10 tx=0x7f4538005da0 comp rx=0 tx=0).stop 2024-09-19T09:28:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.297+0000 7f4549f99640 1 -- 172.21.15.16:0/1521368105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4544101de0 msgr2=0x7f4544198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.297+0000 7f4549f99640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4544101de0 0x7f4544198300 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f453400c920 tx=0x7f453400cde0 comp rx=0 tx=0).stop 2024-09-19T09:28:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.297+0000 7f4549f99640 1 -- 172.21.15.16:0/1521368105 shutdown_connections 2024-09-19T09:28:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.297+0000 7f4549f99640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f452c0a32c0 0x7f452c0a5780 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.297+0000 7f4549f99640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4544102790 0x7f4544198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.297+0000 7f4549f99640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4544106f10 0x7f454419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.297+0000 7f4549f99640 1 --2- 172.21.15.16:0/1521368105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4544101de0 0x7f4544198300 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.297+0000 7f4549f99640 1 -- 172.21.15.16:0/1521368105 >> 172.21.15.16:0/1521368105 conn(0x7f45440fd8c0 msgr2=0x7f45440fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.297+0000 7f4549f99640 1 -- 172.21.15.16:0/1521368105 shutdown_connections 2024-09-19T09:28:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:55.297+0000 7f4549f99640 1 -- 172.21.15.16:0/1521368105 wait complete. 2024-09-19T09:28:55.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:55 smithi142 ceph-mon[35032]: pgmap v377: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.11k op/s 2024-09-19T09:28:55.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:55 smithi097 ceph-mon[34864]: pgmap v377: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 3.11k op/s 2024-09-19T09:28:55.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:28:56.189 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:56.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:56 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1521368105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:56.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:56 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1521368105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:56.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:56 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1521368105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:28:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.664+0000 7f71bc71d640 1 -- 172.21.15.16:0/3553680101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4101de0 msgr2=0x7f71b41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.664+0000 7f71bc71d640 1 --2- 172.21.15.16:0/3553680101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4101de0 0x7f71b41021c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009900 tx=0x7f71b002a410 comp rx=0 tx=0).stop 2024-09-19T09:28:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.664+0000 7f71bc71d640 1 -- 172.21.15.16:0/3553680101 shutdown_connections 2024-09-19T09:28:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.664+0000 7f71bc71d640 1 --2- 172.21.15.16:0/3553680101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b4106f10 0x7f71b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.664+0000 7f71bc71d640 1 --2- 172.21.15.16:0/3553680101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71b4102790 0x7f71b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.664+0000 7f71bc71d640 1 --2- 172.21.15.16:0/3553680101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4101de0 0x7f71b41021c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.664+0000 7f71bc71d640 1 -- 172.21.15.16:0/3553680101 >> 172.21.15.16:0/3553680101 conn(0x7f71b40fd8c0 msgr2=0x7f71b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.664+0000 7f71bc71d640 1 -- 172.21.15.16:0/3553680101 shutdown_connections 2024-09-19T09:28:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.664+0000 7f71bc71d640 1 -- 172.21.15.16:0/3553680101 wait complete. 2024-09-19T09:28:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71bc71d640 1 Processor -- start 2024-09-19T09:28:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71bc71d640 1 -- start start 2024-09-19T09:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71bc71d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4101de0 0x7f71b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71bc71d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b4102790 0x7f71b4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71bc71d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71b4106f10 0x7f71b4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71bc71d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4111f70 con 0x7f71b4101de0 2024-09-19T09:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71bc71d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4111df0 con 0x7f71b4106f10 2024-09-19T09:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71bc71d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41120f0 con 0x7f71b4102790 2024-09-19T09:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71ba4b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4101de0 0x7f71b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71ba4b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4101de0 0x7f71b40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40678/0 (socket says 172.21.15.16:40678) 2024-09-19T09:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71ba4b9640 1 -- 172.21.15.16:0/2395966212 learned_addr learned my addr 172.21.15.16:0/2395966212 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71b9cb8640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b4102790 0x7f71b4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71bacba640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71b4106f10 0x7f71b4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71ba4b9640 1 -- 172.21.15.16:0/2395966212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b4102790 msgr2=0x7f71b4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71ba4b9640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b4102790 0x7f71b4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71ba4b9640 1 -- 172.21.15.16:0/2395966212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71b4106f10 msgr2=0x7f71b4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.665+0000 7f71ba4b9640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71b4106f10 0x7f71b4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.666+0000 7f71ba4b9640 1 -- 172.21.15.16:0/2395966212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b41a6890 con 0x7f71b4101de0 2024-09-19T09:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.666+0000 7f71bacba640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71b4106f10 0x7f71b4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.666+0000 7f71b9cb8640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b4102790 0x7f71b4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.666+0000 7f71ba4b9640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4101de0 0x7f71b40744f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f71b0006010 tx=0x7f71b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.666+0000 7f71ab7fe640 1 -- 172.21.15.16:0/2395966212 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b003f030 con 0x7f71b4101de0 2024-09-19T09:28:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.666+0000 7f71ab7fe640 1 -- 172.21.15.16:0/2395966212 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b0003ce0 con 0x7f71b4101de0 2024-09-19T09:28:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.666+0000 7f71bc71d640 1 -- 172.21.15.16:0/2395966212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b41a6b20 con 0x7f71b4101de0 2024-09-19T09:28:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.666+0000 7f71ab7fe640 1 -- 172.21.15.16:0/2395966212 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0035070 con 0x7f71b4101de0 2024-09-19T09:28:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.666+0000 7f71bc71d640 1 -- 172.21.15.16:0/2395966212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b41a7000 con 0x7f71b4101de0 2024-09-19T09:28:56.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.668+0000 7f71ab7fe640 1 -- 172.21.15.16:0/2395966212 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f71b0042030 con 0x7f71b4101de0 2024-09-19T09:28:56.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.668+0000 7f71beaf6740 1 Processor -- start 2024-09-19T09:28:56.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.669+0000 7f71beaf6740 1 -- start start 2024-09-19T09:28:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.669+0000 7f71beaf6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 0x55be2018f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.669+0000 7f71beaf6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 0x55be2018fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.669+0000 7f71beaf6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be2019c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.669+0000 7f71beaf6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be200f52f0 con 0x55be20190b40 2024-09-19T09:28:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.669+0000 7f71beaf6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55be2009ccf0 con 0x55be2019a370 2024-09-19T09:28:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.669+0000 7f71beaf6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55be200f12b0 con 0x55be2018f5d0 2024-09-19T09:28:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.669+0000 7f71bacba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 0x55be2018f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.669+0000 7f71bacba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 0x55be2018f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40682/0 (socket says 172.21.15.16:40682) 2024-09-19T09:28:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.669+0000 7f71bacba640 1 -- 172.21.15.16:0/2226173267 learned_addr learned my addr 172.21.15.16:0/2226173267 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.669+0000 7f71ba4b9640 1 --2- 172.21.15.16:0/2226173267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 0x55be2018fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.669+0000 7f71ab7fe640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f71a00a3240 0x7f71a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.669+0000 7f71b9cb8640 1 --2- 172.21.15.16:0/2226173267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be2019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.670+0000 7f71bacba640 1 -- 172.21.15.16:0/2226173267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 msgr2=0x55be2018fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.670+0000 7f71bacba640 1 --2- 172.21.15.16:0/2226173267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 0x55be2018fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.670+0000 7f71bacba640 1 -- 172.21.15.16:0/2226173267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 msgr2=0x55be2019c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.670+0000 7f71bacba640 1 --2- 172.21.15.16:0/2226173267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be2019c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.670+0000 7f71bacba640 1 -- 172.21.15.16:0/2226173267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be200813a0 con 0x55be20190b40 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.670+0000 7f71b9cb8640 1 --2- 172.21.15.16:0/2226173267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be2019c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.670+0000 7f71ab7fe640 1 -- 172.21.15.16:0/2395966212 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f71b00eff10 con 0x7f71b4101de0 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.670+0000 7f71bacba640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f71a00a3240 0x7f71a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.670+0000 7f71bacba640 1 --2- 172.21.15.16:0/2226173267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 0x55be2018f090 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f71ac008f60 tx=0x7f71ac008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f665d69958897c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.670+0000 7f71a8ff9640 1 -- 172.21.15.16:0/2226173267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac01a8e0 con 0x55be20190b40 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.670+0000 7f71a8ff9640 1 -- 172.21.15.16:0/2226173267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac019410 con 0x55be20190b40 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.670+0000 7f71a8ff9640 1 -- 172.21.15.16:0/2226173267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac00e610 con 0x55be20190b40 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.671+0000 7f71beaf6740 1 -- 172.21.15.16:0/2226173267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 msgr2=0x55be2018f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.671+0000 7f71beaf6740 1 --2- 172.21.15.16:0/2226173267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 0x55be2018f090 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f71ac008f60 tx=0x7f71ac008f90 comp rx=0 tx=0).stop 2024-09-19T09:28:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.671+0000 7f71bacba640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f71a00a3240 0x7f71a00a5700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f71ac01e000 tx=0x7f71ac01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.671+0000 7f71beaf6740 1 -- 172.21.15.16:0/2226173267 shutdown_connections 2024-09-19T09:28:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.671+0000 7f71beaf6740 1 --2- 172.21.15.16:0/2226173267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 0x55be2018fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.671+0000 7f71beaf6740 1 --2- 172.21.15.16:0/2226173267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be2019c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.671+0000 7f71beaf6740 1 --2- 172.21.15.16:0/2226173267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 0x55be2018f090 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.671+0000 7f71beaf6740 1 -- 172.21.15.16:0/2226173267 >> 172.21.15.16:0/2226173267 conn(0x55be20158310 msgr2=0x55be2018bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.671+0000 7f71beaf6740 1 -- 172.21.15.16:0/2226173267 shutdown_connections 2024-09-19T09:28:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.671+0000 7f71beaf6740 1 -- 172.21.15.16:0/2226173267 wait complete. 2024-09-19T09:28:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.671+0000 7f71beaf6740 1 -- start start 2024-09-19T09:28:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.671+0000 7f71beaf6740 1 Processor -- start 2024-09-19T09:28:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71beaf6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 0x55be20199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71beaf6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 0x55be20196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71beaf6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be20197560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71beaf6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be2019d070 con 0x55be20190b40 2024-09-19T09:28:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71beaf6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55be200b0060 con 0x55be2019a370 2024-09-19T09:28:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71beaf6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55be2019d1f0 con 0x55be2018f5d0 2024-09-19T09:28:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71beaf6740 10 client.0 _ll_register_callbacks cb 0x55be1fe038c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71b9cb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 0x55be20196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71b9cb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 0x55be20196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40688/0 (socket says 172.21.15.16:40688) 2024-09-19T09:28:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71b9cb8640 1 -- 172.21.15.16:0/252225195 learned_addr learned my addr 172.21.15.16:0/252225195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71bacba640 1 --2- 172.21.15.16:0/252225195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be20197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71ba4b9640 1 --2- 172.21.15.16:0/252225195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 0x55be20199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71b9cb8640 1 -- 172.21.15.16:0/252225195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 msgr2=0x55be20199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71b9cb8640 1 --2- 172.21.15.16:0/252225195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 0x55be20199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71b9cb8640 1 -- 172.21.15.16:0/252225195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 msgr2=0x55be20197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71b9cb8640 1 --2- 172.21.15.16:0/252225195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be20197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.672+0000 7f71b9cb8640 1 -- 172.21.15.16:0/252225195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be1fe663c0 con 0x55be20190b40 2024-09-19T09:28:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f71b9cb8640 1 --2- 172.21.15.16:0/252225195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 0x55be20196fc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f71a40106a0 tx=0x7f71a4010aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1fee74d8422181 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f719dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f719dffb640 1 -- 172.21.15.16:0/252225195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4004220 con 0x55be20190b40 2024-09-19T09:28:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f719dffb640 1 -- 172.21.15.16:0/252225195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f71a40043c0 con 0x55be20190b40 2024-09-19T09:28:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f71beaf6740 1 -- 172.21.15.16:0/252225195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ac019410 con 0x55be20190b40 2024-09-19T09:28:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f719dffb640 1 -- 172.21.15.16:0/252225195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71a4023050 con 0x55be20190b40 2024-09-19T09:28:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f71beaf6740 10 client.15780 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f71beaf6740 1 -- 172.21.15.16:0/252225195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be201a8b80 con 0x55be20190b40 2024-09-19T09:28:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f719dffb640 1 -- 172.21.15.16:0/252225195 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac019410 con 0x55be20190b40 2024-09-19T09:28:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f719dffb640 1 -- 172.21.15.16:0/252225195 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f71a400f040 con 0x55be20190b40 2024-09-19T09:28:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f71beaf6740 10 client.15780 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f71beaf6740 10 client.15780 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-19T09:28:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f71beaf6740 20 client.15780 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f71beaf6740 1 --2- 172.21.15.16:0/252225195 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55be201aa080 0x55be201ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f71beaf6740 4 client.15780 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.673+0000 7f71beaf6740 1 -- 172.21.15.16:0/252225195 --> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be20102860 con 0x55be201aa080 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.674+0000 7f71ba4b9640 1 --2- 172.21.15.16:0/252225195 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55be201aa080 0x55be201ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.674+0000 7f71ba4b9640 1 --2- 172.21.15.16:0/252225195 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55be201aa080 0x55be201ac4a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c979181d345b2e88 server_cookie=cf25408a70ceec81 in_seq=0 out_seq=0 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.674+0000 7f719dffb640 10 client.15780 ms_handle_connect on v2:172.21.15.142:6832/3906366829 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f719dffb640 1 -- 172.21.15.16:0/252225195 <== mds.4 v2:172.21.15.142:6832/3906366829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be20102860 con 0x55be201aa080 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f719dffb640 10 client.15780 handle_command_reply: tid=0 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f71beaf6740 1 client.15780 shutdown 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f71beaf6740 1 -- 172.21.15.16:0/252225195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 msgr2=0x55be20196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f71beaf6740 1 --2- 172.21.15.16:0/252225195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 0x55be20196fc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f71a40106a0 tx=0x7f71a4010aa0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f71beaf6740 1 -- 172.21.15.16:0/252225195 shutdown_connections 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f71beaf6740 1 --2- 172.21.15.16:0/252225195 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55be201aa080 0x55be201ac4a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f71beaf6740 1 --2- 172.21.15.16:0/252225195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be20197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f71beaf6740 1 --2- 172.21.15.16:0/252225195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be20190b40 0x55be20196fc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f71beaf6740 1 --2- 172.21.15.16:0/252225195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 0x55be20199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f71beaf6740 1 -- 172.21.15.16:0/252225195 >> 172.21.15.16:0/252225195 conn(0x55be20158310 msgr2=0x55be20191930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:56.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f719dffb640 0 client.15780 ms_handle_reset on v2:172.21.15.142:6832/3906366829 2024-09-19T09:28:56.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f71beaf6740 1 -- 172.21.15.16:0/252225195 shutdown_connections 2024-09-19T09:28:56.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f71beaf6740 1 -- 172.21.15.16:0/252225195 wait complete. 2024-09-19T09:28:56.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.675+0000 7f71beaf6740 20 client.15780 trim_cache size 0 max 16384 2024-09-19T09:28:56.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.688+0000 7f71beaf6740 1 Processor -- start 2024-09-19T09:28:56.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.688+0000 7f71beaf6740 1 -- start start 2024-09-19T09:28:56.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.688+0000 7f71beaf6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2018f5d0 0x55be2018be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.688+0000 7f71beaf6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be20190b40 0x55be201a6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71beaf6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be20196880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71beaf6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41af780 con 0x55be2018f5d0 2024-09-19T09:28:56.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71beaf6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41b12c0 con 0x55be2019a370 2024-09-19T09:28:56.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71beaf6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41afe50 con 0x55be20190b40 2024-09-19T09:28:56.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71ba4b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2018f5d0 0x55be2018be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71ba4b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2018f5d0 0x55be2018be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40700/0 (socket says 172.21.15.16:40700) 2024-09-19T09:28:56.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71ba4b9640 1 -- 172.21.15.16:0/1746180029 learned_addr learned my addr 172.21.15.16:0/1746180029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:56.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71b9cb8640 1 --2- 172.21.15.16:0/1746180029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be20190b40 0x55be201a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71bacba640 1 --2- 172.21.15.16:0/1746180029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be20196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71ba4b9640 1 -- 172.21.15.16:0/1746180029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be20190b40 msgr2=0x55be201a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71ba4b9640 1 --2- 172.21.15.16:0/1746180029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be20190b40 0x55be201a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71ba4b9640 1 -- 172.21.15.16:0/1746180029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 msgr2=0x55be20196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71ba4b9640 1 --2- 172.21.15.16:0/1746180029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be20196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71ba4b9640 1 -- 172.21.15.16:0/1746180029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be20195510 con 0x55be2018f5d0 2024-09-19T09:28:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71b9cb8640 1 --2- 172.21.15.16:0/1746180029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be20190b40 0x55be201a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71bacba640 1 --2- 172.21.15.16:0/1746180029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be20196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f71ba4b9640 1 --2- 172.21.15.16:0/1746180029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2018f5d0 0x55be2018be10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f71b00bc540 tx=0x7f71b00bc430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1115da8722f34d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f719ffff640 1 -- 172.21.15.16:0/1746180029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0049040 con 0x55be2018f5d0 2024-09-19T09:28:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f719ffff640 1 -- 172.21.15.16:0/1746180029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b00da070 con 0x55be2018f5d0 2024-09-19T09:28:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.689+0000 7f719ffff640 1 -- 172.21.15.16:0/1746180029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b00edae0 con 0x55be2018f5d0 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.690+0000 7f71beaf6740 1 -- 172.21.15.16:0/1746180029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2018f5d0 msgr2=0x55be2018be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.690+0000 7f71beaf6740 1 --2- 172.21.15.16:0/1746180029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2018f5d0 0x55be2018be10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f71b00bc540 tx=0x7f71b00bc430 comp rx=0 tx=0).stop 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.690+0000 7f71beaf6740 1 -- 172.21.15.16:0/1746180029 shutdown_connections 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.690+0000 7f71beaf6740 1 --2- 172.21.15.16:0/1746180029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be20190b40 0x55be201a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.690+0000 7f71beaf6740 1 --2- 172.21.15.16:0/1746180029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be2019a370 0x55be20196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.690+0000 7f71beaf6740 1 --2- 172.21.15.16:0/1746180029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2018f5d0 0x55be2018be10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.690+0000 7f71beaf6740 1 -- 172.21.15.16:0/1746180029 >> 172.21.15.16:0/1746180029 conn(0x55be20158310 msgr2=0x55be2018b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.690+0000 7f71beaf6740 1 -- 172.21.15.16:0/1746180029 shutdown_connections 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.690+0000 7f71beaf6740 1 -- 172.21.15.16:0/1746180029 wait complete. 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.690+0000 7f71beaf6740 1 -- start start 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.690+0000 7f71beaf6740 1 Processor -- start 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71beaf6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 0x55be201a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71beaf6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be20190b40 0x55be201a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71beaf6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2019a370 0x55be201937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71beaf6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41b0bf0 con 0x55be2019a370 2024-09-19T09:28:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71beaf6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41b0520 con 0x55be20190b40 2024-09-19T09:28:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71beaf6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41ad740 con 0x55be2018f5d0 2024-09-19T09:28:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71beaf6740 10 client.0 _ll_register_callbacks cb 0x7f71a40094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71bacba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2019a370 0x55be201937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71bacba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2019a370 0x55be201937d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40716/0 (socket says 172.21.15.16:40716) 2024-09-19T09:28:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71bacba640 1 -- 172.21.15.16:0/1958219091 learned_addr learned my addr 172.21.15.16:0/1958219091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71b9cb8640 1 --2- 172.21.15.16:0/1958219091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be20190b40 0x55be201a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71ba4b9640 1 --2- 172.21.15.16:0/1958219091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 0x55be201a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71bacba640 1 -- 172.21.15.16:0/1958219091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 msgr2=0x55be201a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71bacba640 1 --2- 172.21.15.16:0/1958219091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 0x55be201a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71bacba640 1 -- 172.21.15.16:0/1958219091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be20190b40 msgr2=0x55be201a2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71bacba640 1 --2- 172.21.15.16:0/1958219091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be20190b40 0x55be201a2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71bacba640 1 -- 172.21.15.16:0/1958219091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be20193ee0 con 0x55be2019a370 2024-09-19T09:28:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f71bacba640 1 --2- 172.21.15.16:0/1958219091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2019a370 0x55be201937d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005cc0 tx=0x7f71ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d440fab49b1a9bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f719dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.691+0000 7f719dffb640 1 -- 172.21.15.16:0/1958219091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac008580 con 0x55be2019a370 2024-09-19T09:28:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.692+0000 7f719dffb640 1 -- 172.21.15.16:0/1958219091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f71ac019e30 con 0x55be2019a370 2024-09-19T09:28:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.692+0000 7f71beaf6740 1 -- 172.21.15.16:0/1958219091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b00da070 con 0x55be2019a370 2024-09-19T09:28:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.692+0000 7f719dffb640 1 -- 172.21.15.16:0/1958219091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71ac011440 con 0x55be2019a370 2024-09-19T09:28:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.692+0000 7f71beaf6740 10 client.15786 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.692+0000 7f71beaf6740 1 -- 172.21.15.16:0/1958219091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be202146b0 con 0x55be2019a370 2024-09-19T09:28:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.692+0000 7f719dffb640 1 -- 172.21.15.16:0/1958219091 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b00da070 con 0x55be2019a370 2024-09-19T09:28:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.692+0000 7f719dffb640 1 -- 172.21.15.16:0/1958219091 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f71ac019410 con 0x55be2019a370 2024-09-19T09:28:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.692+0000 7f71beaf6740 10 client.15786 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.692+0000 7f71beaf6740 10 client.15786 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-19T09:28:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.692+0000 7f71beaf6740 20 client.15786 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.692+0000 7f71beaf6740 1 --2- 172.21.15.16:0/1958219091 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55be20214a90 0x55be20216eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.692+0000 7f71beaf6740 4 client.15786 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:28:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.692+0000 7f71beaf6740 1 -- 172.21.15.16:0/1958219091 --> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be200fcd70 con 0x55be20214a90 2024-09-19T09:28:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f71ba4b9640 1 --2- 172.21.15.16:0/1958219091 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55be20214a90 0x55be20216eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f71ba4b9640 1 --2- 172.21.15.16:0/1958219091 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55be20214a90 0x55be20216eb0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3faf7fb5374db8a server_cookie=39e51ce6fa054e75 in_seq=0 out_seq=0 2024-09-19T09:28:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f719dffb640 10 client.15786 ms_handle_connect on v2:172.21.15.142:6832/3906366829 2024-09-19T09:28:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f719dffb640 1 -- 172.21.15.16:0/1958219091 <== mds.4 v2:172.21.15.142:6832/3906366829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be200fcd70 con 0x55be20214a90 2024-09-19T09:28:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f719dffb640 10 client.15786 handle_command_reply: tid=0 2024-09-19T09:28:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f71beaf6740 1 client.15786 shutdown 2024-09-19T09:28:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f71beaf6740 1 -- 172.21.15.16:0/1958219091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2019a370 msgr2=0x55be201937d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f71beaf6740 1 --2- 172.21.15.16:0/1958219091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2019a370 0x55be201937d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005cc0 tx=0x7f71ac00ee80 comp rx=0 tx=0).stop 2024-09-19T09:28:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f71beaf6740 1 -- 172.21.15.16:0/1958219091 shutdown_connections 2024-09-19T09:28:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f71beaf6740 1 --2- 172.21.15.16:0/1958219091 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x55be20214a90 0x55be20216eb0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f71beaf6740 1 --2- 172.21.15.16:0/1958219091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be2019a370 0x55be201937d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f71beaf6740 1 --2- 172.21.15.16:0/1958219091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be20190b40 0x55be201a2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f71beaf6740 1 --2- 172.21.15.16:0/1958219091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be2018f5d0 0x55be201a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f719dffb640 0 client.15786 ms_handle_reset on v2:172.21.15.142:6832/3906366829 2024-09-19T09:28:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.693+0000 7f71beaf6740 1 -- 172.21.15.16:0/1958219091 >> 172.21.15.16:0/1958219091 conn(0x55be20158310 msgr2=0x55be20195810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71beaf6740 1 -- 172.21.15.16:0/1958219091 shutdown_connections 2024-09-19T09:28:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71beaf6740 1 -- 172.21.15.16:0/1958219091 wait complete. 2024-09-19T09:28:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71beaf6740 20 client.15786 trim_cache size 0 max 16384 2024-09-19T09:28:56.703 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:28:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71bc71d640 1 -- 172.21.15.16:0/2395966212 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f71a00a3240 msgr2=0x7f71a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71bc71d640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f71a00a3240 0x7f71a00a5700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f71ac01e000 tx=0x7f71ac01f000 comp rx=0 tx=0).stop 2024-09-19T09:28:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71bc71d640 1 -- 172.21.15.16:0/2395966212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4101de0 msgr2=0x7f71b40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71bc71d640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4101de0 0x7f71b40744f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f71b0006010 tx=0x7f71b00042f0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71bc71d640 1 -- 172.21.15.16:0/2395966212 shutdown_connections 2024-09-19T09:28:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71bc71d640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f71a00a3240 0x7f71a00a5700 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71bc71d640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b4102790 0x7f71b4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71bc71d640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71b4106f10 0x7f71b4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71bc71d640 1 --2- 172.21.15.16:0/2395966212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4101de0 0x7f71b40744f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71bc71d640 1 -- 172.21.15.16:0/2395966212 >> 172.21.15.16:0/2395966212 conn(0x7f71b40fd8c0 msgr2=0x7f71b40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71bc71d640 1 -- 172.21.15.16:0/2395966212 shutdown_connections 2024-09-19T09:28:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:56.694+0000 7f71bc71d640 1 -- 172.21.15.16:0/2395966212 wait complete. 2024-09-19T09:28:57.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:57 smithi142 ceph-mon[35032]: pgmap v378: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 688 KiB/s wr, 3.03k op/s 2024-09-19T09:28:57.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:57 smithi097 ceph-mon[34864]: pgmap v378: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 688 KiB/s wr, 3.03k op/s 2024-09-19T09:28:57.420 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:57 smithi016 ceph-mon[30979]: pgmap v378: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 688 KiB/s wr, 3.03k op/s 2024-09-19T09:28:57.448 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:28:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.140+0000 7fbe21f3a640 1 -- 172.21.15.16:0/3960430709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe1c103660 msgr2=0x7fbe1c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.140+0000 7fbe21f3a640 1 --2- 172.21.15.16:0/3960430709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe1c103660 0x7fbe1c10bb50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009900 tx=0x7fbe1002a480 comp rx=0 tx=0).stop 2024-09-19T09:28:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.140+0000 7fbe21f3a640 1 -- 172.21.15.16:0/3960430709 shutdown_connections 2024-09-19T09:28:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.140+0000 7fbe21f3a640 1 --2- 172.21.15.16:0/3960430709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe1c10c170 0x7fbe1c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.140+0000 7fbe21f3a640 1 --2- 172.21.15.16:0/3960430709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe1c103660 0x7fbe1c10bb50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.140+0000 7fbe21f3a640 1 --2- 172.21.15.16:0/3960430709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe1c102d40 0x7fbe1c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.140+0000 7fbe21f3a640 1 -- 172.21.15.16:0/3960430709 >> 172.21.15.16:0/3960430709 conn(0x7fbe1c0fc7a0 msgr2=0x7fbe1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.140+0000 7fbe21f3a640 1 -- 172.21.15.16:0/3960430709 shutdown_connections 2024-09-19T09:28:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.140+0000 7fbe21f3a640 1 -- 172.21.15.16:0/3960430709 wait complete. 2024-09-19T09:28:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.141+0000 7fbe21f3a640 1 Processor -- start 2024-09-19T09:28:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.141+0000 7fbe21f3a640 1 -- start start 2024-09-19T09:28:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.141+0000 7fbe21f3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe1c102d40 0x7fbe1c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.141+0000 7fbe21f3a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe1c103660 0x7fbe1c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe21f3a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe1c10c170 0x7fbe1c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe21f3a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c111fb0 con 0x7fbe1c102d40 2024-09-19T09:28:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe21f3a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c111e30 con 0x7fbe1c103660 2024-09-19T09:28:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe21f3a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c112130 con 0x7fbe1c10c170 2024-09-19T09:28:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe1b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe1c102d40 0x7fbe1c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe1b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe1c102d40 0x7fbe1c19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40740/0 (socket says 172.21.15.16:40740) 2024-09-19T09:28:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe1b7fe640 1 -- 172.21.15.16:0/4157118311 learned_addr learned my addr 172.21.15.16:0/4157118311 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe1affd640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe1c103660 0x7fbe1c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe1c10c170 0x7fbe1c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe1b7fe640 1 -- 172.21.15.16:0/4157118311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe1c10c170 msgr2=0x7fbe1c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe1b7fe640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe1c10c170 0x7fbe1c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe1b7fe640 1 -- 172.21.15.16:0/4157118311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe1c103660 msgr2=0x7fbe1c19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe1b7fe640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe1c103660 0x7fbe1c19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe1b7fe640 1 -- 172.21.15.16:0/4157118311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c19f6a0 con 0x7fbe1c102d40 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe1c10c170 0x7fbe1c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe1b7fe640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe1c102d40 0x7fbe1c19a6f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00b4a0 tx=0x7fbe0c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe18ff9640 1 -- 172.21.15.16:0/4157118311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c014070 con 0x7fbe1c102d40 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe18ff9640 1 -- 172.21.15.16:0/4157118311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe0c010af0 con 0x7fbe1c102d40 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe21f3a640 1 -- 172.21.15.16:0/4157118311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1c19f990 con 0x7fbe1c102d40 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.142+0000 7fbe18ff9640 1 -- 172.21.15.16:0/4157118311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c017d10 con 0x7fbe1c102d40 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.143+0000 7fbe21f3a640 1 -- 172.21.15.16:0/4157118311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe1c1a7270 con 0x7fbe1c102d40 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe18ff9640 1 -- 172.21.15.16:0/4157118311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fbe0c017070 con 0x7fbe1c102d40 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe24313740 1 Processor -- start 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe24313740 1 -- start start 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe24313740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce31137b40 0x55ce31136090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe24313740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce311365d0 0x55ce31136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe24313740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31141370 0x55ce31143760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe24313740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce3109c2f0 con 0x55ce31137b40 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe24313740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ce31043cf0 con 0x55ce31141370 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe24313740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ce310982b0 con 0x55ce311365d0 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce31137b40 0x55ce31136090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce31137b40 0x55ce31136090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40752/0 (socket says 172.21.15.16:40752) 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe1bfff640 1 -- 172.21.15.16:0/1779880385 learned_addr learned my addr 172.21.15.16:0/1779880385 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe1affd640 1 --2- 172.21.15.16:0/1779880385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31141370 0x55ce31143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe1b7fe640 1 --2- 172.21.15.16:0/1779880385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce311365d0 0x55ce31136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe1bfff640 1 -- 172.21.15.16:0/1779880385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce311365d0 msgr2=0x55ce31136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/1779880385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce311365d0 0x55ce31136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe1bfff640 1 -- 172.21.15.16:0/1779880385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31141370 msgr2=0x55ce31143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/1779880385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31141370 0x55ce31143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.146+0000 7fbe1bfff640 1 -- 172.21.15.16:0/1779880385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce310283a0 con 0x55ce31137b40 2024-09-19T09:28:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.147+0000 7fbe18ff9640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbe040a3240 0x7fbe040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.147+0000 7fbe1b7fe640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbe040a3240 0x7fbe040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.147+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/1779880385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce31137b40 0x55ce31136090 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400af40 tx=0x7fbe1400e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c433ecfb4cb64b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.147+0000 7fbe18ff9640 1 -- 172.21.15.16:0/4157118311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fbe0c0ced50 con 0x7fbe1c102d40 2024-09-19T09:28:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.147+0000 7fbe01ffb640 1 -- 172.21.15.16:0/1779880385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14003b70 con 0x55ce31137b40 2024-09-19T09:28:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.147+0000 7fbe01ffb640 1 -- 172.21.15.16:0/1779880385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe14010b20 con 0x55ce31137b40 2024-09-19T09:28:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.147+0000 7fbe01ffb640 1 -- 172.21.15.16:0/1779880385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1400fbb0 con 0x55ce31137b40 2024-09-19T09:28:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.148+0000 7fbe24313740 1 -- 172.21.15.16:0/1779880385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce31137b40 msgr2=0x55ce31136090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.148+0000 7fbe24313740 1 --2- 172.21.15.16:0/1779880385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce31137b40 0x55ce31136090 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400af40 tx=0x7fbe1400e7e0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.148+0000 7fbe1b7fe640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbe040a3240 0x7fbe040a5700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0977d0 tx=0x7fbe0c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.148+0000 7fbe24313740 1 -- 172.21.15.16:0/1779880385 shutdown_connections 2024-09-19T09:28:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.148+0000 7fbe24313740 1 --2- 172.21.15.16:0/1779880385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce311365d0 0x55ce31136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.148+0000 7fbe24313740 1 --2- 172.21.15.16:0/1779880385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31141370 0x55ce31143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.148+0000 7fbe24313740 1 --2- 172.21.15.16:0/1779880385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce31137b40 0x55ce31136090 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.148+0000 7fbe24313740 1 -- 172.21.15.16:0/1779880385 >> 172.21.15.16:0/1779880385 conn(0x55ce310ff310 msgr2=0x55ce31132c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.148+0000 7fbe24313740 1 -- 172.21.15.16:0/1779880385 shutdown_connections 2024-09-19T09:28:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.148+0000 7fbe24313740 1 -- 172.21.15.16:0/1779880385 wait complete. 2024-09-19T09:28:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.149+0000 7fbe24313740 1 -- start start 2024-09-19T09:28:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.149+0000 7fbe24313740 1 Processor -- start 2024-09-19T09:28:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.149+0000 7fbe24313740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce31140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.149+0000 7fbe24313740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31137b40 0x55ce3113df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.149+0000 7fbe24313740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31141370 0x55ce3113e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.149+0000 7fbe24313740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce31144070 con 0x55ce311365d0 2024-09-19T09:28:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.149+0000 7fbe24313740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ce31057060 con 0x55ce31137b40 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.149+0000 7fbe24313740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ce311441f0 con 0x55ce31141370 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.149+0000 7fbe1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce31140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.149+0000 7fbe24313740 10 client.0 _ll_register_callbacks cb 0x55ce30daa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.149+0000 7fbe1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce31140cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40764/0 (socket says 172.21.15.16:40764) 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.149+0000 7fbe1bfff640 1 -- 172.21.15.16:0/671917105 learned_addr learned my addr 172.21.15.16:0/671917105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.150+0000 7fbe1affd640 1 --2- 172.21.15.16:0/671917105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31137b40 0x55ce3113df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.150+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/671917105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31141370 0x55ce3113e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.150+0000 7fbe1bfff640 1 -- 172.21.15.16:0/671917105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31141370 msgr2=0x55ce3113e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.150+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/671917105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31141370 0x55ce3113e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.150+0000 7fbe1bfff640 1 -- 172.21.15.16:0/671917105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31137b40 msgr2=0x55ce3113df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.150+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/671917105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31137b40 0x55ce3113df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.150+0000 7fbe1bfff640 1 -- 172.21.15.16:0/671917105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce30e0d3c0 con 0x55ce311365d0 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.150+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/671917105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce31140cd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400b010 tx=0x7fbe1400efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c52acc07e516d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.150+0000 7fbdfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.150+0000 7fbdfaffd640 1 -- 172.21.15.16:0/671917105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14004c40 con 0x55ce311365d0 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.150+0000 7fbdfaffd640 1 -- 172.21.15.16:0/671917105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fbe14004de0 con 0x55ce311365d0 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.150+0000 7fbe24313740 1 -- 172.21.15.16:0/671917105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe14010b20 con 0x55ce311365d0 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.151+0000 7fbdfaffd640 1 -- 172.21.15.16:0/671917105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbe1400f930 con 0x55ce311365d0 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.151+0000 7fbe24313740 10 client.15798 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.151+0000 7fbe24313740 1 -- 172.21.15.16:0/671917105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ce3114fa80 con 0x55ce311365d0 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.151+0000 7fbdfaffd640 1 -- 172.21.15.16:0/671917105 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe14010b20 con 0x55ce311365d0 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.151+0000 7fbdfaffd640 1 -- 172.21.15.16:0/671917105 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbe14028020 con 0x55ce311365d0 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.151+0000 7fbe24313740 10 client.15798 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.151+0000 7fbe24313740 10 client.15798 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.151+0000 7fbe24313740 20 client.15798 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.151+0000 7fbe24313740 1 --2- 172.21.15.16:0/671917105 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ce31150e80 0x55ce31153270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.151+0000 7fbe24313740 4 client.15798 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.151+0000 7fbe24313740 1 -- 172.21.15.16:0/671917105 --> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce310a9860 con 0x55ce31150e80 2024-09-19T09:28:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.151+0000 7fbe1affd640 1 --2- 172.21.15.16:0/671917105 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ce31150e80 0x55ce31153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.152+0000 7fbe1affd640 1 --2- 172.21.15.16:0/671917105 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ce31150e80 0x55ce31153270 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cac7efdeed8fe13 server_cookie=5de9d1e64c01070f in_seq=0 out_seq=0 2024-09-19T09:28:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.152+0000 7fbdfaffd640 10 client.15798 ms_handle_connect on v2:172.21.15.97:6832/108652274 2024-09-19T09:28:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.152+0000 7fbdfaffd640 1 -- 172.21.15.16:0/671917105 <== mds.3 v2:172.21.15.97:6832/108652274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce310a9860 con 0x55ce31150e80 2024-09-19T09:28:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.152+0000 7fbdfaffd640 10 client.15798 handle_command_reply: tid=0 2024-09-19T09:28:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.153+0000 7fbe24313740 1 client.15798 shutdown 2024-09-19T09:28:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.153+0000 7fbe24313740 1 -- 172.21.15.16:0/671917105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 msgr2=0x55ce31140cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.153+0000 7fbe24313740 1 --2- 172.21.15.16:0/671917105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce31140cd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400b010 tx=0x7fbe1400efb0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.153+0000 7fbe24313740 1 -- 172.21.15.16:0/671917105 shutdown_connections 2024-09-19T09:28:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.153+0000 7fbe24313740 1 --2- 172.21.15.16:0/671917105 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ce31150e80 0x55ce31153270 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.153+0000 7fbe24313740 1 --2- 172.21.15.16:0/671917105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31141370 0x55ce3113e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.153+0000 7fbe24313740 1 --2- 172.21.15.16:0/671917105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31137b40 0x55ce3113df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.153+0000 7fbe24313740 1 --2- 172.21.15.16:0/671917105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce31140cd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.153+0000 7fbe24313740 1 -- 172.21.15.16:0/671917105 >> 172.21.15.16:0/671917105 conn(0x55ce310ff310 msgr2=0x55ce31138960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.153+0000 7fbdfaffd640 0 client.15798 ms_handle_reset on v2:172.21.15.97:6832/108652274 2024-09-19T09:28:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.153+0000 7fbe24313740 1 -- 172.21.15.16:0/671917105 shutdown_connections 2024-09-19T09:28:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.153+0000 7fbe24313740 1 -- 172.21.15.16:0/671917105 wait complete. 2024-09-19T09:28:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.153+0000 7fbe24313740 20 client.15798 trim_cache size 0 max 16384 2024-09-19T09:28:58.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.166+0000 7fbe24313740 1 Processor -- start 2024-09-19T09:28:58.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.166+0000 7fbe24313740 1 -- start start 2024-09-19T09:28:58.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.166+0000 7fbe24313740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce3113d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.166+0000 7fbe24313740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31137b40 0x55ce31139120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.166+0000 7fbe24313740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31141370 0x55ce31139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.166+0000 7fbe24313740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1af8f0 con 0x55ce311365d0 2024-09-19T09:28:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.166+0000 7fbe24313740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1b1430 con 0x55ce31137b40 2024-09-19T09:28:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.166+0000 7fbe24313740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1affc0 con 0x55ce31141370 2024-09-19T09:28:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.166+0000 7fbe1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce3113d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.166+0000 7fbe1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce3113d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40766/0 (socket says 172.21.15.16:40766) 2024-09-19T09:28:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.167+0000 7fbe1bfff640 1 -- 172.21.15.16:0/3938167965 learned_addr learned my addr 172.21.15.16:0/3938167965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.167+0000 7fbe1affd640 1 --2- 172.21.15.16:0/3938167965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31137b40 0x55ce31139120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.167+0000 7fbe1bfff640 1 -- 172.21.15.16:0/3938167965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31141370 msgr2=0x55ce31139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.167+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/3938167965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31141370 0x55ce31139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.167+0000 7fbe1bfff640 1 -- 172.21.15.16:0/3938167965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31137b40 msgr2=0x55ce31139120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.167+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/3938167965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31137b40 0x55ce31139120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.167+0000 7fbe1bfff640 1 -- 172.21.15.16:0/3938167965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce311408f0 con 0x55ce311365d0 2024-09-19T09:28:58.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.167+0000 7fbe1affd640 1 --2- 172.21.15.16:0/3938167965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31137b40 0x55ce31139120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:58.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.167+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/3938167965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce3113d840 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbe14013040 tx=0x7fbe1400a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c90003d71c1f66a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:58.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.167+0000 7fbe017fa640 1 -- 172.21.15.16:0/3938167965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14005950 con 0x55ce311365d0 2024-09-19T09:28:58.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.167+0000 7fbe017fa640 1 -- 172.21.15.16:0/3938167965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe1401f030 con 0x55ce311365d0 2024-09-19T09:28:58.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.167+0000 7fbe017fa640 1 -- 172.21.15.16:0/3938167965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1402a6c0 con 0x55ce311365d0 2024-09-19T09:28:58.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.168+0000 7fbe24313740 1 -- 172.21.15.16:0/3938167965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 msgr2=0x55ce3113d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.168+0000 7fbe24313740 1 --2- 172.21.15.16:0/3938167965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce3113d840 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbe14013040 tx=0x7fbe1400a3d0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.168+0000 7fbe24313740 1 -- 172.21.15.16:0/3938167965 shutdown_connections 2024-09-19T09:28:58.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.168+0000 7fbe24313740 1 --2- 172.21.15.16:0/3938167965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31141370 0x55ce31139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.168+0000 7fbe24313740 1 --2- 172.21.15.16:0/3938167965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31137b40 0x55ce31139120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.168+0000 7fbe24313740 1 --2- 172.21.15.16:0/3938167965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce3113d840 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.168+0000 7fbe24313740 1 -- 172.21.15.16:0/3938167965 >> 172.21.15.16:0/3938167965 conn(0x55ce310ff310 msgr2=0x55ce3113d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.168+0000 7fbe24313740 1 -- 172.21.15.16:0/3938167965 shutdown_connections 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.168+0000 7fbe24313740 1 -- 172.21.15.16:0/3938167965 wait complete. 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.168+0000 7fbe24313740 1 -- start start 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe24313740 1 Processor -- start 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe24313740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce31153260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe24313740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31137b40 0x55ce311537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe24313740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31141370 0x55ce311b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe24313740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1b0d60 con 0x55ce311365d0 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe24313740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1b0690 con 0x55ce31141370 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe24313740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1ad8b0 con 0x55ce31137b40 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe24313740 10 client.0 _ll_register_callbacks cb 0x7fbe14014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce31153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce31153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40778/0 (socket says 172.21.15.16:40778) 2024-09-19T09:28:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe1bfff640 1 -- 172.21.15.16:0/1065001820 learned_addr learned my addr 172.21.15.16:0/1065001820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:58.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/1065001820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31141370 0x55ce311b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe1affd640 1 --2- 172.21.15.16:0/1065001820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31137b40 0x55ce311537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe1bfff640 1 -- 172.21.15.16:0/1065001820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31137b40 msgr2=0x55ce311537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/1065001820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31137b40 0x55ce311537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe1bfff640 1 -- 172.21.15.16:0/1065001820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31141370 msgr2=0x55ce311b97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/1065001820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31141370 0x55ce311b97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.169+0000 7fbe1bfff640 1 -- 172.21.15.16:0/1065001820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce311b9cf0 con 0x55ce311365d0 2024-09-19T09:28:58.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.170+0000 7fbe1bfff640 1 --2- 172.21.15.16:0/1065001820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce31153260 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400ae60 tx=0x7fbe14010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70856f2cce77e5ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:58.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.170+0000 7fbdfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.170+0000 7fbdfaffd640 1 -- 172.21.15.16:0/1065001820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe140109a0 con 0x55ce311365d0 2024-09-19T09:28:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.170+0000 7fbdfaffd640 1 -- 172.21.15.16:0/1065001820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fbe1400fad0 con 0x55ce311365d0 2024-09-19T09:28:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.170+0000 7fbe24313740 1 -- 172.21.15.16:0/1065001820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe1401f030 con 0x55ce311365d0 2024-09-19T09:28:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.170+0000 7fbdfaffd640 1 -- 172.21.15.16:0/1065001820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbe1400fcd0 con 0x55ce311365d0 2024-09-19T09:28:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.170+0000 7fbe24313740 10 client.15804 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.170+0000 7fbe24313740 1 -- 172.21.15.16:0/1065001820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ce311baef0 con 0x55ce311365d0 2024-09-19T09:28:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.170+0000 7fbdfaffd640 1 -- 172.21.15.16:0/1065001820 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe1401f030 con 0x55ce311365d0 2024-09-19T09:28:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.170+0000 7fbdfaffd640 1 -- 172.21.15.16:0/1065001820 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbe1403e070 con 0x55ce311365d0 2024-09-19T09:28:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.170+0000 7fbe24313740 10 client.15804 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.170+0000 7fbe24313740 10 client.15804 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-19T09:28:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.170+0000 7fbe24313740 20 client.15804 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.171+0000 7fbe24313740 1 --2- 172.21.15.16:0/1065001820 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ce311bb500 0x55ce311bd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.171+0000 7fbe24313740 4 client.15804 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:28:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.171+0000 7fbe24313740 1 -- 172.21.15.16:0/1065001820 --> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce310a3d70 con 0x55ce311bb500 2024-09-19T09:28:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.171+0000 7fbe1affd640 1 --2- 172.21.15.16:0/1065001820 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ce311bb500 0x55ce311bd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.171+0000 7fbe1affd640 1 --2- 172.21.15.16:0/1065001820 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ce311bb500 0x55ce311bd8f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=650bb2b131d3ea0e server_cookie=f864c4975dbe2b38 in_seq=0 out_seq=0 2024-09-19T09:28:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.171+0000 7fbdfaffd640 10 client.15804 ms_handle_connect on v2:172.21.15.97:6832/108652274 2024-09-19T09:28:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.171+0000 7fbdfaffd640 1 -- 172.21.15.16:0/1065001820 <== mds.3 v2:172.21.15.97:6832/108652274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce310a3d70 con 0x55ce311bb500 2024-09-19T09:28:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbdfaffd640 10 client.15804 handle_command_reply: tid=0 2024-09-19T09:28:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbe24313740 1 client.15804 shutdown 2024-09-19T09:28:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbe24313740 1 -- 172.21.15.16:0/1065001820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 msgr2=0x55ce31153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbe24313740 1 --2- 172.21.15.16:0/1065001820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce31153260 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400ae60 tx=0x7fbe14010d00 comp rx=0 tx=0).stop 2024-09-19T09:28:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbe24313740 1 -- 172.21.15.16:0/1065001820 shutdown_connections 2024-09-19T09:28:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbe24313740 1 --2- 172.21.15.16:0/1065001820 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55ce311bb500 0x55ce311bd8f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbe24313740 1 --2- 172.21.15.16:0/1065001820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce31137b40 0x55ce311537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbe24313740 1 --2- 172.21.15.16:0/1065001820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ce31141370 0x55ce311b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbe24313740 1 --2- 172.21.15.16:0/1065001820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce311365d0 0x55ce31153260 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbdfaffd640 0 client.15804 ms_handle_reset on v2:172.21.15.97:6832/108652274 2024-09-19T09:28:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbe24313740 1 -- 172.21.15.16:0/1065001820 >> 172.21.15.16:0/1065001820 conn(0x55ce310ff310 msgr2=0x55ce3113f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbe24313740 1 -- 172.21.15.16:0/1065001820 shutdown_connections 2024-09-19T09:28:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbe24313740 1 -- 172.21.15.16:0/1065001820 wait complete. 2024-09-19T09:28:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.172+0000 7fbe24313740 20 client.15804 trim_cache size 0 max 16384 2024-09-19T09:28:58.179 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:28:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.173+0000 7fbe21f3a640 1 -- 172.21.15.16:0/4157118311 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbe040a3240 msgr2=0x7fbe040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.173+0000 7fbe21f3a640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbe040a3240 0x7fbe040a5700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0977d0 tx=0x7fbe0c0046c0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.173+0000 7fbe21f3a640 1 -- 172.21.15.16:0/4157118311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe1c102d40 msgr2=0x7fbe1c19a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.173+0000 7fbe21f3a640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe1c102d40 0x7fbe1c19a6f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00b4a0 tx=0x7fbe0c00b960 comp rx=0 tx=0).stop 2024-09-19T09:28:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.173+0000 7fbe21f3a640 1 -- 172.21.15.16:0/4157118311 shutdown_connections 2024-09-19T09:28:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.173+0000 7fbe21f3a640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbe040a3240 0x7fbe040a5700 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.173+0000 7fbe21f3a640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe1c10c170 0x7fbe1c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.173+0000 7fbe21f3a640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe1c103660 0x7fbe1c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.173+0000 7fbe21f3a640 1 --2- 172.21.15.16:0/4157118311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe1c102d40 0x7fbe1c19a6f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.173+0000 7fbe21f3a640 1 -- 172.21.15.16:0/4157118311 >> 172.21.15.16:0/4157118311 conn(0x7fbe1c0fc7a0 msgr2=0x7fbe1c101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.173+0000 7fbe21f3a640 1 -- 172.21.15.16:0/4157118311 shutdown_connections 2024-09-19T09:28:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:58.173+0000 7fbe21f3a640 1 -- 172.21.15.16:0/4157118311 wait complete. 2024-09-19T09:28:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.226+0000 7f816d61e640 1 -- 172.21.15.16:0/901690867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168107880 msgr2=0x7f8168109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.226+0000 7f816d61e640 1 --2- 172.21.15.16:0/901690867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168107880 0x7f8168109c90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8160009ca0 tx=0x7f816002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 -- 172.21.15.16:0/901690867 shutdown_connections 2024-09-19T09:28:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 --2- 172.21.15.16:0/901690867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168104f50 0x7f8168107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 --2- 172.21.15.16:0/901690867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168107880 0x7f8168109c90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 --2- 172.21.15.16:0/901690867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8168102620 0x7f8168104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 -- 172.21.15.16:0/901690867 >> 172.21.15.16:0/901690867 conn(0x7f81680fc1e0 msgr2=0x7f81680fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 -- 172.21.15.16:0/901690867 shutdown_connections 2024-09-19T09:28:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 -- 172.21.15.16:0/901690867 wait complete. 2024-09-19T09:28:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 Processor -- start 2024-09-19T09:28:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 -- start start 2024-09-19T09:28:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168102620 0x7f8168101f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8168104f50 0x7f81681005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168107880 0x7f8168100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f816810d830 con 0x7f8168102620 2024-09-19T09:28:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f816810d6b0 con 0x7f8168104f50 2024-09-19T09:28:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.227+0000 7f816d61e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f816810d9b0 con 0x7f8168107880 2024-09-19T09:28:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f8166ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168102620 0x7f8168101f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f8166ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168102620 0x7f8168101f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40782/0 (socket says 172.21.15.16:40782) 2024-09-19T09:28:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f8166ffd640 1 -- 172.21.15.16:0/551870375 learned_addr learned my addr 172.21.15.16:0/551870375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f81677fe640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168107880 0x7f8168100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f81667fc640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8168104f50 0x7f81681005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f8166ffd640 1 -- 172.21.15.16:0/551870375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168107880 msgr2=0x7f8168100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f8166ffd640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168107880 0x7f8168100b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f8166ffd640 1 -- 172.21.15.16:0/551870375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8168104f50 msgr2=0x7f81681005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f8166ffd640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8168104f50 0x7f81681005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f8166ffd640 1 -- 172.21.15.16:0/551870375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81681013a0 con 0x7f8168102620 2024-09-19T09:28:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f8166ffd640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168102620 0x7f8168101f90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f815800c890 tx=0x7f815800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f814ffff640 1 -- 172.21.15.16:0/551870375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158004220 con 0x7f8168102620 2024-09-19T09:28:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f814ffff640 1 -- 172.21.15.16:0/551870375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81580043c0 con 0x7f8168102620 2024-09-19T09:28:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f816d61e640 1 -- 172.21.15.16:0/551870375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81681a1e60 con 0x7f8168102620 2024-09-19T09:28:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f814ffff640 1 -- 172.21.15.16:0/551870375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81580049e0 con 0x7f8168102620 2024-09-19T09:28:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.228+0000 7f816d61e640 1 -- 172.21.15.16:0/551870375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81681a2270 con 0x7f8168102620 2024-09-19T09:28:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f814ffff640 1 -- 172.21.15.16:0/551870375 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f81580079b0 con 0x7f8168102620 2024-09-19T09:28:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f816fffd740 1 Processor -- start 2024-09-19T09:28:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f816fffd740 1 -- start start 2024-09-19T09:28:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f816fffd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad09a0 0x55cd01ad0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f816fffd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad12e0 0x55cd01ad5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f816fffd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad6370 0x55cd01ad8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f816fffd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cd01a342f0 con 0x55cd01ad09a0 2024-09-19T09:28:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f816fffd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cd019dc270 con 0x55cd01ad6370 2024-09-19T09:28:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f816fffd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cd01a302b0 con 0x55cd01ad12e0 2024-09-19T09:28:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f81677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad09a0 0x55cd01ad0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f81677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad09a0 0x55cd01ad0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40794/0 (socket says 172.21.15.16:40794) 2024-09-19T09:28:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f81677fe640 1 -- 172.21.15.16:0/1465591535 learned_addr learned my addr 172.21.15.16:0/1465591535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f8166ffd640 1 --2- 172.21.15.16:0/1465591535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad12e0 0x55cd01ad5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f814ffff640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f81500a32f0 0x7f81500a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.231+0000 7f81667fc640 1 --2- 172.21.15.16:0/1465591535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad6370 0x55cd01ad8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.232+0000 7f81677fe640 1 -- 172.21.15.16:0/1465591535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad12e0 msgr2=0x55cd01ad5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.232+0000 7f81677fe640 1 --2- 172.21.15.16:0/1465591535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad12e0 0x55cd01ad5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.232+0000 7f81677fe640 1 -- 172.21.15.16:0/1465591535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad6370 msgr2=0x55cd01ad8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.232+0000 7f81677fe640 1 --2- 172.21.15.16:0/1465591535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad6370 0x55cd01ad8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.232+0000 7f81677fe640 1 -- 172.21.15.16:0/1465591535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd019db340 con 0x55cd01ad09a0 2024-09-19T09:28:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.232+0000 7f81667fc640 1 --2- 172.21.15.16:0/1465591535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad6370 0x55cd01ad8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.232+0000 7f8166ffd640 1 --2- 172.21.15.16:0/1465591535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad12e0 0x55cd01ad5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.232+0000 7f814ffff640 1 -- 172.21.15.16:0/551870375 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f81580cfa20 con 0x7f8168102620 2024-09-19T09:28:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.232+0000 7f81677fe640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f81500a32f0 0x7f81500a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.232+0000 7f81677fe640 1 --2- 172.21.15.16:0/1465591535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad09a0 0x55cd01ad0da0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f816806a980 tx=0x7f81600099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35610c7e17afd8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.232+0000 7f814d7fa640 1 -- 172.21.15.16:0/1465591535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160004040 con 0x55cd01ad09a0 2024-09-19T09:28:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.232+0000 7f814d7fa640 1 -- 172.21.15.16:0/1465591535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8160036040 con 0x55cd01ad09a0 2024-09-19T09:28:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.232+0000 7f814d7fa640 1 -- 172.21.15.16:0/1465591535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816002fc80 con 0x55cd01ad09a0 2024-09-19T09:28:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 -- 172.21.15.16:0/1465591535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad09a0 msgr2=0x55cd01ad0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 --2- 172.21.15.16:0/1465591535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad09a0 0x55cd01ad0da0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f816806a980 tx=0x7f81600099c0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f81677fe640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f81500a32f0 0x7f81500a57b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8160031c10 tx=0x7f8160049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 -- 172.21.15.16:0/1465591535 shutdown_connections 2024-09-19T09:28:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 --2- 172.21.15.16:0/1465591535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad12e0 0x55cd01ad5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 --2- 172.21.15.16:0/1465591535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad6370 0x55cd01ad8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 --2- 172.21.15.16:0/1465591535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad09a0 0x55cd01ad0da0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 -- 172.21.15.16:0/1465591535 >> 172.21.15.16:0/1465591535 conn(0x55cd01acc010 msgr2=0x55cd01acc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 -- 172.21.15.16:0/1465591535 shutdown_connections 2024-09-19T09:28:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 -- 172.21.15.16:0/1465591535 wait complete. 2024-09-19T09:28:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 -- start start 2024-09-19T09:28:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 Processor -- start 2024-09-19T09:28:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad09a0 0x55cd01adaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad12e0 0x55cd01adaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad6370 0x55cd01ae1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cd01ad91c0 con 0x55cd01ad12e0 2024-09-19T09:28:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cd019bd290 con 0x55cd01ad6370 2024-09-19T09:28:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.233+0000 7f816fffd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cd01ad9340 con 0x55cd01ad09a0 2024-09-19T09:28:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f816fffd740 10 client.0 _ll_register_callbacks cb 0x55cd019ee140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f81667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad12e0 0x55cd01adaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f81667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad12e0 0x55cd01adaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40798/0 (socket says 172.21.15.16:40798) 2024-09-19T09:28:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f81667fc640 1 -- 172.21.15.16:0/3647135420 learned_addr learned my addr 172.21.15.16:0/3647135420 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f8166ffd640 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad09a0 0x55cd01adaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f81677fe640 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad6370 0x55cd01ae1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f81667fc640 1 -- 172.21.15.16:0/3647135420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad09a0 msgr2=0x55cd01adaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f81667fc640 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad09a0 0x55cd01adaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f81667fc640 1 -- 172.21.15.16:0/3647135420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad6370 msgr2=0x55cd01ae1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f81667fc640 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad6370 0x55cd01ae1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f81667fc640 1 -- 172.21.15.16:0/3647135420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd01747210 con 0x55cd01ad12e0 2024-09-19T09:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f8166ffd640 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad09a0 0x55cd01adaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f81667fc640 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad12e0 0x55cd01adaf60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f815c00f3e0 tx=0x7f815c00f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f46281490cac737 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f814aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f814aff5640 1 -- 172.21.15.16:0/3647135420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c004220 con 0x55cd01ad12e0 2024-09-19T09:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f814aff5640 1 -- 172.21.15.16:0/3647135420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f815c0043c0 con 0x55cd01ad12e0 2024-09-19T09:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f816fffd740 1 -- 172.21.15.16:0/3647135420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8160036040 con 0x55cd01ad12e0 2024-09-19T09:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f814aff5640 1 -- 172.21.15.16:0/3647135420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f815c011710 con 0x55cd01ad12e0 2024-09-19T09:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f816fffd740 10 client.15816 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.234+0000 7f816fffd740 1 -- 172.21.15.16:0/3647135420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd01813620 con 0x55cd01ad12e0 2024-09-19T09:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.235+0000 7f814aff5640 1 -- 172.21.15.16:0/3647135420 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8160036040 con 0x55cd01ad12e0 2024-09-19T09:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.235+0000 7f814aff5640 1 -- 172.21.15.16:0/3647135420 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f815c003ca0 con 0x55cd01ad12e0 2024-09-19T09:28:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.235+0000 7f816fffd740 10 client.15816 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.235+0000 7f816fffd740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.235+0000 7f816fffd740 20 client.15816 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.235+0000 7f816fffd740 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cd01ae6610 0x55cd01ae8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.235+0000 7f816fffd740 4 client.15816 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.235+0000 7f816fffd740 1 -- 172.21.15.16:0/3647135420 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd01a41a20 con 0x55cd01ae6610 2024-09-19T09:28:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.235+0000 7f8166ffd640 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cd01ae6610 0x55cd01ae8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.236+0000 7f814aff5640 10 client.15816 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.236+0000 7f8166ffd640 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cd01ae6610 0x55cd01ae8a60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ea807809869f1dc server_cookie=1c1dee70f1ab1e28 in_seq=0 out_seq=0 2024-09-19T09:28:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.236+0000 7f814aff5640 1 -- 172.21.15.16:0/3647135420 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd01a41a20 con 0x55cd01ae6610 2024-09-19T09:28:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.236+0000 7f814aff5640 10 client.15816 handle_command_reply: tid=0 2024-09-19T09:28:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.236+0000 7f816fffd740 1 client.15816 shutdown 2024-09-19T09:28:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.237+0000 7f816fffd740 1 -- 172.21.15.16:0/3647135420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad12e0 msgr2=0x55cd01adaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.237+0000 7f816fffd740 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad12e0 0x55cd01adaf60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f815c00f3e0 tx=0x7f815c00f8a0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.237+0000 7f816fffd740 1 -- 172.21.15.16:0/3647135420 shutdown_connections 2024-09-19T09:28:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.237+0000 7f816fffd740 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cd01ae6610 0x55cd01ae8a60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.237+0000 7f816fffd740 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad6370 0x55cd01ae1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.237+0000 7f816fffd740 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad12e0 0x55cd01adaf60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.237+0000 7f816fffd740 1 --2- 172.21.15.16:0/3647135420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad09a0 0x55cd01adaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.237+0000 7f814aff5640 0 client.15816 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.237+0000 7f816fffd740 1 -- 172.21.15.16:0/3647135420 >> 172.21.15.16:0/3647135420 conn(0x55cd01acc010 msgr2=0x55cd01acd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.237+0000 7f816fffd740 1 -- 172.21.15.16:0/3647135420 shutdown_connections 2024-09-19T09:28:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.237+0000 7f816fffd740 1 -- 172.21.15.16:0/3647135420 wait complete. 2024-09-19T09:28:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.237+0000 7f816fffd740 20 client.15816 trim_cache size 0 max 16384 2024-09-19T09:28:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.251+0000 7f816fffd740 1 Processor -- start 2024-09-19T09:28:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.251+0000 7f816fffd740 1 -- start start 2024-09-19T09:28:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.251+0000 7f816fffd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad09a0 0x55cd01acd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.251+0000 7f816fffd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad12e0 0x55cd01ae1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.251+0000 7f816fffd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 0x55cd01acd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.251+0000 7f816fffd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81681ab230 con 0x55cd01ad6370 2024-09-19T09:28:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.251+0000 7f816fffd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f81681acd70 con 0x55cd01ad09a0 2024-09-19T09:28:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.251+0000 7f816fffd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f81681ab900 con 0x55cd01ad12e0 2024-09-19T09:28:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f81677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 0x55cd01acd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f81677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 0x55cd01acd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40814/0 (socket says 172.21.15.16:40814) 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f81677fe640 1 -- 172.21.15.16:0/3402442812 learned_addr learned my addr 172.21.15.16:0/3402442812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f8166ffd640 1 --2- 172.21.15.16:0/3402442812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad09a0 0x55cd01acd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f81667fc640 1 --2- 172.21.15.16:0/3402442812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad12e0 0x55cd01ae1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f81677fe640 1 -- 172.21.15.16:0/3402442812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad12e0 msgr2=0x55cd01ae1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f81677fe640 1 --2- 172.21.15.16:0/3402442812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad12e0 0x55cd01ae1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f81677fe640 1 -- 172.21.15.16:0/3402442812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad09a0 msgr2=0x55cd01acd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f81677fe640 1 --2- 172.21.15.16:0/3402442812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad09a0 0x55cd01acd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f81677fe640 1 -- 172.21.15.16:0/3402442812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd01ae4fd0 con 0x55cd01ad6370 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f81677fe640 1 --2- 172.21.15.16:0/3402442812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 0x55cd01acd940 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8160009430 tx=0x7f8160009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a4fb2596389b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f814cff9640 1 -- 172.21.15.16:0/3402442812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816002f610 con 0x55cd01ad6370 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f814cff9640 1 -- 172.21.15.16:0/3402442812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8160036030 con 0x55cd01ad6370 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.252+0000 7f814cff9640 1 -- 172.21.15.16:0/3402442812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81600305c0 con 0x55cd01ad6370 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 -- 172.21.15.16:0/3402442812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 msgr2=0x55cd01acd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 --2- 172.21.15.16:0/3402442812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 0x55cd01acd940 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8160009430 tx=0x7f8160009360 comp rx=0 tx=0).stop 2024-09-19T09:28:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 -- 172.21.15.16:0/3402442812 shutdown_connections 2024-09-19T09:28:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 --2- 172.21.15.16:0/3402442812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad12e0 0x55cd01ae1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 --2- 172.21.15.16:0/3402442812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 0x55cd01acd940 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 --2- 172.21.15.16:0/3402442812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad09a0 0x55cd01acd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 -- 172.21.15.16:0/3402442812 >> 172.21.15.16:0/3402442812 conn(0x55cd01acc010 msgr2=0x55cd01acb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 -- 172.21.15.16:0/3402442812 shutdown_connections 2024-09-19T09:28:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 -- 172.21.15.16:0/3402442812 wait complete. 2024-09-19T09:28:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 -- start start 2024-09-19T09:28:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 Processor -- start 2024-09-19T09:28:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad09a0 0x55cd01adf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad12e0 0x55cd01adfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 0x55cd01ae0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81681ac6a0 con 0x55cd01ad6370 2024-09-19T09:28:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f81681abfd0 con 0x55cd01ad12e0 2024-09-19T09:28:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.253+0000 7f816fffd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f81681a91f0 con 0x55cd01ad09a0 2024-09-19T09:28:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f816fffd740 10 client.0 _ll_register_callbacks cb 0x7f815c00a390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f81677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 0x55cd01ae0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f81677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 0x55cd01ae0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40820/0 (socket says 172.21.15.16:40820) 2024-09-19T09:28:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f81667fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad12e0 0x55cd01adfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f81677fe640 1 -- 172.21.15.16:0/1455570728 learned_addr learned my addr 172.21.15.16:0/1455570728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f8166ffd640 1 --2- 172.21.15.16:0/1455570728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad09a0 0x55cd01adf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f81677fe640 1 -- 172.21.15.16:0/1455570728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad09a0 msgr2=0x55cd01adf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f81677fe640 1 --2- 172.21.15.16:0/1455570728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad09a0 0x55cd01adf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f81677fe640 1 -- 172.21.15.16:0/1455570728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad12e0 msgr2=0x55cd01adfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f81677fe640 1 --2- 172.21.15.16:0/1455570728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad12e0 0x55cd01adfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f81677fe640 1 -- 172.21.15.16:0/1455570728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd01ae1330 con 0x55cd01ad6370 2024-09-19T09:28:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f81677fe640 1 --2- 172.21.15.16:0/1455570728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 0x55cd01ae0d20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8160009b20 tx=0x7f8160009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8776450e2d88768a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f814aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f814aff5640 1 -- 172.21.15.16:0/1455570728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160061040 con 0x55cd01ad6370 2024-09-19T09:28:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f816fffd740 1 -- 172.21.15.16:0/1455570728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8160036030 con 0x55cd01ad6370 2024-09-19T09:28:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f814aff5640 1 -- 172.21.15.16:0/1455570728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f8160004600 con 0x55cd01ad6370 2024-09-19T09:28:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f814aff5640 1 -- 172.21.15.16:0/1455570728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8160004800 con 0x55cd01ad6370 2024-09-19T09:28:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.254+0000 7f816fffd740 10 client.15822 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.255+0000 7f816fffd740 1 -- 172.21.15.16:0/1455570728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd01b51510 con 0x55cd01ad6370 2024-09-19T09:28:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.255+0000 7f814aff5640 1 -- 172.21.15.16:0/1455570728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8160036030 con 0x55cd01ad6370 2024-09-19T09:28:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.255+0000 7f814aff5640 1 -- 172.21.15.16:0/1455570728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8160064070 con 0x55cd01ad6370 2024-09-19T09:28:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.255+0000 7f816fffd740 10 client.15822 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.255+0000 7f816fffd740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:28:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.255+0000 7f816fffd740 20 client.15822 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.255+0000 7f816fffd740 1 --2- 172.21.15.16:0/1455570728 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cd01b51bb0 0x55cd01b53fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.255+0000 7f816fffd740 4 client.15822 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:28:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.255+0000 7f816fffd740 1 -- 172.21.15.16:0/1455570728 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd01a3bd70 con 0x55cd01b51bb0 2024-09-19T09:28:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.255+0000 7f8166ffd640 1 --2- 172.21.15.16:0/1455570728 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cd01b51bb0 0x55cd01b53fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.255+0000 7f814aff5640 10 client.15822 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.255+0000 7f8166ffd640 1 --2- 172.21.15.16:0/1455570728 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cd01b51bb0 0x55cd01b53fd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76d079c906cb633 server_cookie=c9f4abc714a3dc2b in_seq=0 out_seq=0 2024-09-19T09:28:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f814aff5640 1 -- 172.21.15.16:0/1455570728 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd01a3bd70 con 0x55cd01b51bb0 2024-09-19T09:28:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f814aff5640 10 client.15822 handle_command_reply: tid=0 2024-09-19T09:28:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f816fffd740 1 client.15822 shutdown 2024-09-19T09:28:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f816fffd740 1 -- 172.21.15.16:0/1455570728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 msgr2=0x55cd01ae0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f816fffd740 1 --2- 172.21.15.16:0/1455570728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 0x55cd01ae0d20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8160009b20 tx=0x7f8160009b50 comp rx=0 tx=0).stop 2024-09-19T09:28:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f816fffd740 1 -- 172.21.15.16:0/1455570728 shutdown_connections 2024-09-19T09:28:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f816fffd740 1 --2- 172.21.15.16:0/1455570728 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cd01b51bb0 0x55cd01b53fd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f816fffd740 1 --2- 172.21.15.16:0/1455570728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cd01ad6370 0x55cd01ae0d20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f816fffd740 1 --2- 172.21.15.16:0/1455570728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd01ad12e0 0x55cd01adfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f814aff5640 0 client.15822 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:28:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f816fffd740 1 --2- 172.21.15.16:0/1455570728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd01ad09a0 0x55cd01adf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f816fffd740 1 -- 172.21.15.16:0/1455570728 >> 172.21.15.16:0/1455570728 conn(0x55cd01acc010 msgr2=0x55cd01acb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f816fffd740 1 -- 172.21.15.16:0/1455570728 shutdown_connections 2024-09-19T09:28:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f816fffd740 1 -- 172.21.15.16:0/1455570728 wait complete. 2024-09-19T09:28:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.265+0000 7f816fffd740 20 client.15822 trim_cache size 0 max 16384 2024-09-19T09:28:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-19T09:28:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-19T09:28:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-19T09:28:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-19T09:28:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.266+0000 7f816d61e640 1 -- 172.21.15.16:0/551870375 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f81500a32f0 msgr2=0x7f81500a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.266+0000 7f816d61e640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f81500a32f0 0x7f81500a57b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8160031c10 tx=0x7f8160049000 comp rx=0 tx=0).stop 2024-09-19T09:28:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.266+0000 7f816d61e640 1 -- 172.21.15.16:0/551870375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168102620 msgr2=0x7f8168101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.266+0000 7f816d61e640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168102620 0x7f8168101f90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f815800c890 tx=0x7f815800cd50 comp rx=0 tx=0).stop 2024-09-19T09:28:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.266+0000 7f816d61e640 1 -- 172.21.15.16:0/551870375 shutdown_connections 2024-09-19T09:28:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.266+0000 7f816d61e640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f81500a32f0 0x7f81500a57b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.266+0000 7f816d61e640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168107880 0x7f8168100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.266+0000 7f816d61e640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8168104f50 0x7f81681005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.266+0000 7f816d61e640 1 --2- 172.21.15.16:0/551870375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168102620 0x7f8168101f90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.266+0000 7f816d61e640 1 -- 172.21.15.16:0/551870375 >> 172.21.15.16:0/551870375 conn(0x7f81680fc1e0 msgr2=0x7f8168102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.266+0000 7f816d61e640 1 -- 172.21.15.16:0/551870375 shutdown_connections 2024-09-19T09:28:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:28:58.266+0000 7f816d61e640 1 -- 172.21.15.16:0/551870375 wait complete. 2024-09-19T09:28:58.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:28:59.211 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:28:59.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:28:59 smithi142 ceph-mon[35032]: pgmap v379: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 689 KiB/s wr, 3.04k op/s 2024-09-19T09:28:59.380 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:28:59 smithi016 ceph-mon[30979]: pgmap v379: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 689 KiB/s wr, 3.04k op/s 2024-09-19T09:28:59.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:28:59 smithi097 ceph-mon[34864]: pgmap v379: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 689 KiB/s wr, 3.04k op/s 2024-09-19T09:28:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.707+0000 7fc50a92c640 1 -- 172.21.15.16:0/1945270635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc504102d40 msgr2=0x7fc504103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.707+0000 7fc50a92c640 1 --2- 172.21.15.16:0/1945270635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc504102d40 0x7fc504103120 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009900 tx=0x7fc4f402a480 comp rx=0 tx=0).stop 2024-09-19T09:28:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.708+0000 7fc50a92c640 1 -- 172.21.15.16:0/1945270635 shutdown_connections 2024-09-19T09:28:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.708+0000 7fc50a92c640 1 --2- 172.21.15.16:0/1945270635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc504103660 0x7fc50410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.708+0000 7fc50a92c640 1 --2- 172.21.15.16:0/1945270635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc50410c170 0x7fc50410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.708+0000 7fc50a92c640 1 --2- 172.21.15.16:0/1945270635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc504102d40 0x7fc504103120 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.708+0000 7fc50a92c640 1 -- 172.21.15.16:0/1945270635 >> 172.21.15.16:0/1945270635 conn(0x7fc5040fc780 msgr2=0x7fc5040feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.708+0000 7fc50a92c640 1 -- 172.21.15.16:0/1945270635 shutdown_connections 2024-09-19T09:28:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.708+0000 7fc50a92c640 1 -- 172.21.15.16:0/1945270635 wait complete. 2024-09-19T09:28:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.708+0000 7fc50a92c640 1 Processor -- start 2024-09-19T09:28:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.708+0000 7fc50a92c640 1 -- start start 2024-09-19T09:28:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc50a92c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc504102d40 0x7fc504198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc50a92c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc504103660 0x7fc504198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc50a92c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc50410c170 0x7fc50419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc50a92c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc504112030 con 0x7fc504103660 2024-09-19T09:28:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc50a92c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc504111eb0 con 0x7fc50410c170 2024-09-19T09:28:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc50a92c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc5041121b0 con 0x7fc504102d40 2024-09-19T09:28:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc5037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc504103660 0x7fc504198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc503fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc504102d40 0x7fc504198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc5037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc504103660 0x7fc504198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40852/0 (socket says 172.21.15.16:40852) 2024-09-19T09:28:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc5037fe640 1 -- 172.21.15.16:0/2871695394 learned_addr learned my addr 172.21.15.16:0/2871695394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc508ec9640 1 --2- 172.21.15.16:0/2871695394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc50410c170 0x7fc50419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc5037fe640 1 -- 172.21.15.16:0/2871695394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc504102d40 msgr2=0x7fc504198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc5037fe640 1 --2- 172.21.15.16:0/2871695394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc504102d40 0x7fc504198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc5037fe640 1 -- 172.21.15.16:0/2871695394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc50410c170 msgr2=0x7fc50419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc5037fe640 1 --2- 172.21.15.16:0/2871695394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc50410c170 0x7fc50419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc5037fe640 1 -- 172.21.15.16:0/2871695394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50419d540 con 0x7fc504103660 2024-09-19T09:28:59.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.709+0000 7fc5037fe640 1 --2- 172.21.15.16:0/2871695394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc504103660 0x7fc504198ad0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800b3f0 tx=0x7fc4f800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:59.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.710+0000 7fc5017fa640 1 -- 172.21.15.16:0/2871695394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8015020 con 0x7fc504103660 2024-09-19T09:28:59.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.710+0000 7fc50a92c640 1 -- 172.21.15.16:0/2871695394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50419d830 con 0x7fc504103660 2024-09-19T09:28:59.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.710+0000 7fc5017fa640 1 -- 172.21.15.16:0/2871695394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f8011d10 con 0x7fc504103660 2024-09-19T09:28:59.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.710+0000 7fc5017fa640 1 -- 172.21.15.16:0/2871695394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f800f2e0 con 0x7fc504103660 2024-09-19T09:28:59.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.710+0000 7fc50a92c640 1 -- 172.21.15.16:0/2871695394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5041a5110 con 0x7fc504103660 2024-09-19T09:28:59.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.712+0000 7fc5017fa640 1 -- 172.21.15.16:0/2871695394 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fc4f8018020 con 0x7fc504103660 2024-09-19T09:28:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.712+0000 7fc50cd05740 1 Processor -- start 2024-09-19T09:28:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc50cd05740 1 -- start start 2024-09-19T09:28:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc50cd05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec096e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc50cd05740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec096ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc50cd05740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec097b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc50cd05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fec08d42f0 con 0x55fec096fb40 2024-09-19T09:28:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc50cd05740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fec087bcf0 con 0x55fec096e5d0 2024-09-19T09:28:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc50cd05740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fec08d02b0 con 0x55fec0979370 2024-09-19T09:28:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc508ec9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec096e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc508ec9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec096e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40856/0 (socket says 172.21.15.16:40856) 2024-09-19T09:28:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc508ec9640 1 -- 172.21.15.16:0/4102069902 learned_addr learned my addr 172.21.15.16:0/4102069902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc5017fa640 1 --2- 172.21.15.16:0/2871695394 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc4ec0a3240 0x7fc4ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc503fff640 1 --2- 172.21.15.16:0/4102069902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec096ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc5037fe640 1 --2- 172.21.15.16:0/4102069902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec097b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc5017fa640 1 -- 172.21.15.16:0/2871695394 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc4f80cf100 con 0x7fc504103660 2024-09-19T09:28:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc508ec9640 1 -- 172.21.15.16:0/4102069902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 msgr2=0x55fec097b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc508ec9640 1 --2- 172.21.15.16:0/4102069902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec097b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc508ec9640 1 -- 172.21.15.16:0/4102069902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 msgr2=0x55fec096ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc508ec9640 1 --2- 172.21.15.16:0/4102069902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec096ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.713+0000 7fc508ec9640 1 -- 172.21.15.16:0/4102069902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fec08603a0 con 0x55fec096fb40 2024-09-19T09:28:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.714+0000 7fc508ec9640 1 --2- 172.21.15.16:0/2871695394 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc4ec0a3240 0x7fc4ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.714+0000 7fc508ec9640 1 --2- 172.21.15.16:0/4102069902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec096e090 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00a500 tx=0x7fc4fc00a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad10929981f3c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.714+0000 7fc4ea7fc640 1 -- 172.21.15.16:0/4102069902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc01a040 con 0x55fec096fb40 2024-09-19T09:28:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.714+0000 7fc4ea7fc640 1 -- 172.21.15.16:0/4102069902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4fc021ba0 con 0x55fec096fb40 2024-09-19T09:28:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.714+0000 7fc4ea7fc640 1 -- 172.21.15.16:0/4102069902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc0203b0 con 0x55fec096fb40 2024-09-19T09:28:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.715+0000 7fc50cd05740 1 -- 172.21.15.16:0/4102069902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 msgr2=0x55fec096e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.715+0000 7fc508ec9640 1 --2- 172.21.15.16:0/2871695394 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc4ec0a3240 0x7fc4ec0a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc020000 tx=0x7fc4fc021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.715+0000 7fc50cd05740 1 --2- 172.21.15.16:0/4102069902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec096e090 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00a500 tx=0x7fc4fc00a9c0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.715+0000 7fc50cd05740 1 -- 172.21.15.16:0/4102069902 shutdown_connections 2024-09-19T09:28:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.715+0000 7fc50cd05740 1 --2- 172.21.15.16:0/4102069902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec097b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.715+0000 7fc50cd05740 1 --2- 172.21.15.16:0/4102069902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec096ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.715+0000 7fc50cd05740 1 --2- 172.21.15.16:0/4102069902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec096e090 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.715+0000 7fc50cd05740 1 -- 172.21.15.16:0/4102069902 >> 172.21.15.16:0/4102069902 conn(0x55fec0937310 msgr2=0x55fec096ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.715+0000 7fc50cd05740 1 -- 172.21.15.16:0/4102069902 shutdown_connections 2024-09-19T09:28:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.715+0000 7fc50cd05740 1 -- 172.21.15.16:0/4102069902 wait complete. 2024-09-19T09:28:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.715+0000 7fc50cd05740 1 -- start start 2024-09-19T09:28:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.715+0000 7fc50cd05740 1 Processor -- start 2024-09-19T09:28:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc50cd05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096e5d0 0x55fec0978d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc50cd05740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec096fb40 0x55fec0976000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc50cd05740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec0979370 0x55fec09765f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc50cd05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fec097c070 con 0x55fec096e5d0 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc50cd05740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fec088f060 con 0x55fec0979370 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc50cd05740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55fec097c1f0 con 0x55fec096fb40 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc50cd05740 10 client.0 _ll_register_callbacks cb 0x55fec05e28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc503fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096e5d0 0x55fec0978d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc503fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096e5d0 0x55fec0978d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40858/0 (socket says 172.21.15.16:40858) 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc503fff640 1 -- 172.21.15.16:0/4032057801 learned_addr learned my addr 172.21.15.16:0/4032057801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc5037fe640 1 --2- 172.21.15.16:0/4032057801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec096fb40 0x55fec0976000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc508ec9640 1 --2- 172.21.15.16:0/4032057801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec0979370 0x55fec09765f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc503fff640 1 -- 172.21.15.16:0/4032057801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec096fb40 msgr2=0x55fec0976000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc503fff640 1 --2- 172.21.15.16:0/4032057801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec096fb40 0x55fec0976000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc503fff640 1 -- 172.21.15.16:0/4032057801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec0979370 msgr2=0x55fec09765f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc503fff640 1 --2- 172.21.15.16:0/4032057801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec0979370 0x55fec09765f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc503fff640 1 -- 172.21.15.16:0/4032057801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fec06453c0 con 0x55fec096e5d0 2024-09-19T09:28:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc503fff640 1 --2- 172.21.15.16:0/4032057801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096e5d0 0x55fec0978d00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009900 tx=0x7fc4f4031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c160e7fe7cc0f76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.716+0000 7fc4e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc4e37fe640 1 -- 172.21.15.16:0/4032057801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4035040 con 0x55fec096e5d0 2024-09-19T09:28:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc4e37fe640 1 -- 172.21.15.16:0/4032057801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc4f4004380 con 0x55fec096e5d0 2024-09-19T09:28:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc50cd05740 1 -- 172.21.15.16:0/4032057801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4fc021ba0 con 0x55fec096e5d0 2024-09-19T09:28:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc4e37fe640 1 -- 172.21.15.16:0/4032057801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4f4005020 con 0x55fec096e5d0 2024-09-19T09:28:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc50cd05740 10 client.15834 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc50cd05740 1 -- 172.21.15.16:0/4032057801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fec0987ae0 con 0x55fec096e5d0 2024-09-19T09:28:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc4e37fe640 1 -- 172.21.15.16:0/4032057801 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4fc021ba0 con 0x55fec096e5d0 2024-09-19T09:28:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc4e37fe640 1 -- 172.21.15.16:0/4032057801 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc4f40035f0 con 0x55fec096e5d0 2024-09-19T09:28:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc50cd05740 10 client.15834 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc50cd05740 10 client.15834 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T09:28:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc50cd05740 20 client.15834 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc50cd05740 1 --2- 172.21.15.16:0/4032057801 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55fec0988ff0 0x55fec098b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc50cd05740 4 client.15834 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:28:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.717+0000 7fc50cd05740 1 -- 172.21.15.16:0/4032057801 --> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fec08e1860 con 0x55fec0988ff0 2024-09-19T09:28:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.718+0000 7fc5037fe640 1 --2- 172.21.15.16:0/4032057801 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55fec0988ff0 0x55fec098b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.718+0000 7fc5037fe640 1 --2- 172.21.15.16:0/4032057801 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55fec0988ff0 0x55fec098b3e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=911599d5fdf95a39 server_cookie=235fab5a2e573d99 in_seq=0 out_seq=0 2024-09-19T09:28:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.718+0000 7fc4e37fe640 10 client.15834 ms_handle_connect on v2:172.21.15.97:6836/2018284957 2024-09-19T09:28:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc4e37fe640 1 -- 172.21.15.16:0/4032057801 <== mds.2 v2:172.21.15.97:6836/2018284957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fec08e1860 con 0x55fec0988ff0 2024-09-19T09:28:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc4e37fe640 10 client.15834 handle_command_reply: tid=0 2024-09-19T09:28:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc50cd05740 1 client.15834 shutdown 2024-09-19T09:28:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc50cd05740 1 -- 172.21.15.16:0/4032057801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096e5d0 msgr2=0x55fec0978d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc50cd05740 1 --2- 172.21.15.16:0/4032057801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096e5d0 0x55fec0978d00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009900 tx=0x7fc4f4031a60 comp rx=0 tx=0).stop 2024-09-19T09:28:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc50cd05740 1 -- 172.21.15.16:0/4032057801 shutdown_connections 2024-09-19T09:28:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc50cd05740 1 --2- 172.21.15.16:0/4032057801 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55fec0988ff0 0x55fec098b3e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc50cd05740 1 --2- 172.21.15.16:0/4032057801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec096fb40 0x55fec0976000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc50cd05740 1 --2- 172.21.15.16:0/4032057801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec0979370 0x55fec09765f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc50cd05740 1 --2- 172.21.15.16:0/4032057801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096e5d0 0x55fec0978d00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc50cd05740 1 -- 172.21.15.16:0/4032057801 >> 172.21.15.16:0/4032057801 conn(0x55fec0937310 msgr2=0x55fec0970960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc4e37fe640 0 client.15834 ms_handle_reset on v2:172.21.15.97:6836/2018284957 2024-09-19T09:28:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc50cd05740 1 -- 172.21.15.16:0/4032057801 shutdown_connections 2024-09-19T09:28:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc50cd05740 1 -- 172.21.15.16:0/4032057801 wait complete. 2024-09-19T09:28:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.719+0000 7fc50cd05740 20 client.15834 trim_cache size 0 max 16384 2024-09-19T09:28:59.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.732+0000 7fc50cd05740 1 Processor -- start 2024-09-19T09:28:59.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.732+0000 7fc50cd05740 1 -- start start 2024-09-19T09:28:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc50cd05740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec0985310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc50cd05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec0990cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc50cd05740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec09911f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc50cd05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5041ad600 con 0x55fec096fb40 2024-09-19T09:28:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc50cd05740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc5041af140 con 0x55fec096e5d0 2024-09-19T09:28:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc50cd05740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc5041adcd0 con 0x55fec0979370 2024-09-19T09:28:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc5037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec0990cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc5037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec0990cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40864/0 (socket says 172.21.15.16:40864) 2024-09-19T09:28:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc5037fe640 1 -- 172.21.15.16:0/3220199698 learned_addr learned my addr 172.21.15.16:0/3220199698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc503fff640 1 --2- 172.21.15.16:0/3220199698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec0985310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc5037fe640 1 -- 172.21.15.16:0/3220199698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 msgr2=0x55fec09911f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:59.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc508ec9640 1 --2- 172.21.15.16:0/3220199698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec09911f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc5037fe640 1 --2- 172.21.15.16:0/3220199698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec09911f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc5037fe640 1 -- 172.21.15.16:0/3220199698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 msgr2=0x55fec0985310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc5037fe640 1 --2- 172.21.15.16:0/3220199698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec0985310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc5037fe640 1 -- 172.21.15.16:0/3220199698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fec0979070 con 0x55fec096fb40 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc503fff640 1 --2- 172.21.15.16:0/3220199698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec0985310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc508ec9640 1 --2- 172.21.15.16:0/3220199698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec09911f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc5037fe640 1 --2- 172.21.15.16:0/3220199698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec0990cb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800b3c0 tx=0x7fc4f806e560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c0e78e0242eab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc4e9ffb640 1 -- 172.21.15.16:0/3220199698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f809b6e0 con 0x55fec096fb40 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc4e9ffb640 1 -- 172.21.15.16:0/3220199698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f809fa90 con 0x55fec096fb40 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.733+0000 7fc4e9ffb640 1 -- 172.21.15.16:0/3220199698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8097d50 con 0x55fec096fb40 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.734+0000 7fc50cd05740 1 -- 172.21.15.16:0/3220199698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 msgr2=0x55fec0990cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.734+0000 7fc50cd05740 1 --2- 172.21.15.16:0/3220199698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec0990cb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800b3c0 tx=0x7fc4f806e560 comp rx=0 tx=0).stop 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.734+0000 7fc50cd05740 1 -- 172.21.15.16:0/3220199698 shutdown_connections 2024-09-19T09:28:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.734+0000 7fc50cd05740 1 --2- 172.21.15.16:0/3220199698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec09911f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.734+0000 7fc50cd05740 1 --2- 172.21.15.16:0/3220199698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec0990cb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.734+0000 7fc50cd05740 1 --2- 172.21.15.16:0/3220199698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec0985310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.734+0000 7fc50cd05740 1 -- 172.21.15.16:0/3220199698 >> 172.21.15.16:0/3220199698 conn(0x55fec0937310 msgr2=0x55fec0971290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.734+0000 7fc50cd05740 1 -- 172.21.15.16:0/3220199698 shutdown_connections 2024-09-19T09:28:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.734+0000 7fc50cd05740 1 -- 172.21.15.16:0/3220199698 wait complete. 2024-09-19T09:28:59.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc50cd05740 1 -- start start 2024-09-19T09:28:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc50cd05740 1 Processor -- start 2024-09-19T09:28:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc50cd05740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec097d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc50cd05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec097d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc50cd05740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec097db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc50cd05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5041aea70 con 0x55fec096fb40 2024-09-19T09:28:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc50cd05740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc5041ae3a0 con 0x55fec096e5d0 2024-09-19T09:28:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc50cd05740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc5041ab5c0 con 0x55fec0979370 2024-09-19T09:28:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc50cd05740 10 client.0 _ll_register_callbacks cb 0x7fc4f4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:28:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc5037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec097d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc5037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec097d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40874/0 (socket says 172.21.15.16:40874) 2024-09-19T09:28:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc5037fe640 1 -- 172.21.15.16:0/2769375516 learned_addr learned my addr 172.21.15.16:0/2769375516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:28:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc503fff640 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec097d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.735+0000 7fc508ec9640 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec097db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc5037fe640 1 -- 172.21.15.16:0/2769375516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 msgr2=0x55fec097db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc5037fe640 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec097db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc5037fe640 1 -- 172.21.15.16:0/2769375516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 msgr2=0x55fec097d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:28:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc5037fe640 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec097d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc5037fe640 1 -- 172.21.15.16:0/2769375516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fec09f2c80 con 0x55fec096fb40 2024-09-19T09:28:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc503fff640 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec097d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:28:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc508ec9640 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec097db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:28:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc5037fe640 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec097d640 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc4f809b3e0 tx=0x7fc4f80cf430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff103da90a490b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:28:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc4e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:28:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc4e37fe640 1 -- 172.21.15.16:0/2769375516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80b4060 con 0x55fec096fb40 2024-09-19T09:28:59.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc4e37fe640 1 -- 172.21.15.16:0/2769375516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fc4f8097d50 con 0x55fec096fb40 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc50cd05740 1 -- 172.21.15.16:0/2769375516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f809fa90 con 0x55fec096fb40 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc4e37fe640 1 -- 172.21.15.16:0/2769375516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4f809b6e0 con 0x55fec096fb40 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc50cd05740 10 client.15840 fetch_fsmap learned FSMap version 28 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc50cd05740 1 -- 172.21.15.16:0/2769375516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fec09f3df0 con 0x55fec096fb40 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc4e37fe640 1 -- 172.21.15.16:0/2769375516 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f809fa90 con 0x55fec096fb40 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc4e37fe640 1 -- 172.21.15.16:0/2769375516 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc4f806e170 con 0x55fec096fb40 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc50cd05740 10 client.15840 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc50cd05740 10 client.15840 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc50cd05740 20 client.15840 populate_metadata read hostname 'smithi016' 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.736+0000 7fc50cd05740 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55fec09f4470 0x55fec09f6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.737+0000 7fc50cd05740 4 client.15840 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.737+0000 7fc50cd05740 1 -- 172.21.15.16:0/2769375516 --> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fec08dbd70 con 0x55fec09f4470 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.737+0000 7fc503fff640 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55fec09f4470 0x55fec09f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:28:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.737+0000 7fc503fff640 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55fec09f4470 0x55fec09f6890 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8384f93e8ffd5a04 server_cookie=fc00e72b4e91e9b4 in_seq=0 out_seq=0 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.737+0000 7fc4e37fe640 10 client.15840 ms_handle_connect on v2:172.21.15.97:6836/2018284957 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.737+0000 7fc4e37fe640 1 -- 172.21.15.16:0/2769375516 <== mds.2 v2:172.21.15.97:6836/2018284957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fec08dbd70 con 0x55fec09f4470 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.737+0000 7fc4e37fe640 10 client.15840 handle_command_reply: tid=0 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50cd05740 1 client.15840 shutdown 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50cd05740 1 -- 172.21.15.16:0/2769375516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 msgr2=0x55fec097d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50cd05740 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec097d640 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc4f809b3e0 tx=0x7fc4f80cf430 comp rx=0 tx=0).stop 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50cd05740 1 -- 172.21.15.16:0/2769375516 shutdown_connections 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50cd05740 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55fec09f4470 0x55fec09f6890 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50cd05740 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fec0979370 0x55fec097db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50cd05740 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec096fb40 0x55fec097d640 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50cd05740 1 --2- 172.21.15.16:0/2769375516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fec096e5d0 0x55fec097d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50cd05740 1 -- 172.21.15.16:0/2769375516 >> 172.21.15.16:0/2769375516 conn(0x55fec0937310 msgr2=0x55fec0970110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc4e37fe640 0 client.15840 ms_handle_reset on v2:172.21.15.97:6836/2018284957 2024-09-19T09:28:59.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50cd05740 1 -- 172.21.15.16:0/2769375516 shutdown_connections 2024-09-19T09:28:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50cd05740 1 -- 172.21.15.16:0/2769375516 wait complete. 2024-09-19T09:28:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50cd05740 20 client.15840 trim_cache size 0 max 16384 2024-09-19T09:28:59.745 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:28:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50a92c640 1 -- 172.21.15.16:0/2871695394 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc4ec0a3240 msgr2=0x7fc4ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50a92c640 1 --2- 172.21.15.16:0/2871695394 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc4ec0a3240 0x7fc4ec0a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc020000 tx=0x7fc4fc021000 comp rx=0 tx=0).stop 2024-09-19T09:28:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50a92c640 1 -- 172.21.15.16:0/2871695394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc504103660 msgr2=0x7fc504198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:28:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.738+0000 7fc50a92c640 1 --2- 172.21.15.16:0/2871695394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc504103660 0x7fc504198ad0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800b3f0 tx=0x7fc4f800b8b0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.739+0000 7fc50a92c640 1 -- 172.21.15.16:0/2871695394 shutdown_connections 2024-09-19T09:28:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.739+0000 7fc50a92c640 1 --2- 172.21.15.16:0/2871695394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc50410c170 0x7fc50419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.739+0000 7fc50a92c640 1 --2- 172.21.15.16:0/2871695394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc504103660 0x7fc504198ad0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.739+0000 7fc50a92c640 1 --2- 172.21.15.16:0/2871695394 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fc4ec0a3240 0x7fc4ec0a5700 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.739+0000 7fc50a92c640 1 --2- 172.21.15.16:0/2871695394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc504102d40 0x7fc504198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:28:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.739+0000 7fc50a92c640 1 -- 172.21.15.16:0/2871695394 >> 172.21.15.16:0/2871695394 conn(0x7fc5040fc780 msgr2=0x7fc5040fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:28:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.739+0000 7fc50a92c640 1 -- 172.21.15.16:0/2871695394 shutdown_connections 2024-09-19T09:28:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:28:59.739+0000 7fc50a92c640 1 -- 172.21.15.16:0/2871695394 wait complete. 2024-09-19T09:29:00.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:29:00.679 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:00.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:29:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.201+0000 7fb7d94d0640 1 -- 172.21.15.16:0/679125100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7d4100e20 msgr2=0x7fb7d4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.201+0000 7fb7d94d0640 1 --2- 172.21.15.16:0/679125100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7d4100e20 0x7fb7d4101200 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009900 tx=0x7fb7c402a480 comp rx=0 tx=0).stop 2024-09-19T09:29:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.202+0000 7fb7d94d0640 1 -- 172.21.15.16:0/679125100 shutdown_connections 2024-09-19T09:29:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.202+0000 7fb7d94d0640 1 --2- 172.21.15.16:0/679125100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7d4102570 0x7fb7d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.202+0000 7fb7d94d0640 1 --2- 172.21.15.16:0/679125100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d4102ef0 0x7fb7d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.202+0000 7fb7d94d0640 1 --2- 172.21.15.16:0/679125100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7d4100e20 0x7fb7d4101200 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.202+0000 7fb7d94d0640 1 -- 172.21.15.16:0/679125100 >> 172.21.15.16:0/679125100 conn(0x7fb7d40fc7d0 msgr2=0x7fb7d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.202+0000 7fb7d94d0640 1 -- 172.21.15.16:0/679125100 shutdown_connections 2024-09-19T09:29:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.202+0000 7fb7d94d0640 1 -- 172.21.15.16:0/679125100 wait complete. 2024-09-19T09:29:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.202+0000 7fb7d94d0640 1 Processor -- start 2024-09-19T09:29:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.202+0000 7fb7d94d0640 1 -- start start 2024-09-19T09:29:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d94d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d4100e20 0x7fb7d4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d94d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7d4102570 0x7fb7d41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d94d0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7d4102ef0 0x7fb7d419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d94d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4110f80 con 0x7fb7d4102570 2024-09-19T09:29:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d94d0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4110e00 con 0x7fb7d4100e20 2024-09-19T09:29:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d94d0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4111100 con 0x7fb7d4102ef0 2024-09-19T09:29:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7d4102570 0x7fb7d41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7d4102570 0x7fb7d41989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40886/0 (socket says 172.21.15.16:40886) 2024-09-19T09:29:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d27fc640 1 -- 172.21.15.16:0/1518264294 learned_addr learned my addr 172.21.15.16:0/1518264294 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d2ffd640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d4100e20 0x7fb7d4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d37fe640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7d4102ef0 0x7fb7d419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d27fc640 1 -- 172.21.15.16:0/1518264294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7d4102ef0 msgr2=0x7fb7d419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d27fc640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7d4102ef0 0x7fb7d419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d27fc640 1 -- 172.21.15.16:0/1518264294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d4100e20 msgr2=0x7fb7d4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d27fc640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d4100e20 0x7fb7d4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d27fc640 1 -- 172.21.15.16:0/1518264294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d419d440 con 0x7fb7d4102570 2024-09-19T09:29:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.203+0000 7fb7d27fc640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7d4102570 0x7fb7d41989d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800c920 tx=0x7fb7c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.204+0000 7fb7bbfff640 1 -- 172.21.15.16:0/1518264294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8007c70 con 0x7fb7d4102570 2024-09-19T09:29:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.204+0000 7fb7bbfff640 1 -- 172.21.15.16:0/1518264294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c8007e10 con 0x7fb7d4102570 2024-09-19T09:29:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.204+0000 7fb7d94d0640 1 -- 172.21.15.16:0/1518264294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d419d730 con 0x7fb7d4102570 2024-09-19T09:29:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.204+0000 7fb7bbfff640 1 -- 172.21.15.16:0/1518264294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8018890 con 0x7fb7d4102570 2024-09-19T09:29:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.204+0000 7fb7d94d0640 1 -- 172.21.15.16:0/1518264294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d41a5010 con 0x7fb7d4102570 2024-09-19T09:29:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.206+0000 7fb7bbfff640 1 -- 172.21.15.16:0/1518264294 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb7c8018a70 con 0x7fb7d4102570 2024-09-19T09:29:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.206+0000 7fb7db8a9740 1 Processor -- start 2024-09-19T09:29:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7db8a9740 1 -- start start 2024-09-19T09:29:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7db8a9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2fb40 0x55f892a2e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7db8a9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2e5d0 0x55f892a2ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7db8a9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a3d780 0x55f892a3fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7db8a9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f8929942f0 con 0x55f892a3d780 2024-09-19T09:29:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7db8a9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f89293bcf0 con 0x55f892a2fb40 2024-09-19T09:29:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7db8a9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f8929902b0 con 0x55f892a2e5d0 2024-09-19T09:29:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7d37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2fb40 0x55f892a2e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7d37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2fb40 0x55f892a2e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:41880/0 (socket says 172.21.15.16:41880) 2024-09-19T09:29:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7bbfff640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7bc0a32c0 0x7fb7bc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7d37fe640 1 -- 172.21.15.16:0/4061818377 learned_addr learned my addr 172.21.15.16:0/4061818377 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7d27fc640 1 --2- 172.21.15.16:0/4061818377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a3d780 0x55f892a3fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7d2ffd640 1 --2- 172.21.15.16:0/4061818377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2e5d0 0x55f892a2ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7d37fe640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7bc0a32c0 0x7fb7bc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.207+0000 7fb7bbfff640 1 -- 172.21.15.16:0/1518264294 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb7c80cf950 con 0x7fb7d4102570 2024-09-19T09:29:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.208+0000 7fb7d2ffd640 1 -- 172.21.15.16:0/4061818377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2fb40 msgr2=0x55f892a2e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.208+0000 7fb7d2ffd640 1 --2- 172.21.15.16:0/4061818377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2fb40 0x55f892a2e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.208+0000 7fb7d2ffd640 1 -- 172.21.15.16:0/4061818377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a3d780 msgr2=0x55f892a3fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.208+0000 7fb7d2ffd640 1 --2- 172.21.15.16:0/4061818377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a3d780 0x55f892a3fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.208+0000 7fb7d2ffd640 1 -- 172.21.15.16:0/4061818377 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8929203a0 con 0x55f892a2e5d0 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.208+0000 7fb7d2ffd640 1 --2- 172.21.15.16:0/4061818377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2e5d0 0x55f892a2ea10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4005940 tx=0x7fb7c4004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f936c3e51016a0f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.208+0000 7fb7d37fe640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7bc0a32c0 0x7fb7bc0a5780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009fd0 tx=0x7fb7cc004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.208+0000 7fb7b97fa640 1 -- 172.21.15.16:0/4061818377 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4035030 con 0x55f892a2e5d0 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.208+0000 7fb7b97fa640 1 -- 172.21.15.16:0/4061818377 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c4040070 con 0x55f892a2e5d0 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.208+0000 7fb7b97fa640 1 -- 172.21.15.16:0/4061818377 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4031b50 con 0x55f892a2e5d0 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.209+0000 7fb7db8a9740 1 -- 172.21.15.16:0/4061818377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2e5d0 msgr2=0x55f892a2ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.209+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/4061818377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2e5d0 0x55f892a2ea10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4005940 tx=0x7fb7c4004170 comp rx=0 tx=0).stop 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.209+0000 7fb7db8a9740 1 -- 172.21.15.16:0/4061818377 shutdown_connections 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.209+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/4061818377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2e5d0 0x55f892a2ea10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.209+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/4061818377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a3d780 0x55f892a3fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.209+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/4061818377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2fb40 0x55f892a2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.209+0000 7fb7db8a9740 1 -- 172.21.15.16:0/4061818377 >> 172.21.15.16:0/4061818377 conn(0x55f8929f7310 msgr2=0x55f892a2ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.209+0000 7fb7db8a9740 1 -- 172.21.15.16:0/4061818377 shutdown_connections 2024-09-19T09:29:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.209+0000 7fb7db8a9740 1 -- 172.21.15.16:0/4061818377 wait complete. 2024-09-19T09:29:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.210+0000 7fb7db8a9740 1 -- start start 2024-09-19T09:29:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.210+0000 7fb7db8a9740 1 Processor -- start 2024-09-19T09:29:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7db8a9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2e5d0 0x55f892a3d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7db8a9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2fb40 0x55f892a3a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7db8a9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a3d780 0x55f892a3abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7db8a9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f892a40420 con 0x55f892a3d780 2024-09-19T09:29:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7db8a9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f89294f060 con 0x55f892a2e5d0 2024-09-19T09:29:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7db8a9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f892a405a0 con 0x55f892a2fb40 2024-09-19T09:29:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7db8a9740 10 client.0 _ll_register_callbacks cb 0x55f8926a28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7d27fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2fb40 0x55f892a3a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a3d780 0x55f892a3abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7d27fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2fb40 0x55f892a3a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:55358/0 (socket says 172.21.15.16:55358) 2024-09-19T09:29:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7d27fc640 1 -- 172.21.15.16:0/346936046 learned_addr learned my addr 172.21.15.16:0/346936046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7d2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2e5d0 0x55f892a3d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a3d780 0x55f892a3abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40910/0 (socket says 172.21.15.16:40910) 2024-09-19T09:29:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7d27fc640 1 -- 172.21.15.16:0/346936046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2e5d0 msgr2=0x55f892a3d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7d27fc640 1 --2- 172.21.15.16:0/346936046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2e5d0 0x55f892a3d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7d27fc640 1 -- 172.21.15.16:0/346936046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a3d780 msgr2=0x55f892a3abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7d27fc640 1 --2- 172.21.15.16:0/346936046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a3d780 0x55f892a3abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7d27fc640 1 -- 172.21.15.16:0/346936046 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8927053c0 con 0x55f892a2fb40 2024-09-19T09:29:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7d2ffd640 1 --2- 172.21.15.16:0/346936046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2e5d0 0x55f892a3d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7d27fc640 1 --2- 172.21.15.16:0/346936046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2fb40 0x55f892a3a4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8098380 tx=0x7fb7c8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b31c48a53f60d15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.211+0000 7fb7b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7b6ff5640 1 -- 172.21.15.16:0/346936046 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80b6040 con 0x55f892a2fb40 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7b6ff5640 1 -- 172.21.15.16:0/346936046 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb7c8018d60 con 0x55f892a2fb40 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7db8a9740 1 -- 172.21.15.16:0/346936046 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c4040070 con 0x55f892a2fb40 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7b6ff5640 1 -- 172.21.15.16:0/346936046 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7c80a0320 con 0x55f892a2fb40 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7db8a9740 10 client.25369 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7db8a9740 1 -- 172.21.15.16:0/346936046 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f892a4bfb0 con 0x55f892a2fb40 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7b6ff5640 1 -- 172.21.15.16:0/346936046 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c4040070 con 0x55f892a2fb40 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7b6ff5640 1 -- 172.21.15.16:0/346936046 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb7c80a4750 con 0x55f892a2fb40 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7db8a9740 10 client.25369 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7db8a9740 10 client.25369 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7db8a9740 20 client.25369 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/346936046 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55f892a4d470 0x55f892a4f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7db8a9740 4 client.25369 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.212+0000 7fb7db8a9740 1 -- 172.21.15.16:0/346936046 --> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8929a1860 con 0x55f892a4d470 2024-09-19T09:29:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.213+0000 7fb7d2ffd640 1 --2- 172.21.15.16:0/346936046 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55f892a4d470 0x55f892a4f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.213+0000 7fb7d2ffd640 1 --2- 172.21.15.16:0/346936046 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55f892a4d470 0x55f892a4f860 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9af1396e3a23a3c server_cookie=a378c5b90e198759 in_seq=0 out_seq=0 2024-09-19T09:29:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.213+0000 7fb7b6ff5640 10 client.25369 ms_handle_connect on v2:172.21.15.142:6838/3822134597 2024-09-19T09:29:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7b6ff5640 1 -- 172.21.15.16:0/346936046 <== mds.1 v2:172.21.15.142:6838/3822134597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8929a1860 con 0x55f892a4d470 2024-09-19T09:29:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7b6ff5640 10 client.25369 handle_command_reply: tid=0 2024-09-19T09:29:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7db8a9740 1 client.25369 shutdown 2024-09-19T09:29:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7db8a9740 1 -- 172.21.15.16:0/346936046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2fb40 msgr2=0x55f892a3a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/346936046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2fb40 0x55f892a3a4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8098380 tx=0x7fb7c8013000 comp rx=0 tx=0).stop 2024-09-19T09:29:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7db8a9740 1 -- 172.21.15.16:0/346936046 shutdown_connections 2024-09-19T09:29:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/346936046 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55f892a4d470 0x55f892a4f860 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/346936046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2fb40 0x55f892a3a4b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/346936046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a3d780 0x55f892a3abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/346936046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2e5d0 0x55f892a3d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7b6ff5640 0 client.25369 ms_handle_reset on v2:172.21.15.142:6838/3822134597 2024-09-19T09:29:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7db8a9740 1 -- 172.21.15.16:0/346936046 >> 172.21.15.16:0/346936046 conn(0x55f8929f7310 msgr2=0x55f892a30960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7db8a9740 1 -- 172.21.15.16:0/346936046 shutdown_connections 2024-09-19T09:29:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7db8a9740 1 -- 172.21.15.16:0/346936046 wait complete. 2024-09-19T09:29:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.214+0000 7fb7db8a9740 20 client.25369 trim_cache size 0 max 16384 2024-09-19T09:29:01.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.228+0000 7fb7db8a9740 1 Processor -- start 2024-09-19T09:29:01.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.228+0000 7fb7db8a9740 1 -- start start 2024-09-19T09:29:01.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.228+0000 7fb7db8a9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2e5d0 0x55f892a496c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.228+0000 7fb7db8a9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2fb40 0x55f892a39490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.228+0000 7fb7db8a9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a3d780 0x55f892a55110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.228+0000 7fb7db8a9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41ab2b0 con 0x55f892a2fb40 2024-09-19T09:29:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.228+0000 7fb7db8a9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41acdf0 con 0x55f892a2e5d0 2024-09-19T09:29:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.228+0000 7fb7db8a9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41ab980 con 0x55f892a3d780 2024-09-19T09:29:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2fb40 0x55f892a39490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2fb40 0x55f892a39490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40926/0 (socket says 172.21.15.16:40926) 2024-09-19T09:29:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7d27fc640 1 -- 172.21.15.16:0/3694931999 learned_addr learned my addr 172.21.15.16:0/3694931999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7d37fe640 1 --2- 172.21.15.16:0/3694931999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a3d780 0x55f892a55110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7d2ffd640 1 --2- 172.21.15.16:0/3694931999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2e5d0 0x55f892a496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7d27fc640 1 -- 172.21.15.16:0/3694931999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a3d780 msgr2=0x55f892a55110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7d27fc640 1 --2- 172.21.15.16:0/3694931999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a3d780 0x55f892a55110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7d27fc640 1 -- 172.21.15.16:0/3694931999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2e5d0 msgr2=0x55f892a496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7d27fc640 1 --2- 172.21.15.16:0/3694931999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2e5d0 0x55f892a496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7d27fc640 1 -- 172.21.15.16:0/3694931999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f892a2bdd0 con 0x55f892a2fb40 2024-09-19T09:29:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7d27fc640 1 --2- 172.21.15.16:0/3694931999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2fb40 0x55f892a39490 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8007590 tx=0x7fb7c809a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d016fe19eab02774 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7b8ff9640 1 -- 172.21.15.16:0/3694931999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80b6070 con 0x55f892a2fb40 2024-09-19T09:29:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7b8ff9640 1 -- 172.21.15.16:0/3694931999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c809c1d0 con 0x55f892a2fb40 2024-09-19T09:29:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.229+0000 7fb7b8ff9640 1 -- 172.21.15.16:0/3694931999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80cd4d0 con 0x55f892a2fb40 2024-09-19T09:29:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.230+0000 7fb7db8a9740 1 -- 172.21.15.16:0/3694931999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2fb40 msgr2=0x55f892a39490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.230+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/3694931999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2fb40 0x55f892a39490 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8007590 tx=0x7fb7c809a480 comp rx=0 tx=0).stop 2024-09-19T09:29:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.230+0000 7fb7db8a9740 1 -- 172.21.15.16:0/3694931999 shutdown_connections 2024-09-19T09:29:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.230+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/3694931999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a3d780 0x55f892a55110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.230+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/3694931999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2fb40 0x55f892a39490 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.230+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/3694931999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a2e5d0 0x55f892a496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.230+0000 7fb7db8a9740 1 -- 172.21.15.16:0/3694931999 >> 172.21.15.16:0/3694931999 conn(0x55f8929f7310 msgr2=0x55f892a38d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.230+0000 7fb7db8a9740 1 -- 172.21.15.16:0/3694931999 shutdown_connections 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.230+0000 7fb7db8a9740 1 -- 172.21.15.16:0/3694931999 wait complete. 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.231+0000 7fb7db8a9740 1 -- start start 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.231+0000 7fb7db8a9740 1 Processor -- start 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.231+0000 7fb7db8a9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2e5d0 0x55f892a45ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.232+0000 7fb7d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2e5d0 0x55f892a45ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.232+0000 7fb7d2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2e5d0 0x55f892a45ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40932/0 (socket says 172.21.15.16:40932) 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.232+0000 7fb7db8a9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2fb40 0x55f892a41320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.232+0000 7fb7d2ffd640 1 -- 172.21.15.16:0/2751945860 learned_addr learned my addr 172.21.15.16:0/2751945860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.232+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a3d780 0x55f892a41880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.232+0000 7fb7db8a9740 1 -- 172.21.15.16:0/2751945860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41ac720 con 0x55f892a2e5d0 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.232+0000 7fb7d27fc640 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2fb40 0x55f892a41320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.232+0000 7fb7db8a9740 1 -- 172.21.15.16:0/2751945860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41ac050 con 0x55f892a3d780 2024-09-19T09:29:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.232+0000 7fb7db8a9740 1 -- 172.21.15.16:0/2751945860 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d41a9290 con 0x55f892a2fb40 2024-09-19T09:29:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.232+0000 7fb7d37fe640 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a3d780 0x55f892a41880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.232+0000 7fb7db8a9740 10 client.0 _ll_register_callbacks cb 0x7fb7c809bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7d2ffd640 1 -- 172.21.15.16:0/2751945860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2fb40 msgr2=0x55f892a41320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7d2ffd640 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2fb40 0x55f892a41320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7d2ffd640 1 -- 172.21.15.16:0/2751945860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a3d780 msgr2=0x55f892a41880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7d2ffd640 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a3d780 0x55f892a41880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7d2ffd640 1 -- 172.21.15.16:0/2751945860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f892a31120 con 0x55f892a2e5d0 2024-09-19T09:29:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7d2ffd640 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2e5d0 0x55f892a45ff0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009900 tx=0x7fb7c40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d1510f8b001ada server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7b6ff5640 1 -- 172.21.15.16:0/2751945860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4035040 con 0x55f892a2e5d0 2024-09-19T09:29:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7b6ff5640 1 -- 172.21.15.16:0/2751945860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fb7c4005500 con 0x55f892a2e5d0 2024-09-19T09:29:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7db8a9740 1 -- 172.21.15.16:0/2751945860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c809c1d0 con 0x55f892a2e5d0 2024-09-19T09:29:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7b6ff5640 1 -- 172.21.15.16:0/2751945860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7c40046c0 con 0x55f892a2e5d0 2024-09-19T09:29:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7db8a9740 10 client.15858 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7db8a9740 1 -- 172.21.15.16:0/2751945860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f892ab7f90 con 0x55f892a2e5d0 2024-09-19T09:29:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.233+0000 7fb7b6ff5640 1 -- 172.21.15.16:0/2751945860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c809c1d0 con 0x55f892a2e5d0 2024-09-19T09:29:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.234+0000 7fb7b6ff5640 1 -- 172.21.15.16:0/2751945860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb7c4003c60 con 0x55f892a2e5d0 2024-09-19T09:29:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.234+0000 7fb7db8a9740 10 client.15858 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.234+0000 7fb7db8a9740 10 client.15858 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T09:29:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.234+0000 7fb7db8a9740 20 client.15858 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.234+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55f892ab86b0 0x55f892abaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.234+0000 7fb7db8a9740 4 client.15858 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:29:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.234+0000 7fb7db8a9740 1 -- 172.21.15.16:0/2751945860 --> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f89299bd70 con 0x55f892ab86b0 2024-09-19T09:29:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.234+0000 7fb7d27fc640 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55f892ab86b0 0x55f892abaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.234+0000 7fb7d27fc640 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55f892ab86b0 0x55f892abaae0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eca71a87eae50832 server_cookie=ebdac0e69df57a31 in_seq=0 out_seq=0 2024-09-19T09:29:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.234+0000 7fb7b6ff5640 10 client.15858 ms_handle_connect on v2:172.21.15.142:6838/3822134597 2024-09-19T09:29:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.234+0000 7fb7b6ff5640 1 -- 172.21.15.16:0/2751945860 <== mds.1 v2:172.21.15.142:6838/3822134597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f89299bd70 con 0x55f892ab86b0 2024-09-19T09:29:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.234+0000 7fb7b6ff5640 10 client.15858 handle_command_reply: tid=0 2024-09-19T09:29:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.234+0000 7fb7db8a9740 1 client.15858 shutdown 2024-09-19T09:29:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.235+0000 7fb7db8a9740 1 -- 172.21.15.16:0/2751945860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2e5d0 msgr2=0x55f892a45ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.235+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2e5d0 0x55f892a45ff0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009900 tx=0x7fb7c40042e0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.235+0000 7fb7db8a9740 1 -- 172.21.15.16:0/2751945860 shutdown_connections 2024-09-19T09:29:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.235+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x55f892ab86b0 0x55f892abaae0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.235+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f892a2fb40 0x55f892a41320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.235+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f892a3d780 0x55f892a41880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.235+0000 7fb7db8a9740 1 --2- 172.21.15.16:0/2751945860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f892a2e5d0 0x55f892a45ff0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.235+0000 7fb7db8a9740 1 -- 172.21.15.16:0/2751945860 >> 172.21.15.16:0/2751945860 conn(0x55f8929f7310 msgr2=0x55f892a395a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.235+0000 7fb7b6ff5640 0 client.15858 ms_handle_reset on v2:172.21.15.142:6838/3822134597 2024-09-19T09:29:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.235+0000 7fb7db8a9740 1 -- 172.21.15.16:0/2751945860 shutdown_connections 2024-09-19T09:29:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.235+0000 7fb7db8a9740 1 -- 172.21.15.16:0/2751945860 wait complete. 2024-09-19T09:29:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.235+0000 7fb7db8a9740 20 client.15858 trim_cache size 0 max 16384 2024-09-19T09:29:01.240 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:29:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.236+0000 7fb7d94d0640 1 -- 172.21.15.16:0/1518264294 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7bc0a32c0 msgr2=0x7fb7bc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.236+0000 7fb7d94d0640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7bc0a32c0 0x7fb7bc0a5780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009fd0 tx=0x7fb7cc004ab0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.236+0000 7fb7d94d0640 1 -- 172.21.15.16:0/1518264294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7d4102570 msgr2=0x7fb7d41989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.236+0000 7fb7d94d0640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7d4102570 0x7fb7d41989d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800c920 tx=0x7fb7c800cde0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.236+0000 7fb7d94d0640 1 -- 172.21.15.16:0/1518264294 shutdown_connections 2024-09-19T09:29:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.236+0000 7fb7d94d0640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fb7bc0a32c0 0x7fb7bc0a5780 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.236+0000 7fb7d94d0640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7d4102ef0 0x7fb7d419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.236+0000 7fb7d94d0640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7d4102570 0x7fb7d41989d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.236+0000 7fb7d94d0640 1 --2- 172.21.15.16:0/1518264294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d4100e20 0x7fb7d4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.236+0000 7fb7d94d0640 1 -- 172.21.15.16:0/1518264294 >> 172.21.15.16:0/1518264294 conn(0x7fb7d40fc7d0 msgr2=0x7fb7d406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.236+0000 7fb7d94d0640 1 -- 172.21.15.16:0/1518264294 shutdown_connections 2024-09-19T09:29:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:01.236+0000 7fb7d94d0640 1 -- 172.21.15.16:0/1518264294 wait complete. 2024-09-19T09:29:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:01.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:01 smithi142 ceph-mon[35032]: pgmap v380: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 KiB/s wr, 3.03k op/s 2024-09-19T09:29:01.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:01 smithi097 ceph-mon[34864]: pgmap v380: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 KiB/s wr, 3.03k op/s 2024-09-19T09:29:01.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:01 smithi016 ceph-mon[30979]: pgmap v380: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 KiB/s wr, 3.03k op/s 2024-09-19T09:29:02.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:29:02.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.457+0000 7f9b181f5640 1 -- 172.21.15.16:0/2907042179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b10102570 msgr2=0x7f9b101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:02.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.457+0000 7f9b181f5640 1 --2- 172.21.15.16:0/2907042179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b10102570 0x7f9b101029b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009ca0 tx=0x7f9b0402a950 comp rx=0 tx=0).stop 2024-09-19T09:29:02.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.457+0000 7f9b181f5640 1 -- 172.21.15.16:0/2907042179 shutdown_connections 2024-09-19T09:29:02.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.457+0000 7f9b181f5640 1 --2- 172.21.15.16:0/2907042179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10102ef0 0x7f9b1010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:02.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.457+0000 7f9b181f5640 1 --2- 172.21.15.16:0/2907042179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b10102570 0x7f9b101029b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:02.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.457+0000 7f9b181f5640 1 --2- 172.21.15.16:0/2907042179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b10100e20 0x7f9b10101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:02.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.457+0000 7f9b181f5640 1 -- 172.21.15.16:0/2907042179 >> 172.21.15.16:0/2907042179 conn(0x7f9b100fc7d0 msgr2=0x7f9b100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:02.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.457+0000 7f9b181f5640 1 -- 172.21.15.16:0/2907042179 shutdown_connections 2024-09-19T09:29:02.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.457+0000 7f9b181f5640 1 -- 172.21.15.16:0/2907042179 wait complete. 2024-09-19T09:29:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.458+0000 7f9b181f5640 1 Processor -- start 2024-09-19T09:29:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.458+0000 7f9b181f5640 1 -- start start 2024-09-19T09:29:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.458+0000 7f9b181f5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b10100e20 0x7f9b101993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.458+0000 7f9b181f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b10102570 0x7f9b10199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.458+0000 7f9b181f5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10102ef0 0x7f9b101a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.458+0000 7f9b181f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10110f50 con 0x7f9b10102570 2024-09-19T09:29:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.458+0000 7f9b181f5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10110dd0 con 0x7f9b10100e20 2024-09-19T09:29:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.458+0000 7f9b181f5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101110d0 con 0x7f9b10102ef0 2024-09-19T09:29:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.459+0000 7f9b15790640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b10102570 0x7f9b10199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.459+0000 7f9b15790640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b10102570 0x7f9b10199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40962/0 (socket says 172.21.15.16:40962) 2024-09-19T09:29:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.459+0000 7f9b15790640 1 -- 172.21.15.16:0/1641138201 learned_addr learned my addr 172.21.15.16:0/1641138201 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.459+0000 7f9b15f91640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b10100e20 0x7f9b101993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.459+0000 7f9b16792640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10102ef0 0x7f9b101a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.459+0000 7f9b16792640 1 -- 172.21.15.16:0/1641138201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b10100e20 msgr2=0x7f9b101993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.459+0000 7f9b16792640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b10100e20 0x7f9b101993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.459+0000 7f9b16792640 1 -- 172.21.15.16:0/1641138201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b10102570 msgr2=0x7f9b10199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.459+0000 7f9b16792640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b10102570 0x7f9b10199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.459+0000 7f9b16792640 1 -- 172.21.15.16:0/1641138201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b101a4980 con 0x7f9b10102ef0 2024-09-19T09:29:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.459+0000 7f9b15f91640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b10100e20 0x7f9b101993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.459+0000 7f9b15790640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b10102570 0x7f9b10199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.459+0000 7f9b16792640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10102ef0 0x7f9b101a4440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800d680 tx=0x7f9b0800db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.460+0000 7f9b02ffd640 1 -- 172.21.15.16:0/1641138201 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08017070 con 0x7f9b10102ef0 2024-09-19T09:29:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.460+0000 7f9b181f5640 1 -- 172.21.15.16:0/1641138201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b101a4b50 con 0x7f9b10102ef0 2024-09-19T09:29:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.460+0000 7f9b02ffd640 1 -- 172.21.15.16:0/1641138201 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b08009820 con 0x7f9b10102ef0 2024-09-19T09:29:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.460+0000 7f9b02ffd640 1 -- 172.21.15.16:0/1641138201 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08004e00 con 0x7f9b10102ef0 2024-09-19T09:29:02.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.460+0000 7f9b181f5640 1 -- 172.21.15.16:0/1641138201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b101a5010 con 0x7f9b10102ef0 2024-09-19T09:29:02.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.461+0000 7f9b181f5640 1 -- 172.21.15.16:0/1641138201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b10101200 con 0x7f9b10102ef0 2024-09-19T09:29:02.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.462+0000 7f9b02ffd640 1 -- 172.21.15.16:0/1641138201 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9b080099d0 con 0x7f9b10102ef0 2024-09-19T09:29:02.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.463+0000 7f9b02ffd640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9afc0a3240 0x7f9afc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:02.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.463+0000 7f9b02ffd640 1 -- 172.21.15.16:0/1641138201 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9b080cf4a0 con 0x7f9b10102ef0 2024-09-19T09:29:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.463+0000 7f9b15f91640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9afc0a3240 0x7f9afc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.463+0000 7f9b15f91640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9afc0a3240 0x7f9afc0a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00bfd0 tx=0x7f9b0c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.466+0000 7f9b02ffd640 1 -- 172.21.15.16:0/1641138201 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9b0809bb30 con 0x7f9b10102ef0 2024-09-19T09:29:02.488 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:02.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.687+0000 7f9b181f5640 1 -- 172.21.15.16:0/1641138201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b10109090 con 0x7f9b10102ef0 2024-09-19T09:29:02.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.689+0000 7f9b02ffd640 1 -- 172.21.15.16:0/1641138201 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f9b0809fe30 con 0x7f9b10102ef0 2024-09-19T09:29:02.690 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:29:02.690 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:29:02.691 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:29:02.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.691+0000 7f9b181f5640 1 -- 172.21.15.16:0/1641138201 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9afc0a3240 msgr2=0x7f9afc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.691+0000 7f9b181f5640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9afc0a3240 0x7f9afc0a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00bfd0 tx=0x7f9b0c00b390 comp rx=0 tx=0).stop 2024-09-19T09:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.691+0000 7f9b181f5640 1 -- 172.21.15.16:0/1641138201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10102ef0 msgr2=0x7f9b101a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.691+0000 7f9b181f5640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10102ef0 0x7f9b101a4440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800d680 tx=0x7f9b0800db40 comp rx=0 tx=0).stop 2024-09-19T09:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.691+0000 7f9b181f5640 1 -- 172.21.15.16:0/1641138201 shutdown_connections 2024-09-19T09:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.691+0000 7f9b181f5640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9afc0a3240 0x7f9afc0a5700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.691+0000 7f9b181f5640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b10102ef0 0x7f9b101a4440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.691+0000 7f9b181f5640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b10102570 0x7f9b10199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.691+0000 7f9b181f5640 1 --2- 172.21.15.16:0/1641138201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b10100e20 0x7f9b101993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.691+0000 7f9b181f5640 1 -- 172.21.15.16:0/1641138201 >> 172.21.15.16:0/1641138201 conn(0x7f9b100fc7d0 msgr2=0x7f9b100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.691+0000 7f9b181f5640 1 -- 172.21.15.16:0/1641138201 shutdown_connections 2024-09-19T09:29:02.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:02.691+0000 7f9b181f5640 1 -- 172.21.15.16:0/1641138201 wait complete. 2024-09-19T09:29:03.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.173+0000 7f9e0e0f2640 1 -- 172.21.15.16:0/912977957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e0810c170 msgr2=0x7f9e0810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.173+0000 7f9e0e0f2640 1 --2- 172.21.15.16:0/912977957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e0810c170 0x7f9e0810e610 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009ca0 tx=0x7f9e0002a950 comp rx=0 tx=0).stop 2024-09-19T09:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.173+0000 7f9e0e0f2640 1 -- 172.21.15.16:0/912977957 shutdown_connections 2024-09-19T09:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.173+0000 7f9e0e0f2640 1 --2- 172.21.15.16:0/912977957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e08103660 0x7f9e0810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.173+0000 7f9e0e0f2640 1 --2- 172.21.15.16:0/912977957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e0810c170 0x7f9e0810e610 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.173+0000 7f9e0e0f2640 1 --2- 172.21.15.16:0/912977957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e08102d40 0x7f9e08103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.173+0000 7f9e0e0f2640 1 -- 172.21.15.16:0/912977957 >> 172.21.15.16:0/912977957 conn(0x7f9e080fc7a0 msgr2=0x7f9e080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.173+0000 7f9e0e0f2640 1 -- 172.21.15.16:0/912977957 shutdown_connections 2024-09-19T09:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.173+0000 7f9e0e0f2640 1 -- 172.21.15.16:0/912977957 wait complete. 2024-09-19T09:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.174+0000 7f9e0e0f2640 1 Processor -- start 2024-09-19T09:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.174+0000 7f9e0e0f2640 1 -- start start 2024-09-19T09:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.174+0000 7f9e0e0f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e08102d40 0x7f9e08198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.174+0000 7f9e0e0f2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e08103660 0x7f9e081988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.174+0000 7f9e0e0f2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e0810c170 0x7f9e0819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.174+0000 7f9e0e0f2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08111f50 con 0x7f9e08102d40 2024-09-19T09:29:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.174+0000 7f9e0e0f2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08111dd0 con 0x7f9e08103660 2024-09-19T09:29:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.174+0000 7f9e0e0f2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081120d0 con 0x7f9e0810c170 2024-09-19T09:29:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.174+0000 7f9e077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e08102d40 0x7f9e08198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.174+0000 7f9e077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e08102d40 0x7f9e08198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40984/0 (socket says 172.21.15.16:40984) 2024-09-19T09:29:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.174+0000 7f9e077fe640 1 -- 172.21.15.16:0/2338893656 learned_addr learned my addr 172.21.15.16:0/2338893656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.174+0000 7f9e06ffd640 1 --2- 172.21.15.16:0/2338893656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e08103660 0x7f9e081988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.175+0000 7f9e077fe640 1 -- 172.21.15.16:0/2338893656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e0810c170 msgr2=0x7f9e0819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.175+0000 7f9e077fe640 1 --2- 172.21.15.16:0/2338893656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e0810c170 0x7f9e0819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.175+0000 7f9e077fe640 1 -- 172.21.15.16:0/2338893656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e08103660 msgr2=0x7f9e081988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.175+0000 7f9e077fe640 1 --2- 172.21.15.16:0/2338893656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e08103660 0x7f9e081988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.175+0000 7f9e077fe640 1 -- 172.21.15.16:0/2338893656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0819d320 con 0x7f9e08102d40 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.175+0000 7f9e077fe640 1 --2- 172.21.15.16:0/2338893656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e08102d40 0x7f9e08198370 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9df800c920 tx=0x7f9df800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.175+0000 7f9e04ff9640 1 -- 172.21.15.16:0/2338893656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8007c70 con 0x7f9e08102d40 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.175+0000 7f9e0e0f2640 1 -- 172.21.15.16:0/2338893656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0819d610 con 0x7f9e08102d40 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.175+0000 7f9e04ff9640 1 -- 172.21.15.16:0/2338893656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df8007e10 con 0x7f9e08102d40 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.175+0000 7f9e04ff9640 1 -- 172.21.15.16:0/2338893656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8018890 con 0x7f9e08102d40 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.175+0000 7f9e0e0f2640 1 -- 172.21.15.16:0/2338893656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e081a4ef0 con 0x7f9e08102d40 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.178+0000 7f9e04ff9640 1 -- 172.21.15.16:0/2338893656 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9df8018a70 con 0x7f9e08102d40 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.178+0000 7f9e104cb740 1 Processor -- start 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.178+0000 7f9e104cb740 1 -- start start 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.178+0000 7f9e104cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 0x5595f2632090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.178+0000 7f9e104cb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f26325d0 0x5595f2632a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.178+0000 7f9e104cb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f263d370 0x5595f263f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.178+0000 7f9e104cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5595f25982f0 con 0x5595f2633b40 2024-09-19T09:29:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.178+0000 7f9e104cb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5595f253fcf0 con 0x5595f263d370 2024-09-19T09:29:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.178+0000 7f9e104cb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5595f25942b0 con 0x5595f26325d0 2024-09-19T09:29:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 0x5595f2632090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 0x5595f2632090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40990/0 (socket says 172.21.15.16:40990) 2024-09-19T09:29:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e07fff640 1 -- 172.21.15.16:0/183322652 learned_addr learned my addr 172.21.15.16:0/183322652 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e06ffd640 1 --2- 172.21.15.16:0/183322652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f263d370 0x5595f263f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e077fe640 1 --2- 172.21.15.16:0/183322652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f26325d0 0x5595f2632a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e07fff640 1 -- 172.21.15.16:0/183322652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f26325d0 msgr2=0x5595f2632a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e07fff640 1 --2- 172.21.15.16:0/183322652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f26325d0 0x5595f2632a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e07fff640 1 -- 172.21.15.16:0/183322652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f263d370 msgr2=0x5595f263f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e04ff9640 1 --2- 172.21.15.16:0/2338893656 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9df00a32c0 0x7f9df00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e07fff640 1 --2- 172.21.15.16:0/183322652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f263d370 0x5595f263f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e07fff640 1 -- 172.21.15.16:0/183322652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595f25243a0 con 0x5595f2633b40 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e077fe640 1 --2- 172.21.15.16:0/183322652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f26325d0 0x5595f2632a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e06ffd640 1 --2- 172.21.15.16:0/183322652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f263d370 0x5595f263f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e07fff640 1 --2- 172.21.15.16:0/183322652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 0x5595f2632090 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9e00005e30 tx=0x7f9e00005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6339eb13520a5a84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e04ff9640 1 -- 172.21.15.16:0/2338893656 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9df80cf9d0 con 0x7f9e08102d40 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e07fff640 1 --2- 172.21.15.16:0/2338893656 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9df00a32c0 0x7f9df00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9dedffb640 1 -- 172.21.15.16:0/183322652 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00034040 con 0x5595f2633b40 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9dedffb640 1 -- 172.21.15.16:0/183322652 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e0002f4a0 con 0x5595f2633b40 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9dedffb640 1 -- 172.21.15.16:0/183322652 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00039690 con 0x5595f2633b40 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e104cb740 1 -- 172.21.15.16:0/183322652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 msgr2=0x5595f2632090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e07fff640 1 --2- 172.21.15.16:0/2338893656 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9df00a32c0 0x7f9df00a5780 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e00043000 tx=0x7f9e00039990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e104cb740 1 --2- 172.21.15.16:0/183322652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 0x5595f2632090 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9e00005e30 tx=0x7f9e00005fd0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e104cb740 1 -- 172.21.15.16:0/183322652 shutdown_connections 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e104cb740 1 --2- 172.21.15.16:0/183322652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f26325d0 0x5595f2632a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e104cb740 1 --2- 172.21.15.16:0/183322652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f263d370 0x5595f263f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e104cb740 1 --2- 172.21.15.16:0/183322652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 0x5595f2632090 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e104cb740 1 -- 172.21.15.16:0/183322652 >> 172.21.15.16:0/183322652 conn(0x5595f25fb310 msgr2=0x5595f262ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e104cb740 1 -- 172.21.15.16:0/183322652 shutdown_connections 2024-09-19T09:29:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.179+0000 7f9e104cb740 1 -- 172.21.15.16:0/183322652 wait complete. 2024-09-19T09:29:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e104cb740 1 -- start start 2024-09-19T09:29:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e104cb740 1 Processor -- start 2024-09-19T09:29:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e104cb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 0x5595f263ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e104cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 0x5595f2639f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e104cb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f263d370 0x5595f263a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e104cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5595f2640010 con 0x5595f2633b40 2024-09-19T09:29:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e104cb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5595f2553060 con 0x5595f26325d0 2024-09-19T09:29:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e104cb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5595f2640190 con 0x5595f263d370 2024-09-19T09:29:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e104cb740 10 client.0 _ll_register_callbacks cb 0x5595f22a68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e06ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 0x5595f2639f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e06ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 0x5595f2639f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40996/0 (socket says 172.21.15.16:40996) 2024-09-19T09:29:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e06ffd640 1 -- 172.21.15.16:0/3762566040 learned_addr learned my addr 172.21.15.16:0/3762566040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e07fff640 1 --2- 172.21.15.16:0/3762566040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f263d370 0x5595f263a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.180+0000 7f9e077fe640 1 --2- 172.21.15.16:0/3762566040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 0x5595f263ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9e06ffd640 1 -- 172.21.15.16:0/3762566040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f263d370 msgr2=0x5595f263a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9e06ffd640 1 --2- 172.21.15.16:0/3762566040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f263d370 0x5595f263a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9e06ffd640 1 -- 172.21.15.16:0/3762566040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 msgr2=0x5595f263ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9e06ffd640 1 --2- 172.21.15.16:0/3762566040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 0x5595f263ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9e06ffd640 1 -- 172.21.15.16:0/3762566040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595f23093c0 con 0x5595f2633b40 2024-09-19T09:29:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9e06ffd640 1 --2- 172.21.15.16:0/3762566040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 0x5595f2639f60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00b9b0 tx=0x7f9dfc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700d02dc0c3b22c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9de6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9de6ffd640 1 -- 172.21.15.16:0/3762566040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc005020 con 0x5595f2633b40 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9de6ffd640 1 -- 172.21.15.16:0/3762566040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9dfc0051c0 con 0x5595f2633b40 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9e104cb740 1 -- 172.21.15.16:0/3762566040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e0002f4a0 con 0x5595f2633b40 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9de6ffd640 1 -- 172.21.15.16:0/3762566040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9dfc0053c0 con 0x5595f2633b40 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9e104cb740 10 client.15876 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9e104cb740 1 -- 172.21.15.16:0/3762566040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595f264ba20 con 0x5595f2633b40 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9de6ffd640 1 -- 172.21.15.16:0/3762566040 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e0002f4a0 con 0x5595f2633b40 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.181+0000 7f9de6ffd640 1 -- 172.21.15.16:0/3762566040 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9dfc019030 con 0x5595f2633b40 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.182+0000 7f9e104cb740 10 client.15876 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.182+0000 7f9e104cb740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.182+0000 7f9e104cb740 20 client.15876 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.182+0000 7f9e104cb740 1 --2- 172.21.15.16:0/3762566040 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5595f264cf30 0x5595f264f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.182+0000 7f9e104cb740 4 client.15876 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.182+0000 7f9e104cb740 1 -- 172.21.15.16:0/3762566040 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595f25a5860 con 0x5595f264cf30 2024-09-19T09:29:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.182+0000 7f9e077fe640 1 --2- 172.21.15.16:0/3762566040 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5595f264cf30 0x5595f264f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.182+0000 7f9de6ffd640 10 client.15876 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.182+0000 7f9e077fe640 1 --2- 172.21.15.16:0/3762566040 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5595f264cf30 0x5595f264f390 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=336e2399ba0ed092 server_cookie=3b3ffd495f1c93d2 in_seq=0 out_seq=0 2024-09-19T09:29:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9de6ffd640 1 -- 172.21.15.16:0/3762566040 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595f25a5860 con 0x5595f264cf30 2024-09-19T09:29:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9de6ffd640 10 client.15876 handle_command_reply: tid=0 2024-09-19T09:29:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9e104cb740 1 client.15876 shutdown 2024-09-19T09:29:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9e104cb740 1 -- 172.21.15.16:0/3762566040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 msgr2=0x5595f2639f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9e104cb740 1 --2- 172.21.15.16:0/3762566040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 0x5595f2639f60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00b9b0 tx=0x7f9dfc00be70 comp rx=0 tx=0).stop 2024-09-19T09:29:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9e104cb740 1 -- 172.21.15.16:0/3762566040 shutdown_connections 2024-09-19T09:29:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9e104cb740 1 --2- 172.21.15.16:0/3762566040 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5595f264cf30 0x5595f264f390 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9e104cb740 1 --2- 172.21.15.16:0/3762566040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f263d370 0x5595f263a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9e104cb740 1 --2- 172.21.15.16:0/3762566040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f2633b40 0x5595f2639f60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9de6ffd640 0 client.15876 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9e104cb740 1 --2- 172.21.15.16:0/3762566040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 0x5595f263ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9e104cb740 1 -- 172.21.15.16:0/3762566040 >> 172.21.15.16:0/3762566040 conn(0x5595f25fb310 msgr2=0x5595f2634960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9e104cb740 1 -- 172.21.15.16:0/3762566040 shutdown_connections 2024-09-19T09:29:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.183+0000 7f9e104cb740 1 -- 172.21.15.16:0/3762566040 wait complete. 2024-09-19T09:29:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.184+0000 7f9e104cb740 20 client.15876 trim_cache size 0 max 16384 2024-09-19T09:29:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.196+0000 7f9e104cb740 1 Processor -- start 2024-09-19T09:29:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e104cb740 1 -- start start 2024-09-19T09:29:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e104cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 0x5595f26492b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e104cb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 0x5595f2638f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e104cb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f2633b40 0x5595f2654c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e104cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081ad510 con 0x5595f25fb310 2024-09-19T09:29:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e104cb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081af050 con 0x5595f26325d0 2024-09-19T09:29:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e104cb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081adbe0 con 0x5595f2633b40 2024-09-19T09:29:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 0x5595f26492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 0x5595f26492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41002/0 (socket says 172.21.15.16:41002) 2024-09-19T09:29:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e077fe640 1 -- 172.21.15.16:0/3484575120 learned_addr learned my addr 172.21.15.16:0/3484575120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e06ffd640 1 --2- 172.21.15.16:0/3484575120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 0x5595f2638f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e077fe640 1 -- 172.21.15.16:0/3484575120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f2633b40 msgr2=0x5595f2654c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e077fe640 1 --2- 172.21.15.16:0/3484575120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f2633b40 0x5595f2654c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e077fe640 1 -- 172.21.15.16:0/3484575120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 msgr2=0x5595f2638f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e077fe640 1 --2- 172.21.15.16:0/3484575120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 0x5595f2638f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e077fe640 1 -- 172.21.15.16:0/3484575120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595f2631ec0 con 0x5595f25fb310 2024-09-19T09:29:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9e077fe640 1 --2- 172.21.15.16:0/3484575120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 0x5595f26492b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9df8007590 tx=0x7f9df80cd3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd42a7ecf136348b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.197+0000 7f9ded7fa640 1 -- 172.21.15.16:0/3484575120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df80b4070 con 0x5595f25fb310 2024-09-19T09:29:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.198+0000 7f9ded7fa640 1 -- 172.21.15.16:0/3484575120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df80a4c60 con 0x5595f25fb310 2024-09-19T09:29:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.198+0000 7f9ded7fa640 1 -- 172.21.15.16:0/3484575120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df800fb30 con 0x5595f25fb310 2024-09-19T09:29:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.198+0000 7f9e104cb740 1 -- 172.21.15.16:0/3484575120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 msgr2=0x5595f26492b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.198+0000 7f9e104cb740 1 --2- 172.21.15.16:0/3484575120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 0x5595f26492b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9df8007590 tx=0x7f9df80cd3a0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.198+0000 7f9e104cb740 1 -- 172.21.15.16:0/3484575120 shutdown_connections 2024-09-19T09:29:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.198+0000 7f9e104cb740 1 --2- 172.21.15.16:0/3484575120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f2633b40 0x5595f2654c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.198+0000 7f9e104cb740 1 --2- 172.21.15.16:0/3484575120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 0x5595f2638f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.198+0000 7f9e104cb740 1 --2- 172.21.15.16:0/3484575120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 0x5595f26492b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.198+0000 7f9e104cb740 1 -- 172.21.15.16:0/3484575120 >> 172.21.15.16:0/3484575120 conn(0x5595f263a600 msgr2=0x5595f2638830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.198+0000 7f9e104cb740 1 -- 172.21.15.16:0/3484575120 shutdown_connections 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.198+0000 7f9e104cb740 1 -- 172.21.15.16:0/3484575120 wait complete. 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.198+0000 7f9e104cb740 1 -- start start 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e104cb740 1 Processor -- start 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e104cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 0x5595f2645c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e104cb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 0x5595f2640f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e104cb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f2633b40 0x5595f26414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e104cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081ae980 con 0x5595f25fb310 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e104cb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081ae2b0 con 0x5595f26325d0 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e104cb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081ab4d0 con 0x5595f2633b40 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e104cb740 10 client.0 _ll_register_callbacks cb 0x7f9dfc004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 0x5595f2645c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 0x5595f2645c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41010/0 (socket says 172.21.15.16:41010) 2024-09-19T09:29:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e077fe640 1 -- 172.21.15.16:0/2610382337 learned_addr learned my addr 172.21.15.16:0/2610382337 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e06ffd640 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 0x5595f2640f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e07fff640 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f2633b40 0x5595f26414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e077fe640 1 -- 172.21.15.16:0/2610382337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f2633b40 msgr2=0x5595f26414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e077fe640 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f2633b40 0x5595f26414c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e077fe640 1 -- 172.21.15.16:0/2610382337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 msgr2=0x5595f2640f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e077fe640 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 0x5595f2640f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e077fe640 1 -- 172.21.15.16:0/2610382337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595f2641cf0 con 0x5595f25fb310 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.199+0000 7f9e07fff640 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f2633b40 0x5595f26414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9e077fe640 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 0x5595f2645c10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000c00 tx=0x7f9df809bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5413446a2a521f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9de6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9de6ffd640 1 -- 172.21.15.16:0/2610382337 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df80b4070 con 0x5595f25fb310 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9de6ffd640 1 -- 172.21.15.16:0/2610382337 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9df8018d60 con 0x5595f25fb310 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9e104cb740 1 -- 172.21.15.16:0/2610382337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df80a4c60 con 0x5595f25fb310 2024-09-19T09:29:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9de6ffd640 1 -- 172.21.15.16:0/2610382337 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9df809c6f0 con 0x5595f25fb310 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9e104cb740 10 client.15882 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9e104cb740 1 -- 172.21.15.16:0/2610382337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595f26b7cd0 con 0x5595f25fb310 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9de6ffd640 1 -- 172.21.15.16:0/2610382337 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df80a4c60 con 0x5595f25fb310 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9de6ffd640 1 -- 172.21.15.16:0/2610382337 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9df809c250 con 0x5595f25fb310 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9e104cb740 10 client.15882 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9e104cb740 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9e104cb740 20 client.15882 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9e104cb740 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5595f263d370 0x5595f26ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9e104cb740 4 client.15882 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.200+0000 7f9e104cb740 1 -- 172.21.15.16:0/2610382337 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595f259fd70 con 0x5595f263d370 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.201+0000 7f9e06ffd640 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5595f263d370 0x5595f26ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.201+0000 7f9e06ffd640 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5595f263d370 0x5595f26ba3a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=420593046fd8d579 server_cookie=e472459602a79bd3 in_seq=0 out_seq=0 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.201+0000 7f9de6ffd640 10 client.15882 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.201+0000 7f9de6ffd640 1 -- 172.21.15.16:0/2610382337 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595f259fd70 con 0x5595f263d370 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.201+0000 7f9de6ffd640 10 client.15882 handle_command_reply: tid=0 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.201+0000 7f9e104cb740 1 client.15882 shutdown 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.201+0000 7f9e104cb740 1 -- 172.21.15.16:0/2610382337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 msgr2=0x5595f2645c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.201+0000 7f9e104cb740 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 0x5595f2645c10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000c00 tx=0x7f9df809bbc0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.201+0000 7f9e104cb740 1 -- 172.21.15.16:0/2610382337 shutdown_connections 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.201+0000 7f9e104cb740 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5595f263d370 0x5595f26ba3a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.201+0000 7f9e104cb740 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5595f2633b40 0x5595f26414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e104cb740 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595f26325d0 0x5595f2640f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e104cb740 1 --2- 172.21.15.16:0/2610382337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595f25fb310 0x5595f2645c10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e104cb740 1 -- 172.21.15.16:0/2610382337 >> 172.21.15.16:0/2610382337 conn(0x5595f263a600 msgr2=0x5595f2639050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9de6ffd640 0 client.15882 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e104cb740 1 -- 172.21.15.16:0/2610382337 shutdown_connections 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e104cb740 1 -- 172.21.15.16:0/2610382337 wait complete. 2024-09-19T09:29:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e104cb740 20 client.15882 trim_cache size 0 max 16384 2024-09-19T09:29:03.219 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:29:03.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e0e0f2640 1 -- 172.21.15.16:0/2338893656 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9df00a32c0 msgr2=0x7f9df00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:03.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e0e0f2640 1 --2- 172.21.15.16:0/2338893656 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9df00a32c0 0x7f9df00a5780 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e00043000 tx=0x7f9e00039990 comp rx=0 tx=0).stop 2024-09-19T09:29:03.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e0e0f2640 1 -- 172.21.15.16:0/2338893656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e08102d40 msgr2=0x7f9e08198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:03.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e0e0f2640 1 --2- 172.21.15.16:0/2338893656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e08102d40 0x7f9e08198370 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9df800c920 tx=0x7f9df800cde0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e0e0f2640 1 -- 172.21.15.16:0/2338893656 shutdown_connections 2024-09-19T09:29:03.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e0e0f2640 1 --2- 172.21.15.16:0/2338893656 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f9df00a32c0 0x7f9df00a5780 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e0e0f2640 1 --2- 172.21.15.16:0/2338893656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e0810c170 0x7f9e0819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e0e0f2640 1 --2- 172.21.15.16:0/2338893656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e08103660 0x7f9e081988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e0e0f2640 1 --2- 172.21.15.16:0/2338893656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e08102d40 0x7f9e08198370 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:03.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e0e0f2640 1 -- 172.21.15.16:0/2338893656 >> 172.21.15.16:0/2338893656 conn(0x7f9e080fc7a0 msgr2=0x7f9e080fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:03.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e0e0f2640 1 -- 172.21.15.16:0/2338893656 shutdown_connections 2024-09-19T09:29:03.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:03.202+0000 7f9e0e0f2640 1 -- 172.21.15.16:0/2338893656 wait complete. 2024-09-19T09:29:03.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:03 smithi142 ceph-mon[35032]: pgmap v381: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 597 B/s wr, 1.34k op/s 2024-09-19T09:29:03.382 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:03 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1641138201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:03.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:03 smithi097 ceph-mon[34864]: pgmap v381: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 597 B/s wr, 1.34k op/s 2024-09-19T09:29:03.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:03 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1641138201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:03.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:03 smithi016 ceph-mon[30979]: pgmap v381: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 597 B/s wr, 1.34k op/s 2024-09-19T09:29:03.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:03 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1641138201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:04.681 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 9f439ef8 --release --await' 2024-09-19T09:29:04.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 9f439ef8 --release --await 2024-09-19T09:29:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:05.233 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T09:29:05.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T09:29:05.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:05 smithi016 ceph-mon[30979]: pgmap v382: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.9 KiB/s wr, 1.34k op/s 2024-09-19T09:29:05.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:05 smithi142 ceph-mon[35032]: pgmap v382: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.9 KiB/s wr, 1.34k op/s 2024-09-19T09:29:05.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:05 smithi097 ceph-mon[34864]: pgmap v382: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.9 KiB/s wr, 1.34k op/s 2024-09-19T09:29:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.427+0000 7f952f6c2640 1 -- 172.21.15.16:0/2779230697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9528101420 msgr2=0x7f952810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.427+0000 7f952f6c2640 1 --2- 172.21.15.16:0/2779230697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9528101420 0x7f952810d950 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9520009cd0 tx=0x7f952002a8e0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.427+0000 7f952f6c2640 1 -- 172.21.15.16:0/2779230697 shutdown_connections 2024-09-19T09:29:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.427+0000 7f952f6c2640 1 --2- 172.21.15.16:0/2779230697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9528100a80 0x7f9528100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.427+0000 7f952f6c2640 1 --2- 172.21.15.16:0/2779230697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9528101420 0x7f952810d950 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.427+0000 7f952f6c2640 1 --2- 172.21.15.16:0/2779230697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9528106a80 0x7f9528106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.427+0000 7f952f6c2640 1 -- 172.21.15.16:0/2779230697 >> 172.21.15.16:0/2779230697 conn(0x7f95280fc7a0 msgr2=0x7f95280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.427+0000 7f952f6c2640 1 -- 172.21.15.16:0/2779230697 shutdown_connections 2024-09-19T09:29:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.427+0000 7f952f6c2640 1 -- 172.21.15.16:0/2779230697 wait complete. 2024-09-19T09:29:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.428+0000 7f952f6c2640 1 Processor -- start 2024-09-19T09:29:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.428+0000 7f952f6c2640 1 -- start start 2024-09-19T09:29:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952f6c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9528100a80 0x7f95281972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952f6c2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9528101420 0x7f9528197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952f6c2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9528106a80 0x7f952819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952f6c2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9528110ed0 con 0x7f9528100a80 2024-09-19T09:29:05.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952f6c2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9528110d50 con 0x7f9528106a80 2024-09-19T09:29:05.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952f6c2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9528111050 con 0x7f9528101420 2024-09-19T09:29:05.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952d45e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9528100a80 0x7f95281972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952d45e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9528100a80 0x7f95281972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39946/0 (socket says 172.21.15.16:39946) 2024-09-19T09:29:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952d45e640 1 -- 172.21.15.16:0/3709519947 learned_addr learned my addr 172.21.15.16:0/3709519947 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952cc5d640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9528101420 0x7f9528197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952dc5f640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9528106a80 0x7f952819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952d45e640 1 -- 172.21.15.16:0/3709519947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9528101420 msgr2=0x7f9528197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952d45e640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9528101420 0x7f9528197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952d45e640 1 -- 172.21.15.16:0/3709519947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9528106a80 msgr2=0x7f952819bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952d45e640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9528106a80 0x7f952819bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952d45e640 1 -- 172.21.15.16:0/3709519947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952819c2a0 con 0x7f9528100a80 2024-09-19T09:29:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952dc5f640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9528106a80 0x7f952819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.429+0000 7f952d45e640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9528100a80 0x7f95281972f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f952400e9f0 tx=0x7f952400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.430+0000 7f951e7fc640 1 -- 172.21.15.16:0/3709519947 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952400cd80 con 0x7f9528100a80 2024-09-19T09:29:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.430+0000 7f952f6c2640 1 -- 172.21.15.16:0/3709519947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952819c590 con 0x7f9528100a80 2024-09-19T09:29:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.430+0000 7f951e7fc640 1 -- 172.21.15.16:0/3709519947 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9524010910 con 0x7f9528100a80 2024-09-19T09:29:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.430+0000 7f951e7fc640 1 -- 172.21.15.16:0/3709519947 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240188f0 con 0x7f9528100a80 2024-09-19T09:29:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.430+0000 7f952f6c2640 1 -- 172.21.15.16:0/3709519947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95281a3e70 con 0x7f9528100a80 2024-09-19T09:29:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.432+0000 7f951e7fc640 1 -- 172.21.15.16:0/3709519947 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f9524010ac0 con 0x7f9528100a80 2024-09-19T09:29:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.432+0000 7f9531a9b740 1 Processor -- start 2024-09-19T09:29:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f9531a9b740 1 -- start start 2024-09-19T09:29:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f9531a9b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77270 0x556781b77650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f9531a9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 0x556781b81f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f9531a9b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b82450 0x556781b84840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f9531a9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556781ada620 con 0x556781b77b90 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f9531a9b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556781a81e50 con 0x556781b77270 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f9531a9b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556781ad65e0 con 0x556781b82450 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f952d45e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 0x556781b81f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f951e7fc640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f95140a3240 0x7f95140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f952d45e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 0x556781b81f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39956/0 (socket says 172.21.15.16:39956) 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f952d45e640 1 -- 172.21.15.16:0/83769023 learned_addr learned my addr 172.21.15.16:0/83769023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f952cc5d640 1 --2- 172.21.15.16:0/83769023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b82450 0x556781b84840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f952dc5f640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f95140a3240 0x7f95140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f951e7fc640 1 -- 172.21.15.16:0/3709519947 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f95240cfe10 con 0x7f9528100a80 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f952dc5f640 1 --2- 172.21.15.16:0/83769023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77270 0x556781b77650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f952d45e640 1 -- 172.21.15.16:0/83769023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b82450 msgr2=0x556781b84840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f952d45e640 1 --2- 172.21.15.16:0/83769023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b82450 0x556781b84840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.433+0000 7f952d45e640 1 -- 172.21.15.16:0/83769023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77270 msgr2=0x556781b77650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.434+0000 7f952d45e640 1 --2- 172.21.15.16:0/83769023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77270 0x556781b77650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.434+0000 7f952d45e640 1 -- 172.21.15.16:0/83769023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556781a72d40 con 0x556781b77b90 2024-09-19T09:29:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.434+0000 7f952dc5f640 1 --2- 172.21.15.16:0/83769023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77270 0x556781b77650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.434+0000 7f952d45e640 1 --2- 172.21.15.16:0/83769023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 0x556781b81f10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f95240cd8d0 tx=0x7f952401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb298812fabbf9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.434+0000 7f95137fe640 1 -- 172.21.15.16:0/83769023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952409caf0 con 0x556781b77b90 2024-09-19T09:29:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.434+0000 7f95137fe640 1 -- 172.21.15.16:0/83769023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95240a07a0 con 0x556781b77b90 2024-09-19T09:29:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.434+0000 7f95137fe640 1 -- 172.21.15.16:0/83769023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240bc7b0 con 0x556781b77b90 2024-09-19T09:29:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.435+0000 7f9531a9b740 1 -- 172.21.15.16:0/83769023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 msgr2=0x556781b81f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.435+0000 7f9531a9b740 1 --2- 172.21.15.16:0/83769023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 0x556781b81f10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f95240cd8d0 tx=0x7f952401f000 comp rx=0 tx=0).stop 2024-09-19T09:29:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.435+0000 7f952dc5f640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f95140a3240 0x7f95140a5700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9520005e30 tx=0x7f9520005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.435+0000 7f9531a9b740 1 -- 172.21.15.16:0/83769023 shutdown_connections 2024-09-19T09:29:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.435+0000 7f9531a9b740 1 --2- 172.21.15.16:0/83769023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b82450 0x556781b84840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.435+0000 7f9531a9b740 1 --2- 172.21.15.16:0/83769023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 0x556781b81f10 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.435+0000 7f9531a9b740 1 --2- 172.21.15.16:0/83769023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77270 0x556781b77650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.435+0000 7f9531a9b740 1 -- 172.21.15.16:0/83769023 >> 172.21.15.16:0/83769023 conn(0x556781b73800 msgr2=0x556781b73be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.435+0000 7f9531a9b740 1 -- 172.21.15.16:0/83769023 shutdown_connections 2024-09-19T09:29:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.435+0000 7f9531a9b740 1 -- 172.21.15.16:0/83769023 wait complete. 2024-09-19T09:29:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.435+0000 7f9531a9b740 1 -- start start 2024-09-19T09:29:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.435+0000 7f9531a9b740 1 Processor -- start 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f9531a9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77270 0x556781b7f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f9531a9b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77b90 0x556781b7f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f9531a9b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b82450 0x556781b7fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f9531a9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556781b85020 con 0x556781b77270 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f9531a9b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556781a62f10 con 0x556781b77b90 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f9531a9b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556781b851a0 con 0x556781b82450 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f9531a9b740 10 client.0 _ll_register_callbacks cb 0x556781a86db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f952d45e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77270 0x556781b7f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f952d45e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77270 0x556781b7f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39962/0 (socket says 172.21.15.16:39962) 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f952d45e640 1 -- 172.21.15.16:0/2319811098 learned_addr learned my addr 172.21.15.16:0/2319811098 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f952dc5f640 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b82450 0x556781b7fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f952cc5d640 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77b90 0x556781b7f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f952d45e640 1 -- 172.21.15.16:0/2319811098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b82450 msgr2=0x556781b7fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f952d45e640 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b82450 0x556781b7fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f952d45e640 1 -- 172.21.15.16:0/2319811098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77b90 msgr2=0x556781b7f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f952d45e640 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77b90 0x556781b7f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f952d45e640 1 -- 172.21.15.16:0/2319811098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567817e9580 con 0x556781b77270 2024-09-19T09:29:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.436+0000 7f952dc5f640 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b82450 0x556781b7fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f952d45e640 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77270 0x556781b7f380 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f95240d0100 tx=0x7f952409c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1f4f2e92941893 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9510ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9510ff9640 1 -- 172.21.15.16:0/2319811098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952409caf0 con 0x556781b77270 2024-09-19T09:29:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9510ff9640 1 -- 172.21.15.16:0/2319811098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f95240b52b0 con 0x556781b77270 2024-09-19T09:29:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9531a9b740 1 -- 172.21.15.16:0/2319811098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95240a07a0 con 0x556781b77270 2024-09-19T09:29:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9510ff9640 1 -- 172.21.15.16:0/2319811098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f952409c780 con 0x556781b77270 2024-09-19T09:29:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9531a9b740 10 client.15894 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9531a9b740 1 -- 172.21.15.16:0/2319811098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5567818b3960 con 0x556781b77270 2024-09-19T09:29:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9510ff9640 1 -- 172.21.15.16:0/2319811098 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95240a07a0 con 0x556781b77270 2024-09-19T09:29:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9510ff9640 1 -- 172.21.15.16:0/2319811098 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9524018a90 con 0x556781b77270 2024-09-19T09:29:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9531a9b740 10 client.15894 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9531a9b740 10 client.15894 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:29:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9531a9b740 20 client.15894 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9531a9b740 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x556781b91e10 0x556781b94200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9531a9b740 4 client.15894 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.437+0000 7f9531a9b740 1 -- 172.21.15.16:0/2319811098 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556781ae7b90 con 0x556781b91e10 2024-09-19T09:29:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.438+0000 7f952cc5d640 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x556781b91e10 0x556781b94200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.438+0000 7f952cc5d640 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x556781b91e10 0x556781b94200 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec31355d7494c20 server_cookie=cc4271ae0b64484e in_seq=0 out_seq=0 2024-09-19T09:29:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.438+0000 7f9510ff9640 10 client.15894 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.439+0000 7f9510ff9640 1 -- 172.21.15.16:0/2319811098 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556781ae7b90 con 0x556781b91e10 2024-09-19T09:29:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.439+0000 7f9510ff9640 10 client.15894 handle_command_reply: tid=0 2024-09-19T09:29:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.439+0000 7f9531a9b740 1 client.15894 shutdown 2024-09-19T09:29:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.439+0000 7f9531a9b740 1 -- 172.21.15.16:0/2319811098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77270 msgr2=0x556781b7f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.439+0000 7f9531a9b740 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77270 0x556781b7f380 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f95240d0100 tx=0x7f952409c520 comp rx=0 tx=0).stop 2024-09-19T09:29:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.439+0000 7f9531a9b740 1 -- 172.21.15.16:0/2319811098 shutdown_connections 2024-09-19T09:29:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.439+0000 7f9531a9b740 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x556781b91e10 0x556781b94200 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.439+0000 7f9531a9b740 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b82450 0x556781b7fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.439+0000 7f9531a9b740 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77b90 0x556781b7f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.439+0000 7f9531a9b740 1 --2- 172.21.15.16:0/2319811098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77270 0x556781b7f380 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.439+0000 7f9531a9b740 1 -- 172.21.15.16:0/2319811098 >> 172.21.15.16:0/2319811098 conn(0x556781b73800 msgr2=0x556781b70750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.440+0000 7f9510ff9640 0 client.15894 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.440+0000 7f9531a9b740 1 -- 172.21.15.16:0/2319811098 shutdown_connections 2024-09-19T09:29:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.440+0000 7f9531a9b740 1 -- 172.21.15.16:0/2319811098 wait complete. 2024-09-19T09:29:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.440+0000 7f9531a9b740 20 client.15894 trim_cache size 0 max 16384 2024-09-19T09:29:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.457+0000 7f9531a9b740 1 Processor -- start 2024-09-19T09:29:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.457+0000 7f9531a9b740 1 -- start start 2024-09-19T09:29:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f9531a9b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b73800 0x556781b7a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f9531a9b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b77270 0x556781b8db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f9531a9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 0x556781b96e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f9531a9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95281ab4b0 con 0x556781b77b90 2024-09-19T09:29:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f9531a9b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f95281acff0 con 0x556781b73800 2024-09-19T09:29:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f9531a9b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95281abb80 con 0x556781b77270 2024-09-19T09:29:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f952dc5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 0x556781b96e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f952dc5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 0x556781b96e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39966/0 (socket says 172.21.15.16:39966) 2024-09-19T09:29:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f952d45e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b73800 0x556781b7a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f952dc5f640 1 -- 172.21.15.16:0/1721903865 learned_addr learned my addr 172.21.15.16:0/1721903865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f952cc5d640 1 --2- 172.21.15.16:0/1721903865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b77270 0x556781b8db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f952dc5f640 1 -- 172.21.15.16:0/1721903865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b77270 msgr2=0x556781b8db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f952dc5f640 1 --2- 172.21.15.16:0/1721903865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b77270 0x556781b8db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f952dc5f640 1 -- 172.21.15.16:0/1721903865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b73800 msgr2=0x556781b7a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f952dc5f640 1 --2- 172.21.15.16:0/1721903865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b73800 0x556781b7a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f952dc5f640 1 -- 172.21.15.16:0/1721903865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556781b7e330 con 0x556781b77b90 2024-09-19T09:29:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.458+0000 7f952d45e640 1 --2- 172.21.15.16:0/1721903865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b73800 0x556781b7a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.459+0000 7f952dc5f640 1 --2- 172.21.15.16:0/1721903865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 0x556781b96e40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9520031be0 tx=0x7f952003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6233261e9090f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.459+0000 7f9512ffd640 1 -- 172.21.15.16:0/1721903865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520004040 con 0x556781b77b90 2024-09-19T09:29:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.459+0000 7f9512ffd640 1 -- 172.21.15.16:0/1721903865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9520036040 con 0x556781b77b90 2024-09-19T09:29:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.459+0000 7f9512ffd640 1 -- 172.21.15.16:0/1721903865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520030bc0 con 0x556781b77b90 2024-09-19T09:29:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.460+0000 7f9531a9b740 1 -- 172.21.15.16:0/1721903865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 msgr2=0x556781b96e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.460+0000 7f9531a9b740 1 --2- 172.21.15.16:0/1721903865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 0x556781b96e40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9520031be0 tx=0x7f952003f000 comp rx=0 tx=0).stop 2024-09-19T09:29:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.460+0000 7f9531a9b740 1 -- 172.21.15.16:0/1721903865 shutdown_connections 2024-09-19T09:29:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.460+0000 7f9531a9b740 1 --2- 172.21.15.16:0/1721903865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b77270 0x556781b8db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.460+0000 7f9531a9b740 1 --2- 172.21.15.16:0/1721903865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b77b90 0x556781b96e40 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.460+0000 7f9531a9b740 1 --2- 172.21.15.16:0/1721903865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b73800 0x556781b7a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.460+0000 7f9531a9b740 1 -- 172.21.15.16:0/1721903865 >> 172.21.15.16:0/1721903865 conn(0x556781b7ad70 msgr2=0x556781b8d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:05.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.460+0000 7f9531a9b740 1 -- 172.21.15.16:0/1721903865 shutdown_connections 2024-09-19T09:29:05.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.460+0000 7f9531a9b740 1 -- 172.21.15.16:0/1721903865 wait complete. 2024-09-19T09:29:05.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.460+0000 7f9531a9b740 1 -- start start 2024-09-19T09:29:05.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.460+0000 7f9531a9b740 1 Processor -- start 2024-09-19T09:29:05.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.461+0000 7f9531a9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b73800 0x556781b92e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.461+0000 7f9531a9b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b77270 0x556781b93380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.461+0000 7f9531a9b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77b90 0x556781b938c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.461+0000 7f9531a9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95281ac920 con 0x556781b73800 2024-09-19T09:29:05.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.461+0000 7f9531a9b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f95281ac250 con 0x556781b77b90 2024-09-19T09:29:05.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.461+0000 7f9531a9b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95281a9470 con 0x556781b77270 2024-09-19T09:29:05.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.461+0000 7f9531a9b740 10 client.0 _ll_register_callbacks cb 0x7f9524010840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.461+0000 7f952d45e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b73800 0x556781b92e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.461+0000 7f952d45e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b73800 0x556781b92e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39968/0 (socket says 172.21.15.16:39968) 2024-09-19T09:29:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.461+0000 7f952d45e640 1 -- 172.21.15.16:0/2509476516 learned_addr learned my addr 172.21.15.16:0/2509476516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.461+0000 7f952cc5d640 1 --2- 172.21.15.16:0/2509476516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b77270 0x556781b93380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f952dc5f640 1 --2- 172.21.15.16:0/2509476516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77b90 0x556781b938c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f952d45e640 1 -- 172.21.15.16:0/2509476516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b77270 msgr2=0x556781b93380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f952d45e640 1 --2- 172.21.15.16:0/2509476516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b77270 0x556781b93380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f952d45e640 1 -- 172.21.15.16:0/2509476516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77b90 msgr2=0x556781b938c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f952d45e640 1 --2- 172.21.15.16:0/2509476516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77b90 0x556781b938c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f952d45e640 1 -- 172.21.15.16:0/2509476516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556781bf99d0 con 0x556781b73800 2024-09-19T09:29:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f952d45e640 1 --2- 172.21.15.16:0/2509476516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b73800 0x556781b92e40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9524098850 tx=0x7f952409c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d18d42b3df3926 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f9510ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f9510ff9640 1 -- 172.21.15.16:0/2509476516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952409c1c0 con 0x556781b73800 2024-09-19T09:29:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f9510ff9640 1 -- 172.21.15.16:0/2509476516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9524098140 con 0x556781b73800 2024-09-19T09:29:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f9531a9b740 1 -- 172.21.15.16:0/2509476516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9520036040 con 0x556781b73800 2024-09-19T09:29:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f9510ff9640 1 -- 172.21.15.16:0/2509476516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9524018ca0 con 0x556781b73800 2024-09-19T09:29:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f9531a9b740 10 client.15900 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.462+0000 7f9531a9b740 1 -- 172.21.15.16:0/2509476516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556781bfacf0 con 0x556781b73800 2024-09-19T09:29:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.463+0000 7f9510ff9640 1 -- 172.21.15.16:0/2509476516 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9520036040 con 0x556781b73800 2024-09-19T09:29:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.463+0000 7f9510ff9640 1 -- 172.21.15.16:0/2509476516 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f952409b4b0 con 0x556781b73800 2024-09-19T09:29:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.463+0000 7f9531a9b740 10 client.15900 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.463+0000 7f9531a9b740 10 client.15900 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:29:05.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.463+0000 7f9531a9b740 20 client.15900 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:05.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.463+0000 7f9531a9b740 1 --2- 172.21.15.16:0/2509476516 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x556781b82450 0x556781bfd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:05.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.463+0000 7f9531a9b740 4 client.15900 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9f439ef8", "release": true, "await": true}] 2024-09-19T09:29:05.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.463+0000 7f9531a9b740 1 -- 172.21.15.16:0/2509476516 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9f439ef8", "release": true, "await": true}) -- 0x556781ae20a0 con 0x556781b82450 2024-09-19T09:29:05.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.463+0000 7f952cc5d640 1 --2- 172.21.15.16:0/2509476516 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x556781b82450 0x556781bfd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:05.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.464+0000 7f9510ff9640 10 client.15900 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:05.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.464+0000 7f952cc5d640 1 --2- 172.21.15.16:0/2509476516 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x556781b82450 0x556781bfd290 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9265f4007d80332 server_cookie=ca0294ce36e7cd3f in_seq=0 out_seq=0 2024-09-19T09:29:05.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9510ff9640 1 -- 172.21.15.16:0/2509476516 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556781ae20a0 con 0x556781b82450 2024-09-19T09:29:05.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9510ff9640 10 client.15900 handle_command_reply: tid=0 2024-09-19T09:29:05.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9531a9b740 1 client.15900 shutdown 2024-09-19T09:29:05.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9531a9b740 1 -- 172.21.15.16:0/2509476516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b73800 msgr2=0x556781b92e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:05.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9531a9b740 1 --2- 172.21.15.16:0/2509476516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b73800 0x556781b92e40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9524098850 tx=0x7f952409c4b0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9531a9b740 1 -- 172.21.15.16:0/2509476516 shutdown_connections 2024-09-19T09:29:05.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9531a9b740 1 --2- 172.21.15.16:0/2509476516 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x556781b82450 0x556781bfd290 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9531a9b740 1 --2- 172.21.15.16:0/2509476516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556781b77270 0x556781b93380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9531a9b740 1 --2- 172.21.15.16:0/2509476516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556781b77b90 0x556781b938c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9531a9b740 1 --2- 172.21.15.16:0/2509476516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556781b73800 0x556781b92e40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9531a9b740 1 -- 172.21.15.16:0/2509476516 >> 172.21.15.16:0/2509476516 conn(0x556781b7ad70 msgr2=0x556781b79450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9510ff9640 0 client.15900 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9531a9b740 1 -- 172.21.15.16:0/2509476516 shutdown_connections 2024-09-19T09:29:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9531a9b740 1 -- 172.21.15.16:0/2509476516 wait complete. 2024-09-19T09:29:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.467+0000 7f9531a9b740 20 client.15900 trim_cache size 0 max 16384 2024-09-19T09:29:05.471 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:29:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-19T09:29:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24500, 2024-09-19T09:29:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 8, 2024-09-19T09:29:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-19T09:29:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "9f439ef8": { 2024-09-19T09:29:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 8, 2024-09-19T09:29:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-19T09:29:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-19T09:29:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-19T09:29:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-19T09:29:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-19T09:29:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.3, 2024-09-19T09:29:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-19T09:29:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-19T09:29:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-19T09:29:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-19T09:29:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-19T09:29:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-19T09:29:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:05.473 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:29:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.468+0000 7f952f6c2640 1 -- 172.21.15.16:0/3709519947 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f95140a3240 msgr2=0x7f95140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.468+0000 7f952f6c2640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f95140a3240 0x7f95140a5700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9520005e30 tx=0x7f9520005dc0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.468+0000 7f952f6c2640 1 -- 172.21.15.16:0/3709519947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9528100a80 msgr2=0x7f95281972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.468+0000 7f952f6c2640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9528100a80 0x7f95281972f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f952400e9f0 tx=0x7f952400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.468+0000 7f952f6c2640 1 -- 172.21.15.16:0/3709519947 shutdown_connections 2024-09-19T09:29:05.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.468+0000 7f952f6c2640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f95140a3240 0x7f95140a5700 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.468+0000 7f952f6c2640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9528101420 0x7f9528197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.468+0000 7f952f6c2640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9528106a80 0x7f952819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.468+0000 7f952f6c2640 1 --2- 172.21.15.16:0/3709519947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9528100a80 0x7f95281972f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:05.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.468+0000 7f952f6c2640 1 -- 172.21.15.16:0/3709519947 >> 172.21.15.16:0/3709519947 conn(0x7f95280fc7a0 msgr2=0x7f95280fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:05.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.468+0000 7f952f6c2640 1 -- 172.21.15.16:0/3709519947 shutdown_connections 2024-09-19T09:29:05.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:05.468+0000 7f952f6c2640 1 -- 172.21.15.16:0/3709519947 wait complete. 2024-09-19T09:29:05.476 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T09:29:05.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T09:29:05.515 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-09-19T09:29:05.515 DEBUG:teuthology.orchestra.run.smithi016:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-09-19T09:29:06.195 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9f439ef8', total seconds elapsed: 25.8 2024-09-19T09:29:06.195 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T09:29:06.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:29:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:06.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.747+0000 7f0413913640 1 -- 172.21.15.16:0/1919201221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040c102780 msgr2=0x7f040c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.747+0000 7f0413913640 1 --2- 172.21.15.16:0/1919201221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040c102780 0x7f040c1067d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009900 tx=0x7f03fc02a410 comp rx=0 tx=0).stop 2024-09-19T09:29:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.747+0000 7f0413913640 1 -- 172.21.15.16:0/1919201221 shutdown_connections 2024-09-19T09:29:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.747+0000 7f0413913640 1 --2- 172.21.15.16:0/1919201221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c106f00 0x7f040c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.747+0000 7f0413913640 1 --2- 172.21.15.16:0/1919201221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040c102780 0x7f040c1067d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.747+0000 7f0413913640 1 --2- 172.21.15.16:0/1919201221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f040c101dd0 0x7f040c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.747+0000 7f0413913640 1 -- 172.21.15.16:0/1919201221 >> 172.21.15.16:0/1919201221 conn(0x7f040c0fd8b0 msgr2=0x7f040c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.747+0000 7f0413913640 1 -- 172.21.15.16:0/1919201221 shutdown_connections 2024-09-19T09:29:06.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.747+0000 7f0413913640 1 -- 172.21.15.16:0/1919201221 wait complete. 2024-09-19T09:29:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.748+0000 7f0413913640 1 Processor -- start 2024-09-19T09:29:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.748+0000 7f0413913640 1 -- start start 2024-09-19T09:29:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.748+0000 7f0413913640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f040c101dd0 0x7f040c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.748+0000 7f0413913640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c102780 0x7f040c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.748+0000 7f0413913640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040c106f00 0x7f040c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.748+0000 7f0413913640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f040c111f60 con 0x7f040c106f00 2024-09-19T09:29:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.748+0000 7f0413913640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f040c111de0 con 0x7f040c102780 2024-09-19T09:29:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.748+0000 7f0413913640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f040c1120e0 con 0x7f040c101dd0 2024-09-19T09:29:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.748+0000 7f0411eb0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040c106f00 0x7f040c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.748+0000 7f0411eb0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040c106f00 0x7f040c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39986/0 (socket says 172.21.15.16:39986) 2024-09-19T09:29:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.748+0000 7f0411eb0640 1 -- 172.21.15.16:0/3091507860 learned_addr learned my addr 172.21.15.16:0/3091507860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.748+0000 7f0410eae640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c102780 0x7f040c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.749+0000 7f04116af640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f040c101dd0 0x7f040c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.749+0000 7f0411eb0640 1 -- 172.21.15.16:0/3091507860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f040c101dd0 msgr2=0x7f040c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.749+0000 7f0411eb0640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f040c101dd0 0x7f040c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.749+0000 7f0411eb0640 1 -- 172.21.15.16:0/3091507860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c102780 msgr2=0x7f040c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.749+0000 7f0411eb0640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c102780 0x7f040c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.749+0000 7f0411eb0640 1 -- 172.21.15.16:0/3091507860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040c1a68e0 con 0x7f040c106f00 2024-09-19T09:29:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.749+0000 7f0410eae640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c102780 0x7f040c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.749+0000 7f0411eb0640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040c106f00 0x7f040c074f70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f040400eaa0 tx=0x7f040400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:06.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.749+0000 7f04027fc640 1 -- 172.21.15.16:0/3091507860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040400cdb0 con 0x7f040c106f00 2024-09-19T09:29:06.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.749+0000 7f04027fc640 1 -- 172.21.15.16:0/3091507860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0404005020 con 0x7f040c106f00 2024-09-19T09:29:06.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.749+0000 7f0413913640 1 -- 172.21.15.16:0/3091507860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040c1a6bd0 con 0x7f040c106f00 2024-09-19T09:29:06.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.749+0000 7f04027fc640 1 -- 172.21.15.16:0/3091507860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04040116b0 con 0x7f040c106f00 2024-09-19T09:29:06.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.749+0000 7f0413913640 1 -- 172.21.15.16:0/3091507860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040c1a7090 con 0x7f040c106f00 2024-09-19T09:29:06.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.753+0000 7f04027fc640 1 -- 172.21.15.16:0/3091507860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f0404005560 con 0x7f040c106f00 2024-09-19T09:29:06.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.753+0000 7f0415cec740 1 Processor -- start 2024-09-19T09:29:06.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.753+0000 7f0415cec740 1 -- start start 2024-09-19T09:29:06.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.753+0000 7f0415cec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 0x55a4af0bef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.753+0000 7f0415cec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0bf480 0x55a4af0c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.753+0000 7f0415cec740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0c9d40 0x55a4af0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.753+0000 7f0415cec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a4af024280 con 0x55a4af0beb60 2024-09-19T09:29:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.753+0000 7f0415cec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a4aefcbb10 con 0x55a4af0bf480 2024-09-19T09:29:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.753+0000 7f0415cec740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a4af020240 con 0x55a4af0c9d40 2024-09-19T09:29:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.753+0000 7f0410eae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 0x55a4af0bef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.753+0000 7f0410eae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 0x55a4af0bef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39992/0 (socket says 172.21.15.16:39992) 2024-09-19T09:29:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.753+0000 7f0410eae640 1 -- 172.21.15.16:0/3299399408 learned_addr learned my addr 172.21.15.16:0/3299399408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.754+0000 7f04116af640 1 --2- 172.21.15.16:0/3299399408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0bf480 0x55a4af0c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.754+0000 7f0410eae640 1 --2- 172.21.15.16:0/3299399408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0c9d40 0x55a4af0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.754+0000 7f04027fc640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f03f80a3240 0x7f03f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.754+0000 7f0410eae640 1 -- 172.21.15.16:0/3299399408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0c9d40 msgr2=0x55a4af0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.754+0000 7f0410eae640 1 --2- 172.21.15.16:0/3299399408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0c9d40 0x55a4af0cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.754+0000 7f0410eae640 1 -- 172.21.15.16:0/3299399408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0bf480 msgr2=0x55a4af0c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:06.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.754+0000 7f0410eae640 1 --2- 172.21.15.16:0/3299399408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0bf480 0x55a4af0c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.754+0000 7f0410eae640 1 -- 172.21.15.16:0/3299399408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4aefd81f0 con 0x55a4af0beb60 2024-09-19T09:29:06.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.754+0000 7f04116af640 1 --2- 172.21.15.16:0/3299399408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0bf480 0x55a4af0c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:06.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.754+0000 7f0411eb0640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f03f80a3240 0x7f03f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.754+0000 7f04027fc640 1 -- 172.21.15.16:0/3091507860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f04040cfa40 con 0x7f040c106f00 2024-09-19T09:29:06.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.754+0000 7f0410eae640 1 --2- 172.21.15.16:0/3299399408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 0x55a4af0bef40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f03fc005c40 tx=0x7f03fc005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32d5d59e1e606e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.755+0000 7f03f77fe640 1 -- 172.21.15.16:0/3299399408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc044070 con 0x55a4af0beb60 2024-09-19T09:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.755+0000 7f03f77fe640 1 -- 172.21.15.16:0/3299399408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03fc005500 con 0x55a4af0beb60 2024-09-19T09:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.755+0000 7f03f77fe640 1 -- 172.21.15.16:0/3299399408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc035040 con 0x55a4af0beb60 2024-09-19T09:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.755+0000 7f0415cec740 1 -- 172.21.15.16:0/3299399408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 msgr2=0x55a4af0bef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.755+0000 7f0411eb0640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f03f80a3240 0x7f03f80a5700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f04040cd540 tx=0x7f0404066b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.755+0000 7f0415cec740 1 --2- 172.21.15.16:0/3299399408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 0x55a4af0bef40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f03fc005c40 tx=0x7f03fc005bb0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.756+0000 7f0415cec740 1 -- 172.21.15.16:0/3299399408 shutdown_connections 2024-09-19T09:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.756+0000 7f0415cec740 1 --2- 172.21.15.16:0/3299399408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0c9d40 0x55a4af0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.756+0000 7f0415cec740 1 --2- 172.21.15.16:0/3299399408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0bf480 0x55a4af0c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.756+0000 7f0415cec740 1 --2- 172.21.15.16:0/3299399408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 0x55a4af0bef40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.756+0000 7f0415cec740 1 -- 172.21.15.16:0/3299399408 >> 172.21.15.16:0/3299399408 conn(0x55a4af0bb0f0 msgr2=0x55a4af0bb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.756+0000 7f0415cec740 1 -- 172.21.15.16:0/3299399408 shutdown_connections 2024-09-19T09:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.756+0000 7f0415cec740 1 -- 172.21.15.16:0/3299399408 wait complete. 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.756+0000 7f0415cec740 1 -- start start 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.756+0000 7f0415cec740 1 Processor -- start 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0415cec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0beb60 0x55a4af0c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0415cec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0bf480 0x55a4af0c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0415cec740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0c9d40 0x55a4af0c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0415cec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a4af0cc910 con 0x55a4af0bf480 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0415cec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a4aefdef10 con 0x55a4af0beb60 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0415cec740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a4af0cca90 con 0x55a4af0c9d40 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0415cec740 10 client.0 _ll_register_callbacks cb 0x55a4aefa7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f04116af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0bf480 0x55a4af0c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f04116af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0bf480 0x55a4af0c70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39996/0 (socket says 172.21.15.16:39996) 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f04116af640 1 -- 172.21.15.16:0/950700966 learned_addr learned my addr 172.21.15.16:0/950700966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0410eae640 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0beb60 0x55a4af0c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0410eae640 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0c9d40 0x55a4af0c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0410eae640 1 -- 172.21.15.16:0/950700966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0c9d40 msgr2=0x55a4af0c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0410eae640 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0c9d40 0x55a4af0c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0410eae640 1 -- 172.21.15.16:0/950700966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0bf480 msgr2=0x55a4af0c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0410eae640 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0bf480 0x55a4af0c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f0410eae640 1 -- 172.21.15.16:0/950700966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4aed956a0 con 0x55a4af0beb60 2024-09-19T09:29:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.757+0000 7f04116af640 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0bf480 0x55a4af0c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.758+0000 7f0410eae640 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0beb60 0x55a4af0c6ba0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f03fc02a930 tx=0x7f03fc0048e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7457626a58a0b1de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.758+0000 7f03f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-19T09:29:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.758+0000 7f03f4ff9640 1 -- 172.21.15.16:0/950700966 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc044070 con 0x55a4af0beb60 2024-09-19T09:29:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.758+0000 7f03f4ff9640 1 -- 172.21.15.16:0/950700966 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f03fc004a80 con 0x55a4af0beb60 2024-09-19T09:29:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.758+0000 7f0415cec740 1 -- 172.21.15.16:0/950700966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03fc005500 con 0x55a4af0beb60 2024-09-19T09:29:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.758+0000 7f03f4ff9640 1 -- 172.21.15.16:0/950700966 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03fc035040 con 0x55a4af0beb60 2024-09-19T09:29:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.758+0000 7f0415cec740 10 client.25481 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.758+0000 7f0415cec740 1 -- 172.21.15.16:0/950700966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a4af0d84d0 con 0x55a4af0beb60 2024-09-19T09:29:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.758+0000 7f03f4ff9640 1 -- 172.21.15.16:0/950700966 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03fc005500 con 0x55a4af0beb60 2024-09-19T09:29:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.759+0000 7f03f4ff9640 1 -- 172.21.15.16:0/950700966 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f03fc003890 con 0x55a4af0beb60 2024-09-19T09:29:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.759+0000 7f0415cec740 10 client.25481 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.759+0000 7f0415cec740 10 client.25481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.759+0000 7f0415cec740 20 client.25481 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.759+0000 7f0415cec740 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a4af0d9820 0x55a4af0dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.759+0000 7f0415cec740 4 client.25481 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.759+0000 7f0415cec740 1 -- 172.21.15.16:0/950700966 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4af0317f0 con 0x55a4af0d9820 2024-09-19T09:29:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.759+0000 7f04116af640 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a4af0d9820 0x55a4af0dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.760+0000 7f03f4ff9640 10 client.25481 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.760+0000 7f04116af640 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a4af0d9820 0x55a4af0dbc10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14d29730483ddc88 server_cookie=e1b53dc97d05b267 in_seq=0 out_seq=0 2024-09-19T09:29:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f03f4ff9640 1 -- 172.21.15.16:0/950700966 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4af0317f0 con 0x55a4af0d9820 2024-09-19T09:29:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f03f4ff9640 10 client.25481 handle_command_reply: tid=0 2024-09-19T09:29:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f0415cec740 1 client.25481 shutdown 2024-09-19T09:29:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f0415cec740 1 -- 172.21.15.16:0/950700966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0beb60 msgr2=0x55a4af0c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f0415cec740 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0beb60 0x55a4af0c6ba0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f03fc02a930 tx=0x7f03fc0048e0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f0415cec740 1 -- 172.21.15.16:0/950700966 shutdown_connections 2024-09-19T09:29:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f0415cec740 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a4af0d9820 0x55a4af0dbc10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f0415cec740 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0c9d40 0x55a4af0c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f0415cec740 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0bf480 0x55a4af0c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f0415cec740 1 --2- 172.21.15.16:0/950700966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0beb60 0x55a4af0c6ba0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f03f4ff9640 0 client.25481 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:06.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f0415cec740 1 -- 172.21.15.16:0/950700966 >> 172.21.15.16:0/950700966 conn(0x55a4af0bb0f0 msgr2=0x55a4af0c0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:06.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f0415cec740 1 -- 172.21.15.16:0/950700966 shutdown_connections 2024-09-19T09:29:06.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f0415cec740 1 -- 172.21.15.16:0/950700966 wait complete. 2024-09-19T09:29:06.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.761+0000 7f0415cec740 20 client.25481 trim_cache size 0 max 16384 2024-09-19T09:29:06.778 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.780+0000 7f0415cec740 1 Processor -- start 2024-09-19T09:29:06.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.780+0000 7f0415cec740 1 -- start start 2024-09-19T09:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.780+0000 7f0415cec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 0x55a4af0c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.780+0000 7f0415cec740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0bf480 0x55a4af0d5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.780+0000 7f0415cec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0c9d40 0x55a4af0dec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.780+0000 7f0415cec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f040c1af7a0 con 0x55a4af0beb60 2024-09-19T09:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.780+0000 7f0415cec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f040c1b12e0 con 0x55a4af0c9d40 2024-09-19T09:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.780+0000 7f0415cec740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f040c1afe70 con 0x55a4af0bf480 2024-09-19T09:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.780+0000 7f0410eae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 0x55a4af0c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.781+0000 7f0410eae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 0x55a4af0c15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40010/0 (socket says 172.21.15.16:40010) 2024-09-19T09:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.781+0000 7f0410eae640 1 -- 172.21.15.16:0/2607793945 learned_addr learned my addr 172.21.15.16:0/2607793945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.781+0000 7f04116af640 1 --2- 172.21.15.16:0/2607793945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0bf480 0x55a4af0d5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.781+0000 7f0410eae640 1 --2- 172.21.15.16:0/2607793945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0c9d40 0x55a4af0dec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.781+0000 7f0410eae640 1 -- 172.21.15.16:0/2607793945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0bf480 msgr2=0x55a4af0d5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.781+0000 7f0410eae640 1 --2- 172.21.15.16:0/2607793945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0bf480 0x55a4af0d5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.781+0000 7f0410eae640 1 -- 172.21.15.16:0/2607793945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0c9d40 msgr2=0x55a4af0dec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.781+0000 7f0410eae640 1 --2- 172.21.15.16:0/2607793945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0c9d40 0x55a4af0dec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.781+0000 7f0410eae640 1 -- 172.21.15.16:0/2607793945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4af0baf20 con 0x55a4af0beb60 2024-09-19T09:29:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.781+0000 7f0410eae640 1 --2- 172.21.15.16:0/2607793945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 0x55a4af0c15c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f03fc001eb0 tx=0x7f03fc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a560ad4a498645 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.781+0000 7f03f6ffd640 1 -- 172.21.15.16:0/2607793945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc044040 con 0x55a4af0beb60 2024-09-19T09:29:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.781+0000 7f03f6ffd640 1 -- 172.21.15.16:0/2607793945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03fc005460 con 0x55a4af0beb60 2024-09-19T09:29:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.781+0000 7f03f6ffd640 1 -- 172.21.15.16:0/2607793945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc004950 con 0x55a4af0beb60 2024-09-19T09:29:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.782+0000 7f0415cec740 1 -- 172.21.15.16:0/2607793945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 msgr2=0x55a4af0c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.782+0000 7f0415cec740 1 --2- 172.21.15.16:0/2607793945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 0x55a4af0c15c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f03fc001eb0 tx=0x7f03fc005fd0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.782+0000 7f0415cec740 1 -- 172.21.15.16:0/2607793945 shutdown_connections 2024-09-19T09:29:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.782+0000 7f0415cec740 1 --2- 172.21.15.16:0/2607793945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0bf480 0x55a4af0d5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.782+0000 7f0415cec740 1 --2- 172.21.15.16:0/2607793945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0c9d40 0x55a4af0dec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.782+0000 7f0415cec740 1 --2- 172.21.15.16:0/2607793945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0beb60 0x55a4af0c15c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.782+0000 7f0415cec740 1 -- 172.21.15.16:0/2607793945 >> 172.21.15.16:0/2607793945 conn(0x55a4af0bb0f0 msgr2=0x55a4af0d4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.782+0000 7f0415cec740 1 -- 172.21.15.16:0/2607793945 shutdown_connections 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.782+0000 7f0415cec740 1 -- 172.21.15.16:0/2607793945 wait complete. 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.783+0000 7f0415cec740 1 -- start start 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.783+0000 7f0415cec740 1 Processor -- start 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.783+0000 7f0415cec740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0beb60 0x55a4af0df250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.783+0000 7f0415cec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0bf480 0x55a4af0dc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.783+0000 7f0415cec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0c9d40 0x55a4af0dcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.783+0000 7f0415cec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f040c1b0c10 con 0x55a4af0c9d40 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.783+0000 7f0415cec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f040c1b0540 con 0x55a4af0bf480 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.783+0000 7f0415cec740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f040c1ad760 con 0x55a4af0beb60 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f0415cec740 10 client.0 _ll_register_callbacks cb 0x7f03fc0315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f0410eae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0c9d40 0x55a4af0dcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f0410eae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0c9d40 0x55a4af0dcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40020/0 (socket says 172.21.15.16:40020) 2024-09-19T09:29:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f0410eae640 1 -- 172.21.15.16:0/3261536140 learned_addr learned my addr 172.21.15.16:0/3261536140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f0410eae640 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0beb60 0x55a4af0df250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f04116af640 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0bf480 0x55a4af0dc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f0410eae640 1 -- 172.21.15.16:0/3261536140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0beb60 msgr2=0x55a4af0df250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f0410eae640 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0beb60 0x55a4af0df250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f0410eae640 1 -- 172.21.15.16:0/3261536140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0bf480 msgr2=0x55a4af0dc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f0410eae640 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0bf480 0x55a4af0dc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f0410eae640 1 -- 172.21.15.16:0/3261536140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4af0dd220 con 0x55a4af0c9d40 2024-09-19T09:29:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f04116af640 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0bf480 0x55a4af0dc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f0410eae640 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0c9d40 0x55a4af0dcb30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f03fc033000 tx=0x7f03fc040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3459cfe217577653 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f03f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.784+0000 7f03f4ff9640 1 -- 172.21.15.16:0/3261536140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc06a020 con 0x55a4af0c9d40 2024-09-19T09:29:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.785+0000 7f03f4ff9640 1 -- 172.21.15.16:0/3261536140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f03fc003930 con 0x55a4af0c9d40 2024-09-19T09:29:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.785+0000 7f0415cec740 1 -- 172.21.15.16:0/3261536140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03fc005460 con 0x55a4af0c9d40 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.785+0000 7f03f4ff9640 1 -- 172.21.15.16:0/3261536140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03fc003b30 con 0x55a4af0c9d40 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.785+0000 7f0415cec740 10 client.15918 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.785+0000 7f0415cec740 1 -- 172.21.15.16:0/3261536140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a4af144060 con 0x55a4af0c9d40 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.785+0000 7f03f4ff9640 1 -- 172.21.15.16:0/3261536140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03fc005460 con 0x55a4af0c9d40 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.785+0000 7f03f4ff9640 1 -- 172.21.15.16:0/3261536140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f03fc06f070 con 0x55a4af0c9d40 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.785+0000 7f0415cec740 10 client.15918 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.785+0000 7f0415cec740 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.785+0000 7f0415cec740 20 client.15918 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.785+0000 7f0415cec740 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a4af144530 0x55a4af146920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.785+0000 7f0415cec740 4 client.15918 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.785+0000 7f0415cec740 1 -- 172.21.15.16:0/3261536140 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a4af02bd00 con 0x55a4af144530 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.786+0000 7f04116af640 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a4af144530 0x55a4af146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.786+0000 7f04116af640 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a4af144530 0x55a4af146920 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84fb06cce50cf08f server_cookie=11ad48bb484f4c7c in_seq=0 out_seq=0 2024-09-19T09:29:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.786+0000 7f03f4ff9640 10 client.15918 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.787+0000 7f03f4ff9640 1 -- 172.21.15.16:0/3261536140 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a4af02bd00 con 0x55a4af144530 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.787+0000 7f03f4ff9640 10 client.15918 handle_command_reply: tid=0 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.787+0000 7f0415cec740 1 client.15918 shutdown 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.787+0000 7f0415cec740 1 -- 172.21.15.16:0/3261536140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0c9d40 msgr2=0x55a4af0dcb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.787+0000 7f0415cec740 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0c9d40 0x55a4af0dcb30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f03fc033000 tx=0x7f03fc040040 comp rx=0 tx=0).stop 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.787+0000 7f0415cec740 1 -- 172.21.15.16:0/3261536140 shutdown_connections 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.787+0000 7f0415cec740 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a4af144530 0x55a4af146920 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.787+0000 7f0415cec740 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4af0c9d40 0x55a4af0dcb30 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.787+0000 7f0415cec740 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a4af0bf480 0x55a4af0dc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.787+0000 7f0415cec740 1 --2- 172.21.15.16:0/3261536140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4af0beb60 0x55a4af0df250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.787+0000 7f0415cec740 1 -- 172.21.15.16:0/3261536140 >> 172.21.15.16:0/3261536140 conn(0x55a4af0bb0f0 msgr2=0x55a4af0d5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.788+0000 7f03f4ff9640 0 client.15918 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.788+0000 7f0415cec740 1 -- 172.21.15.16:0/3261536140 shutdown_connections 2024-09-19T09:29:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.788+0000 7f0415cec740 1 -- 172.21.15.16:0/3261536140 wait complete. 2024-09-19T09:29:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.788+0000 7f0415cec740 20 client.15918 trim_cache size 0 max 16384 2024-09-19T09:29:06.793 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:29:06.793 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-19T09:29:06.793 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "f34a5b49-930f-46bd-8468-f703eea30ad5", 2024-09-19T09:29:06.793 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-19T09:29:06.793 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:29:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.789+0000 7f0413913640 1 -- 172.21.15.16:0/3091507860 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f03f80a3240 msgr2=0x7f03f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.789+0000 7f0413913640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f03f80a3240 0x7f03f80a5700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f04040cd540 tx=0x7f0404066b50 comp rx=0 tx=0).stop 2024-09-19T09:29:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.789+0000 7f0413913640 1 -- 172.21.15.16:0/3091507860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040c106f00 msgr2=0x7f040c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.789+0000 7f0413913640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040c106f00 0x7f040c074f70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f040400eaa0 tx=0x7f040400ef60 comp rx=0 tx=0).stop 2024-09-19T09:29:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.789+0000 7f0413913640 1 -- 172.21.15.16:0/3091507860 shutdown_connections 2024-09-19T09:29:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.789+0000 7f0413913640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040c106f00 0x7f040c074f70 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.789+0000 7f0413913640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c102780 0x7f040c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.789+0000 7f0413913640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f03f80a3240 0x7f03f80a5700 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.789+0000 7f0413913640 1 --2- 172.21.15.16:0/3091507860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f040c101dd0 0x7f040c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:06.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.789+0000 7f0413913640 1 -- 172.21.15.16:0/3091507860 >> 172.21.15.16:0/3091507860 conn(0x7f040c0fd8b0 msgr2=0x7f040c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:06.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.790+0000 7f0413913640 1 -- 172.21.15.16:0/3091507860 shutdown_connections 2024-09-19T09:29:06.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:06.790+0000 7f0413913640 1 -- 172.21.15.16:0/3091507860 wait complete. 2024-09-19T09:29:07.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:07 smithi142 ceph-mon[35032]: pgmap v383: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-19T09:29:07.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:07 smithi097 ceph-mon[34864]: pgmap v383: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-19T09:29:07.449 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:07 smithi016 ceph-mon[30979]: pgmap v383: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-19T09:29:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.447+0000 7f206c3a9640 1 -- 172.21.15.16:0/1298747733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206410c170 msgr2=0x7f206410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.447+0000 7f206c3a9640 1 --2- 172.21.15.16:0/1298747733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206410c170 0x7f206410e610 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f205c009ca0 tx=0x7f205c02a950 comp rx=0 tx=0).stop 2024-09-19T09:29:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.447+0000 7f206c3a9640 1 -- 172.21.15.16:0/1298747733 shutdown_connections 2024-09-19T09:29:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.447+0000 7f206c3a9640 1 --2- 172.21.15.16:0/1298747733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206410c170 0x7f206410e610 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.447+0000 7f206c3a9640 1 --2- 172.21.15.16:0/1298747733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2064103660 0x7f206410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.447+0000 7f206c3a9640 1 --2- 172.21.15.16:0/1298747733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2064102d40 0x7f2064103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.447+0000 7f206c3a9640 1 -- 172.21.15.16:0/1298747733 >> 172.21.15.16:0/1298747733 conn(0x7f20640fc7a0 msgr2=0x7f20640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.447+0000 7f206c3a9640 1 -- 172.21.15.16:0/1298747733 shutdown_connections 2024-09-19T09:29:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.447+0000 7f206c3a9640 1 -- 172.21.15.16:0/1298747733 wait complete. 2024-09-19T09:29:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.448+0000 7f206c3a9640 1 Processor -- start 2024-09-19T09:29:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.448+0000 7f206c3a9640 1 -- start start 2024-09-19T09:29:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.448+0000 7f206c3a9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2064102d40 0x7f2064198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.448+0000 7f206c3a9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2064103660 0x7f2064198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.448+0000 7f206c3a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206410c170 0x7f206419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.448+0000 7f206c3a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2064111f70 con 0x7f206410c170 2024-09-19T09:29:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.448+0000 7f206c3a9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2064111df0 con 0x7f2064103660 2024-09-19T09:29:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.448+0000 7f206c3a9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f20641120f0 con 0x7f2064102d40 2024-09-19T09:29:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.448+0000 7f206a946640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206410c170 0x7f206419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.448+0000 7f206a946640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206410c170 0x7f206419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40038/0 (socket says 172.21.15.16:40038) 2024-09-19T09:29:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.448+0000 7f206a946640 1 -- 172.21.15.16:0/163975200 learned_addr learned my addr 172.21.15.16:0/163975200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.448+0000 7f2069944640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2064103660 0x7f2064198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.448+0000 7f206a145640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2064102d40 0x7f2064198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.449+0000 7f206a946640 1 -- 172.21.15.16:0/163975200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2064102d40 msgr2=0x7f2064198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.449+0000 7f206a946640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2064102d40 0x7f2064198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.449+0000 7f206a946640 1 -- 172.21.15.16:0/163975200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2064103660 msgr2=0x7f2064198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.449+0000 7f206a946640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2064103660 0x7f2064198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.449+0000 7f206a946640 1 -- 172.21.15.16:0/163975200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206419d3c0 con 0x7f206410c170 2024-09-19T09:29:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.449+0000 7f206a946640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206410c170 0x7f206419cce0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f205c02ae70 tx=0x7f205c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.449+0000 7f205b7fe640 1 -- 172.21.15.16:0/163975200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c034040 con 0x7f206410c170 2024-09-19T09:29:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.449+0000 7f205b7fe640 1 -- 172.21.15.16:0/163975200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f205c030920 con 0x7f206410c170 2024-09-19T09:29:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.449+0000 7f206c3a9640 1 -- 172.21.15.16:0/163975200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206419d650 con 0x7f206410c170 2024-09-19T09:29:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.449+0000 7f205b7fe640 1 -- 172.21.15.16:0/163975200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c045480 con 0x7f206410c170 2024-09-19T09:29:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.449+0000 7f206c3a9640 1 -- 172.21.15.16:0/163975200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20641a4f90 con 0x7f206410c170 2024-09-19T09:29:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.450+0000 7f206c3a9640 1 -- 172.21.15.16:0/163975200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f206406b070 con 0x7f206410c170 2024-09-19T09:29:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.451+0000 7f205b7fe640 1 -- 172.21.15.16:0/163975200 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f205c037020 con 0x7f206410c170 2024-09-19T09:29:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.452+0000 7f205b7fe640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f20500a3240 0x7f20500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.452+0000 7f205b7fe640 1 -- 172.21.15.16:0/163975200 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f205c0f0830 con 0x7f206410c170 2024-09-19T09:29:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.452+0000 7f206a145640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f20500a3240 0x7f20500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.452+0000 7f206a145640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f20500a3240 0x7f20500a5700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2060005fd0 tx=0x7f2060005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.456+0000 7f205b7fe640 1 -- 172.21.15.16:0/163975200 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f205c0bd550 con 0x7f206410c170 2024-09-19T09:29:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.672+0000 7f206c3a9640 1 -- 172.21.15.16:0/163975200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20641a5160 con 0x7f206410c170 2024-09-19T09:29:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.673+0000 7f205b7fe640 1 -- 172.21.15.16:0/163975200 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f205c0bce50 con 0x7f206410c170 2024-09-19T09:29:07.675 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:29:07.676 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:29:07.676 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:29:07.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.675+0000 7f206c3a9640 1 -- 172.21.15.16:0/163975200 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f20500a3240 msgr2=0x7f20500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:07.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.675+0000 7f206c3a9640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f20500a3240 0x7f20500a5700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2060005fd0 tx=0x7f2060005e20 comp rx=0 tx=0).stop 2024-09-19T09:29:07.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.675+0000 7f206c3a9640 1 -- 172.21.15.16:0/163975200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206410c170 msgr2=0x7f206419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:07.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.675+0000 7f206c3a9640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206410c170 0x7f206419cce0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f205c02ae70 tx=0x7f205c02f5b0 comp rx=0 tx=0).stop 2024-09-19T09:29:07.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.675+0000 7f206c3a9640 1 -- 172.21.15.16:0/163975200 shutdown_connections 2024-09-19T09:29:07.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.675+0000 7f206c3a9640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206410c170 0x7f206419cce0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:07.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.675+0000 7f206c3a9640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2064103660 0x7f2064198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:07.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.675+0000 7f206c3a9640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f20500a3240 0x7f20500a5700 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:07.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.675+0000 7f206c3a9640 1 --2- 172.21.15.16:0/163975200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2064102d40 0x7f2064198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:07.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.675+0000 7f206c3a9640 1 -- 172.21.15.16:0/163975200 >> 172.21.15.16:0/163975200 conn(0x7f20640fc7a0 msgr2=0x7f20640fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:07.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.675+0000 7f206c3a9640 1 -- 172.21.15.16:0/163975200 shutdown_connections 2024-09-19T09:29:07.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:07.675+0000 7f206c3a9640 1 -- 172.21.15.16:0/163975200 wait complete. 2024-09-19T09:29:07.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:29:08.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:08 smithi097 ceph-mon[34864]: scrub queued for path: / 2024-09-19T09:29:08.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:08 smithi097 ceph-mon[34864]: scrub summary: idle+waiting paths [/] 2024-09-19T09:29:08.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:08 smithi097 ceph-mon[34864]: scrub summary: active paths [/] 2024-09-19T09:29:08.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:08 smithi097 ceph-mon[34864]: scrub summary: idle+waiting paths [/] 2024-09-19T09:29:08.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:08 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/163975200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:08.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:08 smithi016 ceph-mon[30979]: scrub queued for path: / 2024-09-19T09:29:08.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:08 smithi016 ceph-mon[30979]: scrub summary: idle+waiting paths [/] 2024-09-19T09:29:08.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:08 smithi016 ceph-mon[30979]: scrub summary: active paths [/] 2024-09-19T09:29:08.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:08 smithi016 ceph-mon[30979]: scrub summary: idle+waiting paths [/] 2024-09-19T09:29:08.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:08 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/163975200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:08.542 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T09:29:08.543 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T09:29:08.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:08 smithi142 ceph-mon[35032]: scrub queued for path: / 2024-09-19T09:29:08.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:08 smithi142 ceph-mon[35032]: scrub summary: idle+waiting paths [/] 2024-09-19T09:29:08.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:08 smithi142 ceph-mon[35032]: scrub summary: active paths [/] 2024-09-19T09:29:08.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:08 smithi142 ceph-mon[35032]: scrub summary: idle+waiting paths [/] 2024-09-19T09:29:08.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:08 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/163975200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:09.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.010+0000 7fe3cc68c640 1 -- 172.21.15.16:0/356122554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3c4102570 msgr2=0x7fe3c41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:09.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.010+0000 7fe3cc68c640 1 --2- 172.21.15.16:0/356122554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3c4102570 0x7fe3c41029b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009900 tx=0x7fe3b402a480 comp rx=0 tx=0).stop 2024-09-19T09:29:09.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.010+0000 7fe3cc68c640 1 -- 172.21.15.16:0/356122554 shutdown_connections 2024-09-19T09:29:09.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.010+0000 7fe3cc68c640 1 --2- 172.21.15.16:0/356122554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c4102ef0 0x7fe3c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.010+0000 7fe3cc68c640 1 --2- 172.21.15.16:0/356122554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3c4102570 0x7fe3c41029b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.010+0000 7fe3cc68c640 1 --2- 172.21.15.16:0/356122554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3c4100e20 0x7fe3c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.010+0000 7fe3cc68c640 1 -- 172.21.15.16:0/356122554 >> 172.21.15.16:0/356122554 conn(0x7fe3c40fc7d0 msgr2=0x7fe3c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:09.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.010+0000 7fe3cc68c640 1 -- 172.21.15.16:0/356122554 shutdown_connections 2024-09-19T09:29:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.010+0000 7fe3cc68c640 1 -- 172.21.15.16:0/356122554 wait complete. 2024-09-19T09:29:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.011+0000 7fe3cc68c640 1 Processor -- start 2024-09-19T09:29:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.011+0000 7fe3cc68c640 1 -- start start 2024-09-19T09:29:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.011+0000 7fe3cc68c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3c4100e20 0x7fe3c41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.011+0000 7fe3cc68c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3c4102570 0x7fe3c4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.011+0000 7fe3cc68c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c4102ef0 0x7fe3c419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.011+0000 7fe3cc68c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4110fa0 con 0x7fe3c4100e20 2024-09-19T09:29:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.011+0000 7fe3cc68c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4110e20 con 0x7fe3c4102ef0 2024-09-19T09:29:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.011+0000 7fe3cc68c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4111120 con 0x7fe3c4102570 2024-09-19T09:29:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.011+0000 7fe3ca428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3c4100e20 0x7fe3c41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.011+0000 7fe3ca428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3c4100e20 0x7fe3c41984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40054/0 (socket says 172.21.15.16:40054) 2024-09-19T09:29:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.011+0000 7fe3ca428640 1 -- 172.21.15.16:0/167810028 learned_addr learned my addr 172.21.15.16:0/167810028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3c9c27640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3c4102570 0x7fe3c4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3cac29640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c4102ef0 0x7fe3c419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3ca428640 1 -- 172.21.15.16:0/167810028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3c4102570 msgr2=0x7fe3c4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3ca428640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3c4102570 0x7fe3c4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3ca428640 1 -- 172.21.15.16:0/167810028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c4102ef0 msgr2=0x7fe3c419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3ca428640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c4102ef0 0x7fe3c419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3ca428640 1 -- 172.21.15.16:0/167810028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c419d480 con 0x7fe3c4100e20 2024-09-19T09:29:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3cac29640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c4102ef0 0x7fe3c419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3c9c27640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3c4102570 0x7fe3c4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3ca428640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3c4100e20 0x7fe3c41984d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000c920 tx=0x7fe3c000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3bb7fe640 1 -- 172.21.15.16:0/167810028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0007c70 con 0x7fe3c4100e20 2024-09-19T09:29:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3cc68c640 1 -- 172.21.15.16:0/167810028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c419d770 con 0x7fe3c4100e20 2024-09-19T09:29:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3bb7fe640 1 -- 172.21.15.16:0/167810028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c0007e10 con 0x7fe3c4100e20 2024-09-19T09:29:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3bb7fe640 1 -- 172.21.15.16:0/167810028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0018890 con 0x7fe3c4100e20 2024-09-19T09:29:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.012+0000 7fe3cc68c640 1 -- 172.21.15.16:0/167810028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c41a5050 con 0x7fe3c4100e20 2024-09-19T09:29:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.015+0000 7fe3cea65740 1 Processor -- start 2024-09-19T09:29:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.015+0000 7fe3cea65740 1 -- start start 2024-09-19T09:29:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.015+0000 7fe3cea65740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170380 0x56469b170760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3cea65740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 0x56469b17f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3cea65740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b17fa90 0x56469b181e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3bb7fe640 1 -- 172.21.15.16:0/167810028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe3c0018a70 con 0x7fe3c4100e20 2024-09-19T09:29:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3cea65740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56469b0d62e0 con 0x56469b170ca0 2024-09-19T09:29:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3cea65740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56469b07dce0 con 0x56469b170380 2024-09-19T09:29:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3cea65740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56469b0d22a0 con 0x56469b17fa90 2024-09-19T09:29:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3ca428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 0x56469b17f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3ca428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 0x56469b17f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40070/0 (socket says 172.21.15.16:40070) 2024-09-19T09:29:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3ca428640 1 -- 172.21.15.16:0/1609400874 learned_addr learned my addr 172.21.15.16:0/1609400874 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3cac29640 1 --2- 172.21.15.16:0/1609400874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170380 0x56469b170760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3c9c27640 1 --2- 172.21.15.16:0/1609400874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b17fa90 0x56469b181e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3ca428640 1 -- 172.21.15.16:0/1609400874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b17fa90 msgr2=0x56469b181e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3ca428640 1 --2- 172.21.15.16:0/1609400874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b17fa90 0x56469b181e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3ca428640 1 -- 172.21.15.16:0/1609400874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170380 msgr2=0x56469b170760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3ca428640 1 --2- 172.21.15.16:0/1609400874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170380 0x56469b170760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.016+0000 7fe3ca428640 1 -- 172.21.15.16:0/1609400874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56469b062440 con 0x56469b170ca0 2024-09-19T09:29:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.017+0000 7fe3ca428640 1 --2- 172.21.15.16:0/1609400874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 0x56469b17f550 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00d3c80 tx=0x7fe3c00d3d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8ac62b638742a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.017+0000 7fe3bb7fe640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe3b00a3240 0x7fe3b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.017+0000 7fe3cac29640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe3b00a3240 0x7fe3b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.017+0000 7fe3bb7fe640 1 -- 172.21.15.16:0/167810028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe3c00cfb10 con 0x7fe3c4100e20 2024-09-19T09:29:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.017+0000 7fe3b8ff9640 1 -- 172.21.15.16:0/1609400874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c009ba90 con 0x56469b170ca0 2024-09-19T09:29:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.017+0000 7fe3b8ff9640 1 -- 172.21.15.16:0/1609400874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c00e1070 con 0x56469b170ca0 2024-09-19T09:29:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.017+0000 7fe3b8ff9640 1 -- 172.21.15.16:0/1609400874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c009bc70 con 0x56469b170ca0 2024-09-19T09:29:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.018+0000 7fe3cac29640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe3b00a3240 0x7fe3b00a5700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc005fd0 tx=0x7fe3bc0098a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.018+0000 7fe3cea65740 1 -- 172.21.15.16:0/1609400874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 msgr2=0x56469b17f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.018+0000 7fe3cea65740 1 --2- 172.21.15.16:0/1609400874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 0x56469b17f550 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00d3c80 tx=0x7fe3c00d3d70 comp rx=0 tx=0).stop 2024-09-19T09:29:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.018+0000 7fe3cea65740 1 -- 172.21.15.16:0/1609400874 shutdown_connections 2024-09-19T09:29:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.018+0000 7fe3cea65740 1 --2- 172.21.15.16:0/1609400874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b17fa90 0x56469b181e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.018+0000 7fe3cea65740 1 --2- 172.21.15.16:0/1609400874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 0x56469b17f550 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.018+0000 7fe3cea65740 1 --2- 172.21.15.16:0/1609400874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170380 0x56469b170760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.018+0000 7fe3cea65740 1 -- 172.21.15.16:0/1609400874 >> 172.21.15.16:0/1609400874 conn(0x56469b16ca30 msgr2=0x56469b16ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.018+0000 7fe3cea65740 1 -- 172.21.15.16:0/1609400874 shutdown_connections 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.018+0000 7fe3cea65740 1 -- 172.21.15.16:0/1609400874 wait complete. 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3cea65740 1 -- start start 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3cea65740 1 Processor -- start 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3cea65740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170380 0x56469b17f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3cea65740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170ca0 0x56469b17c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3cea65740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b17fa90 0x56469b17cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3cea65740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56469b182660 con 0x56469b170380 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3cea65740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56469b091050 con 0x56469b170ca0 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3cea65740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56469b1827e0 con 0x56469b17fa90 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3cea65740 10 client.0 _ll_register_callbacks cb 0x56469b0d1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3ca428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170380 0x56469b17f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3ca428640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170380 0x56469b17f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40076/0 (socket says 172.21.15.16:40076) 2024-09-19T09:29:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3ca428640 1 -- 172.21.15.16:0/3019589959 learned_addr learned my addr 172.21.15.16:0/3019589959 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3c9c27640 1 --2- 172.21.15.16:0/3019589959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170ca0 0x56469b17c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3cac29640 1 --2- 172.21.15.16:0/3019589959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b17fa90 0x56469b17cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3ca428640 1 -- 172.21.15.16:0/3019589959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b17fa90 msgr2=0x56469b17cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.019+0000 7fe3ca428640 1 --2- 172.21.15.16:0/3019589959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b17fa90 0x56469b17cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.020+0000 7fe3ca428640 1 -- 172.21.15.16:0/3019589959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170ca0 msgr2=0x56469b17c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.020+0000 7fe3ca428640 1 --2- 172.21.15.16:0/3019589959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170ca0 0x56469b17c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.020+0000 7fe3ca428640 1 -- 172.21.15.16:0/3019589959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56469ae473c0 con 0x56469b170380 2024-09-19T09:29:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.020+0000 7fe3ca428640 1 --2- 172.21.15.16:0/3019589959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170380 0x56469b17f400 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00d3c50 tx=0x7fe3c0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177db58a9e3cb348 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.020+0000 7fe3adffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.020+0000 7fe3adffb640 1 -- 172.21.15.16:0/3019589959 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c009bf00 con 0x56469b170380 2024-09-19T09:29:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.020+0000 7fe3adffb640 1 -- 172.21.15.16:0/3019589959 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe3c00d31e0 con 0x56469b170380 2024-09-19T09:29:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.020+0000 7fe3cea65740 1 -- 172.21.15.16:0/3019589959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c00e1070 con 0x56469b170380 2024-09-19T09:29:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.020+0000 7fe3adffb640 1 -- 172.21.15.16:0/3019589959 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3c00983d0 con 0x56469b170380 2024-09-19T09:29:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.020+0000 7fe3cea65740 10 client.15936 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.020+0000 7fe3cea65740 1 -- 172.21.15.16:0/3019589959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56469b182660 con 0x56469b170380 2024-09-19T09:29:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.020+0000 7fe3adffb640 1 -- 172.21.15.16:0/3019589959 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c00e1070 con 0x56469b170380 2024-09-19T09:29:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.021+0000 7fe3adffb640 1 -- 172.21.15.16:0/3019589959 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe3c00a0150 con 0x56469b170380 2024-09-19T09:29:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.021+0000 7fe3cea65740 10 client.15936 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.021+0000 7fe3cea65740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.021+0000 7fe3cea65740 20 client.15936 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.021+0000 7fe3cea65740 1 --2- 172.21.15.16:0/3019589959 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56469b18f230 0x56469b191620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.021+0000 7fe3cea65740 4 client.15936 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.021+0000 7fe3cea65740 1 -- 172.21.15.16:0/3019589959 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56469b0e3850 con 0x56469b18f230 2024-09-19T09:29:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.021+0000 7fe3c9c27640 1 --2- 172.21.15.16:0/3019589959 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56469b18f230 0x56469b191620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.022+0000 7fe3c9c27640 1 --2- 172.21.15.16:0/3019589959 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56469b18f230 0x56469b191620 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9fde1a1addcf165 server_cookie=7e643fc5d59e8b20 in_seq=0 out_seq=0 2024-09-19T09:29:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.022+0000 7fe3adffb640 10 client.15936 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.022+0000 7fe3adffb640 1 -- 172.21.15.16:0/3019589959 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56469b0e3850 con 0x56469b18f230 2024-09-19T09:29:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.022+0000 7fe3adffb640 10 client.15936 handle_command_reply: tid=0 2024-09-19T09:29:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.022+0000 7fe3cea65740 1 client.15936 shutdown 2024-09-19T09:29:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.023+0000 7fe3cea65740 1 -- 172.21.15.16:0/3019589959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170380 msgr2=0x56469b17f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.023+0000 7fe3cea65740 1 --2- 172.21.15.16:0/3019589959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170380 0x56469b17f400 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00d3c50 tx=0x7fe3c0020000 comp rx=0 tx=0).stop 2024-09-19T09:29:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.023+0000 7fe3cea65740 1 -- 172.21.15.16:0/3019589959 shutdown_connections 2024-09-19T09:29:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.023+0000 7fe3cea65740 1 --2- 172.21.15.16:0/3019589959 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56469b18f230 0x56469b191620 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.023+0000 7fe3cea65740 1 --2- 172.21.15.16:0/3019589959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b17fa90 0x56469b17cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.023+0000 7fe3cea65740 1 --2- 172.21.15.16:0/3019589959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170ca0 0x56469b17c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.023+0000 7fe3adffb640 0 client.15936 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.023+0000 7fe3cea65740 1 --2- 172.21.15.16:0/3019589959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170380 0x56469b17f400 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.023+0000 7fe3cea65740 1 -- 172.21.15.16:0/3019589959 >> 172.21.15.16:0/3019589959 conn(0x56469b16ca30 msgr2=0x56469b172790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.023+0000 7fe3cea65740 1 -- 172.21.15.16:0/3019589959 shutdown_connections 2024-09-19T09:29:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.023+0000 7fe3cea65740 1 -- 172.21.15.16:0/3019589959 wait complete. 2024-09-19T09:29:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.023+0000 7fe3cea65740 20 client.15936 trim_cache size 0 max 16384 2024-09-19T09:29:09.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.041+0000 7fe3cea65740 1 Processor -- start 2024-09-19T09:29:09.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.041+0000 7fe3cea65740 1 -- start start 2024-09-19T09:29:09.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3cea65740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b170380 0x56469b18b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3cea65740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 0x56469b196c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3cea65740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b17fa90 0x56469b197160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3cea65740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ab330 con 0x56469b170ca0 2024-09-19T09:29:09.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3cea65740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ace70 con 0x56469b17fa90 2024-09-19T09:29:09.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3cea65740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41aba00 con 0x56469b170380 2024-09-19T09:29:09.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3c9c27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 0x56469b196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3c9c27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 0x56469b196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40084/0 (socket says 172.21.15.16:40084) 2024-09-19T09:29:09.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3c9c27640 1 -- 172.21.15.16:0/3292097463 learned_addr learned my addr 172.21.15.16:0/3292097463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:09.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3cac29640 1 --2- 172.21.15.16:0/3292097463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b17fa90 0x56469b197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3ca428640 1 --2- 172.21.15.16:0/3292097463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b170380 0x56469b18b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3c9c27640 1 -- 172.21.15.16:0/3292097463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b170380 msgr2=0x56469b18b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3c9c27640 1 --2- 172.21.15.16:0/3292097463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b170380 0x56469b18b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3c9c27640 1 -- 172.21.15.16:0/3292097463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b17fa90 msgr2=0x56469b197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3c9c27640 1 --2- 172.21.15.16:0/3292097463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b17fa90 0x56469b197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3c9c27640 1 -- 172.21.15.16:0/3292097463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56469b17ef70 con 0x56469b170ca0 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3cac29640 1 --2- 172.21.15.16:0/3292097463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b17fa90 0x56469b197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.042+0000 7fe3c9c27640 1 --2- 172.21.15.16:0/3292097463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 0x56469b196c20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4000c00 tx=0x7fe3b4005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932ada65f827b5fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.043+0000 7fe3affff640 1 -- 172.21.15.16:0/3292097463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4035040 con 0x56469b170ca0 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.043+0000 7fe3affff640 1 -- 172.21.15.16:0/3292097463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3b404a070 con 0x56469b170ca0 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.043+0000 7fe3affff640 1 -- 172.21.15.16:0/3292097463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4003850 con 0x56469b170ca0 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.043+0000 7fe3cea65740 1 -- 172.21.15.16:0/3292097463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 msgr2=0x56469b196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.043+0000 7fe3cea65740 1 --2- 172.21.15.16:0/3292097463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 0x56469b196c20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4000c00 tx=0x7fe3b4005b70 comp rx=0 tx=0).stop 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 -- 172.21.15.16:0/3292097463 shutdown_connections 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 --2- 172.21.15.16:0/3292097463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b17fa90 0x56469b197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 --2- 172.21.15.16:0/3292097463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b170ca0 0x56469b196c20 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 --2- 172.21.15.16:0/3292097463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b170380 0x56469b18b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 -- 172.21.15.16:0/3292097463 >> 172.21.15.16:0/3292097463 conn(0x56469b16ca30 msgr2=0x56469b177d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 -- 172.21.15.16:0/3292097463 shutdown_connections 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 -- 172.21.15.16:0/3292097463 wait complete. 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 -- start start 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 Processor -- start 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b170380 0x56469b183f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170ca0 0x56469b1844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b17fa90 0x56469b1fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ac7a0 con 0x56469b17fa90 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41ac0d0 con 0x56469b170ca0 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.044+0000 7fe3cea65740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41a92f0 con 0x56469b170380 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.045+0000 7fe3cea65740 10 client.0 _ll_register_callbacks cb 0x7fe3bc009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.045+0000 7fe3c9c27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170ca0 0x56469b1844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.045+0000 7fe3ca428640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b170380 0x56469b183f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.045+0000 7fe3cac29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b17fa90 0x56469b1fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.045+0000 7fe3ca428640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b170380 0x56469b183f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:45884/0 (socket says 172.21.15.16:45884) 2024-09-19T09:29:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.045+0000 7fe3ca428640 1 -- 172.21.15.16:0/2539880247 learned_addr learned my addr 172.21.15.16:0/2539880247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3cac29640 1 -- 172.21.15.16:0/2539880247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b170380 msgr2=0x56469b183f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3cac29640 1 --2- 172.21.15.16:0/2539880247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b170380 0x56469b183f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3cac29640 1 -- 172.21.15.16:0/2539880247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170ca0 msgr2=0x56469b1844c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3cac29640 1 --2- 172.21.15.16:0/2539880247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170ca0 0x56469b1844c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3cac29640 1 -- 172.21.15.16:0/2539880247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56469b1771b0 con 0x56469b17fa90 2024-09-19T09:29:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3ca428640 1 --2- 172.21.15.16:0/2539880247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b170380 0x56469b183f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3cac29640 1 --2- 172.21.15.16:0/2539880247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b17fa90 0x56469b1fd3e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x56469b18e030 tx=0x7fe3bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b887447a8f48dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3adffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3adffb640 1 -- 172.21.15.16:0/2539880247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc004b20 con 0x56469b17fa90 2024-09-19T09:29:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3adffb640 1 -- 172.21.15.16:0/2539880247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe3bc004cc0 con 0x56469b17fa90 2024-09-19T09:29:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3cea65740 1 -- 172.21.15.16:0/2539880247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3b404a070 con 0x56469b17fa90 2024-09-19T09:29:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3adffb640 1 -- 172.21.15.16:0/2539880247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3bc0207e0 con 0x56469b17fa90 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3cea65740 10 client.15942 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.046+0000 7fe3cea65740 1 -- 172.21.15.16:0/2539880247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56469b1f4e30 con 0x56469b17fa90 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.047+0000 7fe3adffb640 1 -- 172.21.15.16:0/2539880247 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3b404a070 con 0x56469b17fa90 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.047+0000 7fe3adffb640 1 -- 172.21.15.16:0/2539880247 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe3bc029ca0 con 0x56469b17fa90 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.047+0000 7fe3cea65740 10 client.15942 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.047+0000 7fe3cea65740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.047+0000 7fe3cea65740 20 client.15942 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.047+0000 7fe3cea65740 1 --2- 172.21.15.16:0/2539880247 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56469b1f5600 0x56469b1f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.047+0000 7fe3cea65740 4 client.15942 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.047+0000 7fe3cea65740 1 -- 172.21.15.16:0/2539880247 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56469b0ddd60 con 0x56469b1f5600 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.047+0000 7fe3ca428640 1 --2- 172.21.15.16:0/2539880247 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56469b1f5600 0x56469b1f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3ca428640 1 --2- 172.21.15.16:0/2539880247 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56469b1f5600 0x56469b1f7a40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f211fa8d2c90036 server_cookie=a5a8d4c6c7b1cbfe in_seq=0 out_seq=0 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3adffb640 10 client.15942 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3adffb640 1 -- 172.21.15.16:0/2539880247 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56469b0ddd60 con 0x56469b1f5600 2024-09-19T09:29:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3adffb640 10 client.15942 handle_command_reply: tid=0 2024-09-19T09:29:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3cea65740 1 client.15942 shutdown 2024-09-19T09:29:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3cea65740 1 -- 172.21.15.16:0/2539880247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b17fa90 msgr2=0x56469b1fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3cea65740 1 --2- 172.21.15.16:0/2539880247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b17fa90 0x56469b1fd3e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x56469b18e030 tx=0x7fe3bc004a40 comp rx=0 tx=0).stop 2024-09-19T09:29:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3cea65740 1 -- 172.21.15.16:0/2539880247 shutdown_connections 2024-09-19T09:29:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3cea65740 1 --2- 172.21.15.16:0/2539880247 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56469b1f5600 0x56469b1f7a40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3cea65740 1 --2- 172.21.15.16:0/2539880247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469b17fa90 0x56469b1fd3e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3cea65740 1 --2- 172.21.15.16:0/2539880247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56469b170ca0 0x56469b1844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3cea65740 1 --2- 172.21.15.16:0/2539880247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56469b170380 0x56469b183f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3cea65740 1 -- 172.21.15.16:0/2539880247 >> 172.21.15.16:0/2539880247 conn(0x56469b16ca30 msgr2=0x56469b16b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3adffb640 0 client.15942 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3cea65740 1 -- 172.21.15.16:0/2539880247 shutdown_connections 2024-09-19T09:29:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3cea65740 1 -- 172.21.15.16:0/2539880247 wait complete. 2024-09-19T09:29:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.048+0000 7fe3cea65740 20 client.15942 trim_cache size 0 max 16384 2024-09-19T09:29:09.056 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:29:09.057 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T09:29:09.057 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-19T09:29:09.057 INFO:teuthology.orchestra.run.smithi016.stdout: "f34a5b49-930f-46bd-8468-f703eea30ad5": { 2024-09-19T09:29:09.057 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-19T09:29:09.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "f34a5b49-930f-46bd-8468-f703eea30ad5", 2024-09-19T09:29:09.057 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-19T09:29:09.057 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:09.057 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:09.057 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:29:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.049+0000 7fe3cc68c640 1 -- 172.21.15.16:0/167810028 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe3b00a3240 msgr2=0x7fe3b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.049+0000 7fe3cc68c640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe3b00a3240 0x7fe3b00a5700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc005fd0 tx=0x7fe3bc0098a0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.049+0000 7fe3cc68c640 1 -- 172.21.15.16:0/167810028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3c4100e20 msgr2=0x7fe3c41984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.049+0000 7fe3cc68c640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3c4100e20 0x7fe3c41984d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000c920 tx=0x7fe3c000cde0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.049+0000 7fe3cc68c640 1 -- 172.21.15.16:0/167810028 shutdown_connections 2024-09-19T09:29:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.049+0000 7fe3cc68c640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe3b00a3240 0x7fe3b00a5700 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.049+0000 7fe3cc68c640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3c4102570 0x7fe3c4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.049+0000 7fe3cc68c640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3c4102ef0 0x7fe3c419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.049+0000 7fe3cc68c640 1 --2- 172.21.15.16:0/167810028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3c4100e20 0x7fe3c41984d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.049+0000 7fe3cc68c640 1 -- 172.21.15.16:0/167810028 >> 172.21.15.16:0/167810028 conn(0x7fe3c40fc7d0 msgr2=0x7fe3c406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.049+0000 7fe3cc68c640 1 -- 172.21.15.16:0/167810028 shutdown_connections 2024-09-19T09:29:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:09.049+0000 7fe3cc68c640 1 -- 172.21.15.16:0/167810028 wait complete. 2024-09-19T09:29:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:09.348 INFO:tasks.workunit.client.0.smithi016.stdout:test read from hole 2024-09-19T09:29:09.348 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'test read from hole' 2024-09-19T09:29:09.349 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-09-19T09:29:09.350 INFO:tasks.workunit.client.0.smithi016.stderr:0+0 records in 2024-09-19T09:29:09.350 INFO:tasks.workunit.client.0.smithi016.stderr:0+0 records out 2024-09-19T09:29:09.350 INFO:tasks.workunit.client.0.smithi016.stderr:0 bytes copied, 5.3119e-05 s, 0.0 kB/s 2024-09-19T09:29:09.350 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-09-19T09:29:09.352 INFO:tasks.workunit.client.0.smithi016.stderr:2+0 records in 2024-09-19T09:29:09.352 INFO:tasks.workunit.client.0.smithi016.stderr:2+0 records out 2024-09-19T09:29:09.352 INFO:tasks.workunit.client.0.smithi016.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00123477 s, 829 kB/s 2024-09-19T09:29:09.352 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-09-19T09:29:09.353 INFO:tasks.workunit.client.0.smithi016.stderr:2+0 records in 2024-09-19T09:29:09.353 INFO:tasks.workunit.client.0.smithi016.stderr:2+0 records out 2024-09-19T09:29:09.353 INFO:tasks.workunit.client.0.smithi016.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 4.415e-05 s, 23.2 MB/s 2024-09-19T09:29:09.353 INFO:tasks.workunit.client.0.smithi016.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-09-19T09:29:09.354 INFO:tasks.workunit.client.0.smithi016.stdout:other thing 2024-09-19T09:29:09.354 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'other thing' 2024-09-19T09:29:09.354 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-09-19T09:29:09.354 INFO:tasks.workunit.client.0.smithi016.stderr:1+0 records in 2024-09-19T09:29:09.354 INFO:tasks.workunit.client.0.smithi016.stderr:1+0 records out 2024-09-19T09:29:09.355 INFO:tasks.workunit.client.0.smithi016.stderr:500 bytes copied, 4.2311e-05 s, 11.8 MB/s 2024-09-19T09:29:09.355 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-09-19T09:29:09.361 INFO:tasks.workunit.client.0.smithi016.stderr:0+1 records in 2024-09-19T09:29:09.361 INFO:tasks.workunit.client.0.smithi016.stderr:0+1 records out 2024-09-19T09:29:09.362 INFO:tasks.workunit.client.0.smithi016.stderr:500 bytes copied, 0.00507135 s, 98.6 kB/s 2024-09-19T09:29:09.362 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-09-19T09:29:09.366 INFO:tasks.workunit.client.0.smithi016.stderr:0+1 records in 2024-09-19T09:29:09.366 INFO:tasks.workunit.client.0.smithi016.stderr:0+1 records out 2024-09-19T09:29:09.366 INFO:tasks.workunit.client.0.smithi016.stderr:500 bytes copied, 0.00344487 s, 145 kB/s 2024-09-19T09:29:09.366 INFO:tasks.workunit.client.0.smithi016.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-09-19T09:29:09.366 INFO:tasks.workunit.client.0.smithi016.stdout:OK 2024-09-19T09:29:09.367 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo OK 2024-09-19T09:29:09.367 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T09:29:09.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T09:29:09.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:09 smithi097 ceph-mon[34864]: pgmap v384: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-09-19T09:29:09.390 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:09 smithi016 ceph-mon[30979]: pgmap v384: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-09-19T09:29:09.408 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-09-19T09:29:09.409 DEBUG:teuthology.orchestra.run.smithi016:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-09-19T09:29:09.479 INFO:tasks.workunit.client.0.smithi016.stdout:writing first 3 bytes of 10k file 2024-09-19T09:29:09.479 INFO:tasks.workunit.client.0.smithi016.stdout:reading O_DIRECT 2024-09-19T09:29:09.480 INFO:tasks.workunit.client.0.smithi016.stdout:got 10000 2024-09-19T09:29:09.480 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T09:29:09.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T09:29:09.560 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-09-19T09:29:09.561 DEBUG:teuthology.orchestra.run.smithi016:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-09-19T09:29:09.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:09 smithi142 ceph-mon[35032]: pgmap v384: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-09-19T09:29:09.967 INFO:tasks.cephfs.filesystem:scrub status for tag:f34a5b49-930f-46bd-8468-f703eea30ad5 - {'path': '/', 'tag': 'f34a5b49-930f-46bd-8468-f703eea30ad5', 'options': 'recursive,force'} 2024-09-19T09:29:10.198 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:10 smithi016 ceph-mon[30979]: scrub summary: idle 2024-09-19T09:29:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.202+0000 7f4defa76640 1 -- 172.21.15.16:0/1925111473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de8101de0 msgr2=0x7f4de81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.202+0000 7f4defa76640 1 --2- 172.21.15.16:0/1925111473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de8101de0 0x7f4de81021c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009900 tx=0x7f4de402a410 comp rx=0 tx=0).stop 2024-09-19T09:29:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.202+0000 7f4defa76640 1 -- 172.21.15.16:0/1925111473 shutdown_connections 2024-09-19T09:29:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.202+0000 7f4defa76640 1 --2- 172.21.15.16:0/1925111473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4de8106f10 0x7f4de810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.202+0000 7f4defa76640 1 --2- 172.21.15.16:0/1925111473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4de8102790 0x7f4de81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.202+0000 7f4defa76640 1 --2- 172.21.15.16:0/1925111473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de8101de0 0x7f4de81021c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.202+0000 7f4defa76640 1 -- 172.21.15.16:0/1925111473 >> 172.21.15.16:0/1925111473 conn(0x7f4de80fd8c0 msgr2=0x7f4de80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.202+0000 7f4defa76640 1 -- 172.21.15.16:0/1925111473 shutdown_connections 2024-09-19T09:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.202+0000 7f4defa76640 1 -- 172.21.15.16:0/1925111473 wait complete. 2024-09-19T09:29:10.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.203+0000 7f4defa76640 1 Processor -- start 2024-09-19T09:29:10.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.203+0000 7f4defa76640 1 -- start start 2024-09-19T09:29:10.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.203+0000 7f4defa76640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4de8101de0 0x7f4de8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.203+0000 7f4defa76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de8102790 0x7f4de8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.203+0000 7f4defa76640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4de8106f10 0x7f4de819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.203+0000 7f4defa76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8111fe0 con 0x7f4de8102790 2024-09-19T09:29:10.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.203+0000 7f4defa76640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8111e60 con 0x7f4de8106f10 2024-09-19T09:29:10.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.203+0000 7f4defa76640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8112160 con 0x7f4de8101de0 2024-09-19T09:29:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.203+0000 7f4ded011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de8102790 0x7f4de8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.203+0000 7f4ded011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de8102790 0x7f4de8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40112/0 (socket says 172.21.15.16:40112) 2024-09-19T09:29:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.203+0000 7f4ded011640 1 -- 172.21.15.16:0/36662086 learned_addr learned my addr 172.21.15.16:0/36662086 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.203+0000 7f4dee013640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4de8106f10 0x7f4de819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.204+0000 7f4ded812640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4de8101de0 0x7f4de8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.204+0000 7f4ded011640 1 -- 172.21.15.16:0/36662086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4de8101de0 msgr2=0x7f4de8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.204+0000 7f4ded011640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4de8101de0 0x7f4de8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.204+0000 7f4ded011640 1 -- 172.21.15.16:0/36662086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4de8106f10 msgr2=0x7f4de819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.204+0000 7f4ded011640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4de8106f10 0x7f4de819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.204+0000 7f4ded011640 1 -- 172.21.15.16:0/36662086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de819d3f0 con 0x7f4de8102790 2024-09-19T09:29:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.204+0000 7f4ded812640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4de8101de0 0x7f4de8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.204+0000 7f4ded011640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de8102790 0x7f4de8198980 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800d950 tx=0x7f4dd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.204+0000 7f4ddeffd640 1 -- 172.21.15.16:0/36662086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8017070 con 0x7f4de8102790 2024-09-19T09:29:10.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.204+0000 7f4ddeffd640 1 -- 172.21.15.16:0/36662086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd8009c30 con 0x7f4de8102790 2024-09-19T09:29:10.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.204+0000 7f4defa76640 1 -- 172.21.15.16:0/36662086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de819d6e0 con 0x7f4de8102790 2024-09-19T09:29:10.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.204+0000 7f4ddeffd640 1 -- 172.21.15.16:0/36662086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800f6b0 con 0x7f4de8102790 2024-09-19T09:29:10.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.204+0000 7f4defa76640 1 -- 172.21.15.16:0/36662086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de81a4fc0 con 0x7f4de8102790 2024-09-19T09:29:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.207+0000 7f4ddeffd640 1 -- 172.21.15.16:0/36662086 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4dd80109d0 con 0x7f4de8102790 2024-09-19T09:29:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.207+0000 7f4df1e4f740 1 Processor -- start 2024-09-19T09:29:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4df1e4f740 1 -- start start 2024-09-19T09:29:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4df1e4f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf5750 0x5644cbdf5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4df1e4f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf6070 0x5644cbe04900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4df1e4f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbe04e40 0x5644cbe07230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4df1e4f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5644cbd5a160 con 0x5644cbdf5750 2024-09-19T09:29:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4df1e4f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5644cbd01720 con 0x5644cbe04e40 2024-09-19T09:29:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4df1e4f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5644cbd56120 con 0x5644cbdf6070 2024-09-19T09:29:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4dee013640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf5750 0x5644cbdf5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4dee013640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf5750 0x5644cbdf5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40124/0 (socket says 172.21.15.16:40124) 2024-09-19T09:29:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4dee013640 1 -- 172.21.15.16:0/1713695751 learned_addr learned my addr 172.21.15.16:0/1713695751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4ded011640 1 --2- 172.21.15.16:0/1713695751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbe04e40 0x5644cbe07230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4ded812640 1 --2- 172.21.15.16:0/1713695751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf6070 0x5644cbe04900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4ddeffd640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4dd40a3240 0x7f4dd40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4dee013640 1 -- 172.21.15.16:0/1713695751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf6070 msgr2=0x5644cbe04900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4dee013640 1 --2- 172.21.15.16:0/1713695751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf6070 0x5644cbe04900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4dee013640 1 -- 172.21.15.16:0/1713695751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbe04e40 msgr2=0x5644cbe07230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4dee013640 1 --2- 172.21.15.16:0/1713695751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbe04e40 0x5644cbe07230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4dee013640 1 -- 172.21.15.16:0/1713695751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644cbacbba0 con 0x5644cbdf5750 2024-09-19T09:29:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.208+0000 7f4dee013640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4dd40a3240 0x7f4dd40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.209+0000 7f4ddeffd640 1 -- 172.21.15.16:0/36662086 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4dd80cf5a0 con 0x7f4de8102790 2024-09-19T09:29:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.209+0000 7f4dee013640 1 --2- 172.21.15.16:0/1713695751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf5750 0x5644cbdf5b30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4de000a5d0 tx=0x7f4de000aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdeef930b0d93bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.209+0000 7f4dd3fff640 1 -- 172.21.15.16:0/1713695751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de001a040 con 0x5644cbdf5750 2024-09-19T09:29:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.209+0000 7f4dd3fff640 1 -- 172.21.15.16:0/1713695751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de0021ba0 con 0x5644cbdf5750 2024-09-19T09:29:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.209+0000 7f4dd3fff640 1 -- 172.21.15.16:0/1713695751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de001f350 con 0x5644cbdf5750 2024-09-19T09:29:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.209+0000 7f4dee013640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4dd40a3240 0x7f4dd40a5700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4de0020000 tx=0x7f4de0021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.210+0000 7f4df1e4f740 1 -- 172.21.15.16:0/1713695751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf5750 msgr2=0x5644cbdf5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.210+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/1713695751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf5750 0x5644cbdf5b30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4de000a5d0 tx=0x7f4de000aa90 comp rx=0 tx=0).stop 2024-09-19T09:29:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.210+0000 7f4df1e4f740 1 -- 172.21.15.16:0/1713695751 shutdown_connections 2024-09-19T09:29:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.210+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/1713695751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf6070 0x5644cbe04900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.210+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/1713695751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbe04e40 0x5644cbe07230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.210+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/1713695751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf5750 0x5644cbdf5b30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.210+0000 7f4df1e4f740 1 -- 172.21.15.16:0/1713695751 >> 172.21.15.16:0/1713695751 conn(0x5644cbdf1de0 msgr2=0x5644cbdf21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.210+0000 7f4df1e4f740 1 -- 172.21.15.16:0/1713695751 shutdown_connections 2024-09-19T09:29:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.210+0000 7f4df1e4f740 1 -- 172.21.15.16:0/1713695751 wait complete. 2024-09-19T09:29:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.210+0000 7f4df1e4f740 1 -- start start 2024-09-19T09:29:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.210+0000 7f4df1e4f740 1 Processor -- start 2024-09-19T09:29:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4df1e4f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf5750 0x5644cbe01d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4df1e4f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 0x5644cbe02290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4df1e4f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbe04e40 0x5644cbe027d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4df1e4f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5644cbe07a70 con 0x5644cbdf6070 2024-09-19T09:29:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4df1e4f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5644cbce2900 con 0x5644cbe04e40 2024-09-19T09:29:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4df1e4f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5644cbe07bf0 con 0x5644cbdf5750 2024-09-19T09:29:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4df1e4f740 10 client.0 _ll_register_callbacks cb 0x5644cbce0780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4ded011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 0x5644cbe02290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4ded812640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf5750 0x5644cbe01d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4ded011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 0x5644cbe02290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40132/0 (socket says 172.21.15.16:40132) 2024-09-19T09:29:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4ded011640 1 -- 172.21.15.16:0/3365974011 learned_addr learned my addr 172.21.15.16:0/3365974011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4dee013640 1 --2- 172.21.15.16:0/3365974011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbe04e40 0x5644cbe027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4ded011640 1 -- 172.21.15.16:0/3365974011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf5750 msgr2=0x5644cbe01d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4ded011640 1 --2- 172.21.15.16:0/3365974011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf5750 0x5644cbe01d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4ded011640 1 -- 172.21.15.16:0/3365974011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbe04e40 msgr2=0x5644cbe027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4ded011640 1 --2- 172.21.15.16:0/3365974011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbe04e40 0x5644cbe027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4ded011640 1 -- 172.21.15.16:0/3365974011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644cbd22c50 con 0x5644cbdf6070 2024-09-19T09:29:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4dee013640 1 --2- 172.21.15.16:0/3365974011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbe04e40 0x5644cbe027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4ded011640 1 --2- 172.21.15.16:0/3365974011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 0x5644cbe02290 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4dd809bb50 tx=0x7f4dd8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afbbf4f1fad6600 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.211+0000 7f4dd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4dd17fa640 1 -- 172.21.15.16:0/3365974011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8020070 con 0x5644cbdf6070 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4dd17fa640 1 -- 172.21.15.16:0/3365974011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4dd80d2c10 con 0x5644cbdf6070 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4df1e4f740 1 -- 172.21.15.16:0/3365974011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4de0021ba0 con 0x5644cbdf6070 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4dd17fa640 1 -- 172.21.15.16:0/3365974011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4dd809c2c0 con 0x5644cbdf6070 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4df1e4f740 10 client.15954 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4df1e4f740 1 -- 172.21.15.16:0/3365974011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5644cbe13790 con 0x5644cbdf6070 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4dd17fa640 1 -- 172.21.15.16:0/3365974011 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de0021ba0 con 0x5644cbdf6070 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4dd17fa640 1 -- 172.21.15.16:0/3365974011 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4dd8086140 con 0x5644cbdf6070 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4df1e4f740 10 client.15954 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4df1e4f740 10 client.15954 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4df1e4f740 20 client.15954 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:10.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/3365974011 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5644cbe14c50 0x5644cbe17040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4df1e4f740 4 client.15954 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4df1e4f740 1 -- 172.21.15.16:0/3365974011 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644cbd67890 con 0x5644cbe14c50 2024-09-19T09:29:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.212+0000 7f4ded812640 1 --2- 172.21.15.16:0/3365974011 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5644cbe14c50 0x5644cbe17040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.213+0000 7f4dd17fa640 10 client.15954 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.213+0000 7f4ded812640 1 --2- 172.21.15.16:0/3365974011 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5644cbe14c50 0x5644cbe17040 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a315a8d73b1937 server_cookie=650244505b9e5efa in_seq=0 out_seq=0 2024-09-19T09:29:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4dd17fa640 1 -- 172.21.15.16:0/3365974011 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644cbd67890 con 0x5644cbe14c50 2024-09-19T09:29:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4dd17fa640 10 client.15954 handle_command_reply: tid=0 2024-09-19T09:29:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4df1e4f740 1 client.15954 shutdown 2024-09-19T09:29:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4df1e4f740 1 -- 172.21.15.16:0/3365974011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 msgr2=0x5644cbe02290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/3365974011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 0x5644cbe02290 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4dd809bb50 tx=0x7f4dd8005880 comp rx=0 tx=0).stop 2024-09-19T09:29:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4df1e4f740 1 -- 172.21.15.16:0/3365974011 shutdown_connections 2024-09-19T09:29:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/3365974011 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5644cbe14c50 0x5644cbe17040 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/3365974011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbe04e40 0x5644cbe027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/3365974011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 0x5644cbe02290 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/3365974011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf5750 0x5644cbe01d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4df1e4f740 1 -- 172.21.15.16:0/3365974011 >> 172.21.15.16:0/3365974011 conn(0x5644cbdf1de0 msgr2=0x5644cbdf7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4dd17fa640 0 client.15954 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4df1e4f740 1 -- 172.21.15.16:0/3365974011 shutdown_connections 2024-09-19T09:29:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4df1e4f740 1 -- 172.21.15.16:0/3365974011 wait complete. 2024-09-19T09:29:10.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.214+0000 7f4df1e4f740 20 client.15954 trim_cache size 0 max 16384 2024-09-19T09:29:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4df1e4f740 1 Processor -- start 2024-09-19T09:29:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4df1e4f740 1 -- start start 2024-09-19T09:29:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4df1e4f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbdf5750 0x5644cbdf7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4df1e4f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 0x5644cbe1c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4df1e4f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbe04e40 0x5644cbe1ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4df1e4f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81ab420 con 0x5644cbdf6070 2024-09-19T09:29:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4df1e4f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81acf60 con 0x5644cbdf5750 2024-09-19T09:29:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4df1e4f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81abaf0 con 0x5644cbe04e40 2024-09-19T09:29:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4ded011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 0x5644cbe1c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4ded011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 0x5644cbe1c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40146/0 (socket says 172.21.15.16:40146) 2024-09-19T09:29:10.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4ded011640 1 -- 172.21.15.16:0/1750086108 learned_addr learned my addr 172.21.15.16:0/1750086108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4dee013640 1 --2- 172.21.15.16:0/1750086108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbe04e40 0x5644cbe1ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4ded812640 1 --2- 172.21.15.16:0/1750086108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbdf5750 0x5644cbdf7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4ded011640 1 -- 172.21.15.16:0/1750086108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbe04e40 msgr2=0x5644cbe1ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4ded011640 1 --2- 172.21.15.16:0/1750086108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbe04e40 0x5644cbe1ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4ded011640 1 -- 172.21.15.16:0/1750086108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbdf5750 msgr2=0x5644cbdf7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4ded011640 1 --2- 172.21.15.16:0/1750086108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbdf5750 0x5644cbdf7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4ded011640 1 -- 172.21.15.16:0/1750086108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644cbe176a0 con 0x5644cbdf6070 2024-09-19T09:29:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.231+0000 7f4ded812640 1 --2- 172.21.15.16:0/1750086108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbdf5750 0x5644cbdf7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.232+0000 7f4ded011640 1 --2- 172.21.15.16:0/1750086108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 0x5644cbe1c8d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80a4300 tx=0x7f4dd8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77b855f1ef7ea0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.232+0000 7f4dd37fe640 1 -- 172.21.15.16:0/1750086108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd80b8070 con 0x5644cbdf6070 2024-09-19T09:29:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.232+0000 7f4dd37fe640 1 -- 172.21.15.16:0/1750086108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd809bbc0 con 0x5644cbdf6070 2024-09-19T09:29:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.232+0000 7f4dd37fe640 1 -- 172.21.15.16:0/1750086108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8020030 con 0x5644cbdf6070 2024-09-19T09:29:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.232+0000 7f4df1e4f740 1 -- 172.21.15.16:0/1750086108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 msgr2=0x5644cbe1c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.232+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/1750086108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 0x5644cbe1c8d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80a4300 tx=0x7f4dd8005bd0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.232+0000 7f4df1e4f740 1 -- 172.21.15.16:0/1750086108 shutdown_connections 2024-09-19T09:29:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.232+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/1750086108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbe04e40 0x5644cbe1ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.232+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/1750086108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbdf6070 0x5644cbe1c8d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.232+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/1750086108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbdf5750 0x5644cbdf7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.232+0000 7f4df1e4f740 1 -- 172.21.15.16:0/1750086108 >> 172.21.15.16:0/1750086108 conn(0x5644cbdf1de0 msgr2=0x5644cbdf7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.232+0000 7f4df1e4f740 1 -- 172.21.15.16:0/1750086108 shutdown_connections 2024-09-19T09:29:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.232+0000 7f4df1e4f740 1 -- 172.21.15.16:0/1750086108 wait complete. 2024-09-19T09:29:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.233+0000 7f4df1e4f740 1 -- start start 2024-09-19T09:29:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.233+0000 7f4df1e4f740 1 Processor -- start 2024-09-19T09:29:10.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.233+0000 7f4df1e4f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbdf5750 0x5644cbe09510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.233+0000 7f4df1e4f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf6070 0x5644cbe09a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.233+0000 7f4df1e4f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbe04e40 0x5644cbe78870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.233+0000 7f4df1e4f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81ac890 con 0x5644cbe04e40 2024-09-19T09:29:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.233+0000 7f4df1e4f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81ac1c0 con 0x5644cbdf5750 2024-09-19T09:29:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.233+0000 7f4df1e4f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81a93e0 con 0x5644cbdf6070 2024-09-19T09:29:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.233+0000 7f4df1e4f740 10 client.0 _ll_register_callbacks cb 0x7f4dd809ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.233+0000 7f4dee013640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbe04e40 0x5644cbe78870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.233+0000 7f4dee013640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbe04e40 0x5644cbe78870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40150/0 (socket says 172.21.15.16:40150) 2024-09-19T09:29:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.233+0000 7f4dee013640 1 -- 172.21.15.16:0/4280323772 learned_addr learned my addr 172.21.15.16:0/4280323772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.233+0000 7f4ded011640 1 --2- 172.21.15.16:0/4280323772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf6070 0x5644cbe09a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4ded812640 1 --2- 172.21.15.16:0/4280323772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbdf5750 0x5644cbe09510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4dee013640 1 -- 172.21.15.16:0/4280323772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf6070 msgr2=0x5644cbe09a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4dee013640 1 --2- 172.21.15.16:0/4280323772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf6070 0x5644cbe09a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4dee013640 1 -- 172.21.15.16:0/4280323772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbdf5750 msgr2=0x5644cbe09510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4dee013640 1 --2- 172.21.15.16:0/4280323772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbdf5750 0x5644cbe09510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4dee013640 1 -- 172.21.15.16:0/4280323772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644cbdfcad0 con 0x5644cbe04e40 2024-09-19T09:29:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4dee013640 1 --2- 172.21.15.16:0/4280323772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbe04e40 0x5644cbe78870 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4de00098b0 tx=0x7f4de0009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a760b4b6de705921 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4dd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4dd17fa640 1 -- 172.21.15.16:0/4280323772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de001a040 con 0x5644cbe04e40 2024-09-19T09:29:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4dd17fa640 1 -- 172.21.15.16:0/4280323772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4de0021b20 con 0x5644cbe04e40 2024-09-19T09:29:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4df1e4f740 1 -- 172.21.15.16:0/4280323772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dd809bbc0 con 0x5644cbe04e40 2024-09-19T09:29:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4dd17fa640 1 -- 172.21.15.16:0/4280323772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4de0016510 con 0x5644cbe04e40 2024-09-19T09:29:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4df1e4f740 10 client.15960 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.234+0000 7f4df1e4f740 1 -- 172.21.15.16:0/4280323772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5644cbded3f0 con 0x5644cbe04e40 2024-09-19T09:29:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.235+0000 7f4dd17fa640 1 -- 172.21.15.16:0/4280323772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dd809bbc0 con 0x5644cbe04e40 2024-09-19T09:29:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.235+0000 7f4dd17fa640 1 -- 172.21.15.16:0/4280323772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4de0028070 con 0x5644cbe04e40 2024-09-19T09:29:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.235+0000 7f4df1e4f740 10 client.15960 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.235+0000 7f4df1e4f740 10 client.15960 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:29:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.235+0000 7f4df1e4f740 20 client.15960 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.235+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/4280323772 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5644cbe804e0 0x5644cbe82900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.235+0000 7f4df1e4f740 4 client.15960 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T09:29:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.235+0000 7f4df1e4f740 1 -- 172.21.15.16:0/4280323772 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x5644cbd61be0 con 0x5644cbe804e0 2024-09-19T09:29:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.235+0000 7f4ded812640 1 --2- 172.21.15.16:0/4280323772 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5644cbe804e0 0x5644cbe82900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.236+0000 7f4ded812640 1 --2- 172.21.15.16:0/4280323772 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5644cbe804e0 0x5644cbe82900 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37f6a537e3d826e4 server_cookie=f81f7cc955f8ff65 in_seq=0 out_seq=0 2024-09-19T09:29:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:10.236+0000 7f4dd17fa640 10 client.15960 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:10.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:10 smithi142 ceph-mon[35032]: scrub summary: idle 2024-09-19T09:29:10.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:10 smithi097 ceph-mon[34864]: scrub summary: idle 2024-09-19T09:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.366+0000 7fbeceaba640 1 -- 172.21.15.16:0/1033178363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec81025f0 msgr2=0x7fbec8108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.366+0000 7fbeceaba640 1 --2- 172.21.15.16:0/1033178363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec81025f0 0x7fbec8108e80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009ca0 tx=0x7fbec002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.366+0000 7fbeceaba640 1 -- 172.21.15.16:0/1033178363 shutdown_connections 2024-09-19T09:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.366+0000 7fbeceaba640 1 --2- 172.21.15.16:0/1033178363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8101c30 0x7fbec81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.366+0000 7fbeceaba640 1 --2- 172.21.15.16:0/1033178363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec81025f0 0x7fbec8108e80 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.366+0000 7fbeceaba640 1 --2- 172.21.15.16:0/1033178363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8100a40 0x7fbec8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.367+0000 7fbeceaba640 1 -- 172.21.15.16:0/1033178363 >> 172.21.15.16:0/1033178363 conn(0x7fbec80fc1e0 msgr2=0x7fbec80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.367+0000 7fbeceaba640 1 -- 172.21.15.16:0/1033178363 shutdown_connections 2024-09-19T09:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.367+0000 7fbeceaba640 1 -- 172.21.15.16:0/1033178363 wait complete. 2024-09-19T09:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.367+0000 7fbeceaba640 1 Processor -- start 2024-09-19T09:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.367+0000 7fbeceaba640 1 -- start start 2024-09-19T09:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.367+0000 7fbeceaba640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8100a40 0x7fbec8197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.367+0000 7fbeceaba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8101c30 0x7fbec8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.367+0000 7fbeceaba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec81025f0 0x7fbec819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.367+0000 7fbeceaba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbec810c6c0 con 0x7fbec81025f0 2024-09-19T09:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.367+0000 7fbeceaba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbec810c540 con 0x7fbec8101c30 2024-09-19T09:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.367+0000 7fbeceaba640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbec810c840 con 0x7fbec8100a40 2024-09-19T09:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbecd057640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec81025f0 0x7fbec819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbecd057640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec81025f0 0x7fbec819f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40170/0 (socket says 172.21.15.16:40170) 2024-09-19T09:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbecd057640 1 -- 172.21.15.16:0/2799242313 learned_addr learned my addr 172.21.15.16:0/2799242313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbec7fff640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8101c30 0x7fbec8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbecc856640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8100a40 0x7fbec8197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbecd057640 1 -- 172.21.15.16:0/2799242313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8100a40 msgr2=0x7fbec8197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbecd057640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8100a40 0x7fbec8197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbecd057640 1 -- 172.21.15.16:0/2799242313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8101c30 msgr2=0x7fbec8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbecd057640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8101c30 0x7fbec8198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbecd057640 1 -- 172.21.15.16:0/2799242313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec819f9c0 con 0x7fbec81025f0 2024-09-19T09:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbecd057640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec81025f0 0x7fbec819f350 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbec0031ae0 tx=0x7fbec0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbec5ffb640 1 -- 172.21.15.16:0/2799242313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0003c30 con 0x7fbec81025f0 2024-09-19T09:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbec5ffb640 1 -- 172.21.15.16:0/2799242313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec0003dd0 con 0x7fbec81025f0 2024-09-19T09:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbeceaba640 1 -- 172.21.15.16:0/2799242313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec819fc50 con 0x7fbec81025f0 2024-09-19T09:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.368+0000 7fbec5ffb640 1 -- 172.21.15.16:0/2799242313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec002fcb0 con 0x7fbec81025f0 2024-09-19T09:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.369+0000 7fbeceaba640 1 -- 172.21.15.16:0/2799242313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec81a01a0 con 0x7fbec81025f0 2024-09-19T09:29:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.371+0000 7fbec5ffb640 1 -- 172.21.15.16:0/2799242313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fbec0004050 con 0x7fbec81025f0 2024-09-19T09:29:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.371+0000 7fbed1855740 1 Processor -- start 2024-09-19T09:29:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbed1855740 1 -- start start 2024-09-19T09:29:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbed1855740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde969a0 0x5614cde96da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbed1855740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cde9bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbed1855740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde9c370 0x5614cde9e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbed1855740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5614cddfa2f0 con 0x5614cde972e0 2024-09-19T09:29:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbed1855740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5614cdda2270 con 0x5614cde969a0 2024-09-19T09:29:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbed1855740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5614cddf62b0 con 0x5614cde9c370 2024-09-19T09:29:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbecc856640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cde9bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbec5ffb640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbeb40a32f0 0x7fbeb40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbecc856640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cde9bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40182/0 (socket says 172.21.15.16:40182) 2024-09-19T09:29:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbecc856640 1 -- 172.21.15.16:0/854576905 learned_addr learned my addr 172.21.15.16:0/854576905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbec7fff640 1 --2- 172.21.15.16:0/854576905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde9c370 0x5614cde9e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbec7fff640 1 --2- 172.21.15.16:0/854576905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde969a0 0x5614cde96da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbecd057640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbeb40a32f0 0x7fbeb40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbecc856640 1 -- 172.21.15.16:0/854576905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde9c370 msgr2=0x5614cde9e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbecc856640 1 --2- 172.21.15.16:0/854576905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde9c370 0x5614cde9e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbecc856640 1 -- 172.21.15.16:0/854576905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde969a0 msgr2=0x5614cde96da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbecc856640 1 --2- 172.21.15.16:0/854576905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde969a0 0x5614cde96da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbecc856640 1 -- 172.21.15.16:0/854576905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614cdda1340 con 0x5614cde972e0 2024-09-19T09:29:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbec7fff640 1 --2- 172.21.15.16:0/854576905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde969a0 0x5614cde96da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbec5ffb640 1 -- 172.21.15.16:0/2799242313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fbec00f0d20 con 0x7fbec81025f0 2024-09-19T09:29:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.372+0000 7fbecc856640 1 --2- 172.21.15.16:0/854576905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cde9bda0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800bcb0 tx=0x7fbeb80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0031929d6c93a69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.373+0000 7fbebeffd640 1 -- 172.21.15.16:0/854576905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8004020 con 0x5614cde972e0 2024-09-19T09:29:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.373+0000 7fbebeffd640 1 -- 172.21.15.16:0/854576905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb8005500 con 0x5614cde972e0 2024-09-19T09:29:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.373+0000 7fbebeffd640 1 -- 172.21.15.16:0/854576905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8004d80 con 0x5614cde972e0 2024-09-19T09:29:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.373+0000 7fbecd057640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbeb40a32f0 0x7fbeb40a57b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbec00b9760 tx=0x7fbec002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.373+0000 7fbed1855740 1 -- 172.21.15.16:0/854576905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 msgr2=0x5614cde9bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.373+0000 7fbed1855740 1 --2- 172.21.15.16:0/854576905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cde9bda0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800bcb0 tx=0x7fbeb80107a0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.373+0000 7fbed1855740 1 -- 172.21.15.16:0/854576905 shutdown_connections 2024-09-19T09:29:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.373+0000 7fbed1855740 1 --2- 172.21.15.16:0/854576905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde9c370 0x5614cde9e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.373+0000 7fbed1855740 1 --2- 172.21.15.16:0/854576905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cde9bda0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.373+0000 7fbed1855740 1 --2- 172.21.15.16:0/854576905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde969a0 0x5614cde96da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.373+0000 7fbed1855740 1 -- 172.21.15.16:0/854576905 >> 172.21.15.16:0/854576905 conn(0x5614cde92010 msgr2=0x5614cde92410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.374+0000 7fbed1855740 1 -- 172.21.15.16:0/854576905 shutdown_connections 2024-09-19T09:29:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.374+0000 7fbed1855740 1 -- 172.21.15.16:0/854576905 wait complete. 2024-09-19T09:29:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.374+0000 7fbed1855740 1 -- start start 2024-09-19T09:29:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.374+0000 7fbed1855740 1 Processor -- start 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.374+0000 7fbed1855740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde969a0 0x5614cdea0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.374+0000 7fbed1855740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cdea10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.374+0000 7fbed1855740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde9c370 0x5614cdea80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.374+0000 7fbed1855740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5614cde9f340 con 0x5614cde972e0 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.374+0000 7fbed1855740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5614cdd83290 con 0x5614cde9c370 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.374+0000 7fbed1855740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5614cde9f4c0 con 0x5614cde969a0 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.374+0000 7fbed1855740 10 client.0 _ll_register_callbacks cb 0x5614cddb4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbecc856640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cdea10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbecc856640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cdea10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40192/0 (socket says 172.21.15.16:40192) 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbecc856640 1 -- 172.21.15.16:0/2334113038 learned_addr learned my addr 172.21.15.16:0/2334113038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbec7fff640 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde969a0 0x5614cdea0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbec7fff640 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde9c370 0x5614cdea80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbecc856640 1 -- 172.21.15.16:0/2334113038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde969a0 msgr2=0x5614cdea0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbecc856640 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde969a0 0x5614cdea0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbecc856640 1 -- 172.21.15.16:0/2334113038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde9c370 msgr2=0x5614cdea80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbecc856640 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde9c370 0x5614cdea80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbecc856640 1 -- 172.21.15.16:0/2334113038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614cdb0d210 con 0x5614cde972e0 2024-09-19T09:29:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbec7fff640 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde9c370 0x5614cdea80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbecc856640 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cdea10a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800bfd0 tx=0x7fbeb8004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88138f9258218d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbea7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbea7fff640 1 -- 172.21.15.16:0/2334113038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb802a060 con 0x5614cde972e0 2024-09-19T09:29:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbea7fff640 1 -- 172.21.15.16:0/2334113038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fbeb8004310 con 0x5614cde972e0 2024-09-19T09:29:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.375+0000 7fbed1855740 1 -- 172.21.15.16:0/2334113038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeb8005500 con 0x5614cde972e0 2024-09-19T09:29:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.376+0000 7fbea7fff640 1 -- 172.21.15.16:0/2334113038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbeb80035f0 con 0x5614cde972e0 2024-09-19T09:29:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.376+0000 7fbed1855740 10 client.15972 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.376+0000 7fbed1855740 1 -- 172.21.15.16:0/2334113038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614cdbd9620 con 0x5614cde972e0 2024-09-19T09:29:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.376+0000 7fbea7fff640 1 -- 172.21.15.16:0/2334113038 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb8005500 con 0x5614cde972e0 2024-09-19T09:29:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.376+0000 7fbea7fff640 1 -- 172.21.15.16:0/2334113038 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbeb802f070 con 0x5614cde972e0 2024-09-19T09:29:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.376+0000 7fbed1855740 10 client.15972 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.376+0000 7fbed1855740 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.376+0000 7fbed1855740 20 client.15972 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.376+0000 7fbed1855740 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5614cdeac4f0 0x5614cdeae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.376+0000 7fbed1855740 4 client.15972 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.376+0000 7fbed1855740 1 -- 172.21.15.16:0/2334113038 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614cde07a20 con 0x5614cdeac4f0 2024-09-19T09:29:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.376+0000 7fbec7fff640 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5614cdeac4f0 0x5614cdeae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.377+0000 7fbec7fff640 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5614cdeac4f0 0x5614cdeae8e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da1e81d6a9597ab server_cookie=e2e78751112f4b4c in_seq=0 out_seq=0 2024-09-19T09:29:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.377+0000 7fbea7fff640 10 client.15972 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbea7fff640 1 -- 172.21.15.16:0/2334113038 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614cde07a20 con 0x5614cdeac4f0 2024-09-19T09:29:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbea7fff640 10 client.15972 handle_command_reply: tid=0 2024-09-19T09:29:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbed1855740 1 client.15972 shutdown 2024-09-19T09:29:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbed1855740 1 -- 172.21.15.16:0/2334113038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 msgr2=0x5614cdea10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbed1855740 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cdea10a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800bfd0 tx=0x7fbeb8004790 comp rx=0 tx=0).stop 2024-09-19T09:29:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbed1855740 1 -- 172.21.15.16:0/2334113038 shutdown_connections 2024-09-19T09:29:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbed1855740 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5614cdeac4f0 0x5614cdeae8e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbed1855740 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde9c370 0x5614cdea80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbed1855740 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cdea10a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbea7fff640 0 client.15972 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbed1855740 1 --2- 172.21.15.16:0/2334113038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde969a0 0x5614cdea0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbed1855740 1 -- 172.21.15.16:0/2334113038 >> 172.21.15.16:0/2334113038 conn(0x5614cde92010 msgr2=0x5614cde93820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbed1855740 1 -- 172.21.15.16:0/2334113038 shutdown_connections 2024-09-19T09:29:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbed1855740 1 -- 172.21.15.16:0/2334113038 wait complete. 2024-09-19T09:29:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.378+0000 7fbed1855740 20 client.15972 trim_cache size 0 max 16384 2024-09-19T09:29:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbed1855740 1 Processor -- start 2024-09-19T09:29:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbed1855740 1 -- start start 2024-09-19T09:29:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbed1855740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde969a0 0x5614cde910b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbed1855740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde972e0 0x5614cdea80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbed1855740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde9c370 0x5614cde93400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbed1855740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81a9010 con 0x5614cde9c370 2024-09-19T09:29:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbed1855740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81aab50 con 0x5614cde969a0 2024-09-19T09:29:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbed1855740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81a96e0 con 0x5614cde972e0 2024-09-19T09:29:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbec7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde9c370 0x5614cde93400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbec7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde9c370 0x5614cde93400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40206/0 (socket says 172.21.15.16:40206) 2024-09-19T09:29:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbec7fff640 1 -- 172.21.15.16:0/1254100180 learned_addr learned my addr 172.21.15.16:0/1254100180 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbecc856640 1 --2- 172.21.15.16:0/1254100180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde972e0 0x5614cdea80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbec7fff640 1 --2- 172.21.15.16:0/1254100180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde969a0 0x5614cde910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbecc856640 1 -- 172.21.15.16:0/1254100180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde969a0 msgr2=0x5614cde910b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbecc856640 1 --2- 172.21.15.16:0/1254100180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde969a0 0x5614cde910b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbecc856640 1 -- 172.21.15.16:0/1254100180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde9c370 msgr2=0x5614cde93400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbecc856640 1 --2- 172.21.15.16:0/1254100180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde9c370 0x5614cde93400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbecc856640 1 -- 172.21.15.16:0/1254100180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614cde90650 con 0x5614cde972e0 2024-09-19T09:29:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.393+0000 7fbec7fff640 1 --2- 172.21.15.16:0/1254100180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde9c370 0x5614cde93400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.394+0000 7fbecc856640 1 --2- 172.21.15.16:0/1254100180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde972e0 0x5614cdea80a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8001eb0 tx=0x7fbeb8004810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=759713144c70c24a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.394+0000 7fbebe7fc640 1 -- 172.21.15.16:0/1254100180 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8035070 con 0x5614cde972e0 2024-09-19T09:29:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.394+0000 7fbebe7fc640 1 -- 172.21.15.16:0/1254100180 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb8005500 con 0x5614cde972e0 2024-09-19T09:29:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.394+0000 7fbebe7fc640 1 -- 172.21.15.16:0/1254100180 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb802a040 con 0x5614cde972e0 2024-09-19T09:29:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.394+0000 7fbed1855740 1 -- 172.21.15.16:0/1254100180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde972e0 msgr2=0x5614cdea80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 --2- 172.21.15.16:0/1254100180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde972e0 0x5614cdea80a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8001eb0 tx=0x7fbeb8004810 comp rx=0 tx=0).stop 2024-09-19T09:29:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 -- 172.21.15.16:0/1254100180 shutdown_connections 2024-09-19T09:29:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 --2- 172.21.15.16:0/1254100180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde972e0 0x5614cdea80a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 --2- 172.21.15.16:0/1254100180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde9c370 0x5614cde93400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 --2- 172.21.15.16:0/1254100180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde969a0 0x5614cde910b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 -- 172.21.15.16:0/1254100180 >> 172.21.15.16:0/1254100180 conn(0x5614cde92010 msgr2=0x5614cde909a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 -- 172.21.15.16:0/1254100180 shutdown_connections 2024-09-19T09:29:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 -- 172.21.15.16:0/1254100180 wait complete. 2024-09-19T09:29:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 -- start start 2024-09-19T09:29:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 Processor -- start 2024-09-19T09:29:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde969a0 0x5614cdea5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cdea5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde9c370 0x5614cdea6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81aa480 con 0x5614cde972e0 2024-09-19T09:29:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81a9db0 con 0x5614cde9c370 2024-09-19T09:29:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.395+0000 7fbed1855740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbec81a6fd0 con 0x5614cde969a0 2024-09-19T09:29:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbed1855740 10 client.0 _ll_register_callbacks cb 0x7fbeb0009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbecc856640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cdea5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbecc856640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cdea5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40222/0 (socket says 172.21.15.16:40222) 2024-09-19T09:29:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbecc856640 1 -- 172.21.15.16:0/610984677 learned_addr learned my addr 172.21.15.16:0/610984677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbec7fff640 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde969a0 0x5614cdea5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbec7fff640 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde9c370 0x5614cdea6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbecc856640 1 -- 172.21.15.16:0/610984677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde969a0 msgr2=0x5614cdea5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbecc856640 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde969a0 0x5614cdea5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbecc856640 1 -- 172.21.15.16:0/610984677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde9c370 msgr2=0x5614cdea6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbecc856640 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde9c370 0x5614cdea6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbecc856640 1 -- 172.21.15.16:0/610984677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614cdea7490 con 0x5614cde972e0 2024-09-19T09:29:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbec7fff640 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde969a0 0x5614cdea5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbecc856640 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cdea5ec0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800b7f0 tx=0x7fbeb8004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f7aa1573a399c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbea7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbea7fff640 1 -- 172.21.15.16:0/610984677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb802a040 con 0x5614cde972e0 2024-09-19T09:29:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbea7fff640 1 -- 172.21.15.16:0/610984677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fbeb80039a0 con 0x5614cde972e0 2024-09-19T09:29:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbed1855740 1 -- 172.21.15.16:0/610984677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeb8005500 con 0x5614cde972e0 2024-09-19T09:29:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbea7fff640 1 -- 172.21.15.16:0/610984677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbeb8048050 con 0x5614cde972e0 2024-09-19T09:29:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbed1855740 10 client.15978 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.396+0000 7fbed1855740 1 -- 172.21.15.16:0/610984677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614cdf17290 con 0x5614cde972e0 2024-09-19T09:29:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.397+0000 7fbea7fff640 1 -- 172.21.15.16:0/610984677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb8005500 con 0x5614cde972e0 2024-09-19T09:29:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.397+0000 7fbea7fff640 1 -- 172.21.15.16:0/610984677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbeb802f070 con 0x5614cde972e0 2024-09-19T09:29:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.397+0000 7fbed1855740 10 client.15978 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.397+0000 7fbed1855740 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.397+0000 7fbed1855740 20 client.15978 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.397+0000 7fbed1855740 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5614cdf17bb0 0x5614cdf19fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.397+0000 7fbed1855740 4 client.15978 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:29:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.397+0000 7fbed1855740 1 -- 172.21.15.16:0/610984677 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614cde01d70 con 0x5614cdf17bb0 2024-09-19T09:29:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.397+0000 7fbec7fff640 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5614cdf17bb0 0x5614cdf19fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.397+0000 7fbea7fff640 10 client.15978 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.397+0000 7fbec7fff640 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5614cdf17bb0 0x5614cdf19fd0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=657eb4a22d10cd2e server_cookie=15c331422ca19cb3 in_seq=0 out_seq=0 2024-09-19T09:29:11.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:11 smithi016 ceph-mon[30979]: pgmap v385: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-09-19T09:29:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.406+0000 7f4dd17fa640 1 -- 172.21.15.16:0/4280323772 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5644cbd61be0 con 0x5644cbe804e0 2024-09-19T09:29:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.406+0000 7f4dd17fa640 10 client.15960 handle_command_reply: tid=0 2024-09-19T09:29:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.406+0000 7f4df1e4f740 1 client.15960 shutdown 2024-09-19T09:29:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.407+0000 7f4df1e4f740 1 -- 172.21.15.16:0/4280323772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbe04e40 msgr2=0x5644cbe78870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.407+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/4280323772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbe04e40 0x5644cbe78870 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4de00098b0 tx=0x7f4de0009400 comp rx=0 tx=0).stop 2024-09-19T09:29:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.407+0000 7f4df1e4f740 1 -- 172.21.15.16:0/4280323772 shutdown_connections 2024-09-19T09:29:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.407+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/4280323772 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5644cbe804e0 0x5644cbe82900 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.407+0000 7f4dd17fa640 0 client.15960 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.407+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/4280323772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644cbdf6070 0x5644cbe09a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.407+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/4280323772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5644cbe04e40 0x5644cbe78870 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.407+0000 7f4df1e4f740 1 --2- 172.21.15.16:0/4280323772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644cbdf5750 0x5644cbe09510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.407+0000 7f4df1e4f740 1 -- 172.21.15.16:0/4280323772 >> 172.21.15.16:0/4280323772 conn(0x5644cbdf1de0 msgr2=0x5644cbe02530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.407+0000 7f4df1e4f740 1 -- 172.21.15.16:0/4280323772 shutdown_connections 2024-09-19T09:29:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.407+0000 7f4df1e4f740 1 -- 172.21.15.16:0/4280323772 wait complete. 2024-09-19T09:29:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.407+0000 7f4df1e4f740 20 client.15960 trim_cache size 0 max 16384 2024-09-19T09:29:11.410 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:29:11.410 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-19T09:29:11.411 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24500, 2024-09-19T09:29:11.411 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 10, 2024-09-19T09:29:11.411 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-19T09:29:11.411 INFO:teuthology.orchestra.run.smithi016.stdout: "7b034bee": { 2024-09-19T09:29:11.411 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 10, 2024-09-19T09:29:11.411 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-19T09:29:11.411 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-19T09:29:11.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-19T09:29:11.411 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-19T09:29:11.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:11.411 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-19T09:29:11.412 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.3, 2024-09-19T09:29:11.412 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-19T09:29:11.412 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-19T09:29:11.412 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-19T09:29:11.412 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-19T09:29:11.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-19T09:29:11.412 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-19T09:29:11.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:11.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:11.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:11.413 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:11.413 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:11.413 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:29:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.408+0000 7f4defa76640 1 -- 172.21.15.16:0/36662086 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4dd40a3240 msgr2=0x7f4dd40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.408+0000 7f4defa76640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4dd40a3240 0x7f4dd40a5700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4de0020000 tx=0x7f4de0021000 comp rx=0 tx=0).stop 2024-09-19T09:29:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.408+0000 7f4defa76640 1 -- 172.21.15.16:0/36662086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de8102790 msgr2=0x7f4de8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.408+0000 7f4defa76640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de8102790 0x7f4de8198980 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800d950 tx=0x7f4dd800de10 comp rx=0 tx=0).stop 2024-09-19T09:29:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.408+0000 7f4defa76640 1 -- 172.21.15.16:0/36662086 shutdown_connections 2024-09-19T09:29:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.408+0000 7f4defa76640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4de8106f10 0x7f4de819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.408+0000 7f4defa76640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de8102790 0x7f4de8198980 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.408+0000 7f4defa76640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4dd40a3240 0x7f4dd40a5700 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.408+0000 7f4defa76640 1 --2- 172.21.15.16:0/36662086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4de8101de0 0x7f4de8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.408+0000 7f4defa76640 1 -- 172.21.15.16:0/36662086 >> 172.21.15.16:0/36662086 conn(0x7f4de80fd8c0 msgr2=0x7f4de80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.408+0000 7f4defa76640 1 -- 172.21.15.16:0/36662086 shutdown_connections 2024-09-19T09:29:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:11.408+0000 7f4defa76640 1 -- 172.21.15.16:0/36662086 wait complete. 2024-09-19T09:29:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.413+0000 7fbea7fff640 1 -- 172.21.15.16:0/610984677 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5614cde01d70 con 0x5614cdf17bb0 2024-09-19T09:29:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbea7fff640 10 client.15978 handle_command_reply: tid=0 2024-09-19T09:29:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbed1855740 1 client.15978 shutdown 2024-09-19T09:29:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbed1855740 1 -- 172.21.15.16:0/610984677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 msgr2=0x5614cdea5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbed1855740 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cdea5ec0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800b7f0 tx=0x7fbeb8004b10 comp rx=0 tx=0).stop 2024-09-19T09:29:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbed1855740 1 -- 172.21.15.16:0/610984677 shutdown_connections 2024-09-19T09:29:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbed1855740 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x5614cdf17bb0 0x5614cdf19fd0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbed1855740 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614cde9c370 0x5614cdea6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbea7fff640 0 client.15978 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbed1855740 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614cde972e0 0x5614cdea5ec0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbed1855740 1 --2- 172.21.15.16:0/610984677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614cde969a0 0x5614cdea5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbed1855740 1 -- 172.21.15.16:0/610984677 >> 172.21.15.16:0/610984677 conn(0x5614cde92010 msgr2=0x5614cde911c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbed1855740 1 -- 172.21.15.16:0/610984677 shutdown_connections 2024-09-19T09:29:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbed1855740 1 -- 172.21.15.16:0/610984677 wait complete. 2024-09-19T09:29:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.414+0000 7fbed1855740 20 client.15978 trim_cache size 0 max 16384 2024-09-19T09:29:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-19T09:29:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-19T09:29:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-19T09:29:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-19T09:29:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.415+0000 7fbeceaba640 1 -- 172.21.15.16:0/2799242313 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbeb40a32f0 msgr2=0x7fbeb40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.415+0000 7fbeceaba640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbeb40a32f0 0x7fbeb40a57b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbec00b9760 tx=0x7fbec002a8c0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.415+0000 7fbeceaba640 1 -- 172.21.15.16:0/2799242313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec81025f0 msgr2=0x7fbec819f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.415+0000 7fbeceaba640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec81025f0 0x7fbec819f350 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbec0031ae0 tx=0x7fbec0031ee0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.415+0000 7fbeceaba640 1 -- 172.21.15.16:0/2799242313 shutdown_connections 2024-09-19T09:29:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.415+0000 7fbeceaba640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec81025f0 0x7fbec819f350 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.415+0000 7fbeceaba640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbec8101c30 0x7fbec8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.415+0000 7fbeceaba640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fbeb40a32f0 0x7fbeb40a57b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.415+0000 7fbeceaba640 1 --2- 172.21.15.16:0/2799242313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec8100a40 0x7fbec8197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.415+0000 7fbeceaba640 1 -- 172.21.15.16:0/2799242313 >> 172.21.15.16:0/2799242313 conn(0x7fbec80fc1e0 msgr2=0x7fbec80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.415+0000 7fbeceaba640 1 -- 172.21.15.16:0/2799242313 shutdown_connections 2024-09-19T09:29:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:11.415+0000 7fbeceaba640 1 -- 172.21.15.16:0/2799242313 wait complete. 2024-09-19T09:29:11.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:11 smithi142 ceph-mon[35032]: pgmap v385: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-09-19T09:29:11.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:11 smithi097 ceph-mon[34864]: pgmap v385: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-09-19T09:29:12.111 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7b034bee', quiesce took 5.9 seconds. Will release after: 14.450000000000001 2024-09-19T09:29:13.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:13 smithi016 ceph-mon[30979]: pgmap v386: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-09-19T09:29:13.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:13 smithi142 ceph-mon[35032]: pgmap v386: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-09-19T09:29:13.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:13 smithi097 ceph-mon[34864]: pgmap v386: 97 pgs: 97 active+clean; 1.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-09-19T09:29:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:15 smithi016 ceph-mon[30979]: pgmap v387: 97 pgs: 97 active+clean; 889 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 23 op/s 2024-09-19T09:29:15.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:15 smithi142 ceph-mon[35032]: pgmap v387: 97 pgs: 97 active+clean; 889 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 23 op/s 2024-09-19T09:29:15.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:15 smithi097 ceph-mon[34864]: pgmap v387: 97 pgs: 97 active+clean; 889 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 23 op/s 2024-09-19T09:29:17.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:17 smithi016 ceph-mon[30979]: pgmap v388: 97 pgs: 97 active+clean; 889 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 16 op/s 2024-09-19T09:29:17.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:17 smithi142 ceph-mon[35032]: pgmap v388: 97 pgs: 97 active+clean; 889 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 16 op/s 2024-09-19T09:29:17.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:17 smithi097 ceph-mon[34864]: pgmap v388: 97 pgs: 97 active+clean; 889 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 16 op/s 2024-09-19T09:29:19.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:19 smithi016 ceph-mon[30979]: pgmap v389: 97 pgs: 97 active+clean; 787 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.6 KiB/s wr, 19 op/s 2024-09-19T09:29:19.629 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:19 smithi142 ceph-mon[35032]: pgmap v389: 97 pgs: 97 active+clean; 787 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.6 KiB/s wr, 19 op/s 2024-09-19T09:29:19.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:19 smithi097 ceph-mon[34864]: pgmap v389: 97 pgs: 97 active+clean; 787 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.6 KiB/s wr, 19 op/s 2024-09-19T09:29:21.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:21 smithi016 ceph-mon[30979]: pgmap v390: 97 pgs: 97 active+clean; 787 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 18 op/s 2024-09-19T09:29:21.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:21 smithi142 ceph-mon[35032]: pgmap v390: 97 pgs: 97 active+clean; 787 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 18 op/s 2024-09-19T09:29:21.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:21 smithi097 ceph-mon[34864]: pgmap v390: 97 pgs: 97 active+clean; 787 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 18 op/s 2024-09-19T09:29:23.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:23 smithi016 ceph-mon[30979]: pgmap v391: 97 pgs: 97 active+clean; 787 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.9 KiB/s wr, 16 op/s 2024-09-19T09:29:23.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:23 smithi142 ceph-mon[35032]: pgmap v391: 97 pgs: 97 active+clean; 787 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.9 KiB/s wr, 16 op/s 2024-09-19T09:29:23.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:23 smithi097 ceph-mon[34864]: pgmap v391: 97 pgs: 97 active+clean; 787 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.9 KiB/s wr, 16 op/s 2024-09-19T09:29:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.516+0000 7f984a480640 1 -- 172.21.15.16:0/797588910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98440ffe20 msgr2=0x7f9844100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.516+0000 7f984a480640 1 --2- 172.21.15.16:0/797588910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98440ffe20 0x7f9844100220 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9834009900 tx=0x7f983402a410 comp rx=0 tx=0).stop 2024-09-19T09:29:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.517+0000 7f984a480640 1 -- 172.21.15.16:0/797588910 shutdown_connections 2024-09-19T09:29:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.517+0000 7f984a480640 1 --2- 172.21.15.16:0/797588910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9844103850 0x7f984410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.517+0000 7f984a480640 1 --2- 172.21.15.16:0/797588910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9844102e90 0x7f9844103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.517+0000 7f984a480640 1 --2- 172.21.15.16:0/797588910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98440ffe20 0x7f9844100220 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.517+0000 7f984a480640 1 -- 172.21.15.16:0/797588910 >> 172.21.15.16:0/797588910 conn(0x7f98440fb980 msgr2=0x7f98440fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.517+0000 7f984a480640 1 -- 172.21.15.16:0/797588910 shutdown_connections 2024-09-19T09:29:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.517+0000 7f984a480640 1 -- 172.21.15.16:0/797588910 wait complete. 2024-09-19T09:29:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.517+0000 7f984a480640 1 Processor -- start 2024-09-19T09:29:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.517+0000 7f984a480640 1 -- start start 2024-09-19T09:29:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f984a480640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98440ffe20 0x7f9844197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f984a480640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9844102e90 0x7f98441982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f984a480640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9844103850 0x7f984419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f984a480640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f984410d820 con 0x7f9844103850 2024-09-19T09:29:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f984a480640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f984410d6a0 con 0x7f9844102e90 2024-09-19T09:29:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f984a480640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f984410d9a0 con 0x7f98440ffe20 2024-09-19T09:29:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f9848a1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9844103850 0x7f984419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f9848a1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9844103850 0x7f984419f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50870/0 (socket says 172.21.15.16:50870) 2024-09-19T09:29:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f9848a1d640 1 -- 172.21.15.16:0/3214495631 learned_addr learned my addr 172.21.15.16:0/3214495631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f98437fe640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9844102e90 0x7f98441982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f9843fff640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98440ffe20 0x7f9844197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f9848a1d640 1 -- 172.21.15.16:0/3214495631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98440ffe20 msgr2=0x7f9844197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f9848a1d640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98440ffe20 0x7f9844197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f9848a1d640 1 -- 172.21.15.16:0/3214495631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9844102e90 msgr2=0x7f98441982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f9848a1d640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9844102e90 0x7f98441982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f9848a1d640 1 -- 172.21.15.16:0/3214495631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984419f9c0 con 0x7f9844103850 2024-09-19T09:29:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.518+0000 7f9848a1d640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9844103850 0x7f984419f320 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f983c00cce0 tx=0x7f983c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.519+0000 7f98417fa640 1 -- 172.21.15.16:0/3214495631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c00f450 con 0x7f9844103850 2024-09-19T09:29:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.519+0000 7f98417fa640 1 -- 172.21.15.16:0/3214495631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c00fad0 con 0x7f9844103850 2024-09-19T09:29:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.519+0000 7f98417fa640 1 -- 172.21.15.16:0/3214495631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c018c10 con 0x7f9844103850 2024-09-19T09:29:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.519+0000 7f984a480640 1 -- 172.21.15.16:0/3214495631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984419fcb0 con 0x7f9844103850 2024-09-19T09:29:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.519+0000 7f984a480640 1 -- 172.21.15.16:0/3214495631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441a0180 con 0x7f9844103850 2024-09-19T09:29:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.522+0000 7f98417fa640 1 -- 172.21.15.16:0/3214495631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f983c00f5f0 con 0x7f9844103850 2024-09-19T09:29:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.522+0000 7f984d20e740 1 Processor -- start 2024-09-19T09:29:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.522+0000 7f984d20e740 1 -- start start 2024-09-19T09:29:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.522+0000 7f984d20e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 0x560fbd73ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.522+0000 7f984d20e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73b2e0 0x560fbd73fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.522+0000 7f984d20e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd740370 0x560fbd742830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.522+0000 7f984d20e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560fbd69e2f0 con 0x560fbd73a9a0 2024-09-19T09:29:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.522+0000 7f984d20e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560fbd646270 con 0x560fbd73b2e0 2024-09-19T09:29:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.522+0000 7f984d20e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560fbd69a2b0 con 0x560fbd740370 2024-09-19T09:29:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.522+0000 7f98437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 0x560fbd73ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.522+0000 7f98437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 0x560fbd73ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50876/0 (socket says 172.21.15.16:50876) 2024-09-19T09:29:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.522+0000 7f98437fe640 1 -- 172.21.15.16:0/1570389390 learned_addr learned my addr 172.21.15.16:0/1570389390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f9843fff640 1 --2- 172.21.15.16:0/1570389390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73b2e0 0x560fbd73fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f98437fe640 1 --2- 172.21.15.16:0/1570389390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd740370 0x560fbd742830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f98417fa640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f982c0a32f0 0x7f982c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f98437fe640 1 -- 172.21.15.16:0/1570389390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd740370 msgr2=0x560fbd742830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f98437fe640 1 --2- 172.21.15.16:0/1570389390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd740370 0x560fbd742830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f98437fe640 1 -- 172.21.15.16:0/1570389390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73b2e0 msgr2=0x560fbd73fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f98437fe640 1 --2- 172.21.15.16:0/1570389390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73b2e0 0x560fbd73fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f98437fe640 1 -- 172.21.15.16:0/1570389390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fbd645340 con 0x560fbd73a9a0 2024-09-19T09:29:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f9843fff640 1 --2- 172.21.15.16:0/1570389390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73b2e0 0x560fbd73fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f9848a1d640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f982c0a32f0 0x7f982c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f98417fa640 1 -- 172.21.15.16:0/3214495631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f983c0cffe0 con 0x7f9844103850 2024-09-19T09:29:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f98437fe640 1 --2- 172.21.15.16:0/1570389390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 0x560fbd73ada0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9838013610 tx=0x7f9838009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ec5016e14cafee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f982a7fc640 1 -- 172.21.15.16:0/1570389390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983800f600 con 0x560fbd73a9a0 2024-09-19T09:29:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f982a7fc640 1 -- 172.21.15.16:0/1570389390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983800fc80 con 0x560fbd73a9a0 2024-09-19T09:29:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.523+0000 7f982a7fc640 1 -- 172.21.15.16:0/1570389390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838010b80 con 0x560fbd73a9a0 2024-09-19T09:29:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.524+0000 7f984d20e740 1 -- 172.21.15.16:0/1570389390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 msgr2=0x560fbd73ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.524+0000 7f984d20e740 1 --2- 172.21.15.16:0/1570389390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 0x560fbd73ada0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9838013610 tx=0x7f9838009890 comp rx=0 tx=0).stop 2024-09-19T09:29:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.524+0000 7f9848a1d640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f982c0a32f0 0x7f982c0a57b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f983c00ccb0 tx=0x7f983c091a50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.524+0000 7f984d20e740 1 -- 172.21.15.16:0/1570389390 shutdown_connections 2024-09-19T09:29:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.524+0000 7f984d20e740 1 --2- 172.21.15.16:0/1570389390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd740370 0x560fbd742830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.524+0000 7f984d20e740 1 --2- 172.21.15.16:0/1570389390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73b2e0 0x560fbd73fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.524+0000 7f984d20e740 1 --2- 172.21.15.16:0/1570389390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 0x560fbd73ada0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.524+0000 7f984d20e740 1 -- 172.21.15.16:0/1570389390 >> 172.21.15.16:0/1570389390 conn(0x560fbd736010 msgr2=0x560fbd736410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.524+0000 7f984d20e740 1 -- 172.21.15.16:0/1570389390 shutdown_connections 2024-09-19T09:29:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.524+0000 7f984d20e740 1 -- 172.21.15.16:0/1570389390 wait complete. 2024-09-19T09:29:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.524+0000 7f984d20e740 1 -- start start 2024-09-19T09:29:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.525+0000 7f984d20e740 1 Processor -- start 2024-09-19T09:29:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.525+0000 7f984d20e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73a9a0 0x560fbd744ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.525+0000 7f984d20e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd73b2e0 0x560fbd7450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.525+0000 7f984d20e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 0x560fbd74c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.525+0000 7f984d20e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560fbd743340 con 0x560fbd740370 2024-09-19T09:29:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.525+0000 7f984d20e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560fbd627290 con 0x560fbd73a9a0 2024-09-19T09:29:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.525+0000 7f984d20e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560fbd7434c0 con 0x560fbd73b2e0 2024-09-19T09:29:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.525+0000 7f984d20e740 10 client.0 _ll_register_callbacks cb 0x560fbd658140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.525+0000 7f98437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 0x560fbd74c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.525+0000 7f98437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 0x560fbd74c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50888/0 (socket says 172.21.15.16:50888) 2024-09-19T09:29:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.525+0000 7f98437fe640 1 -- 172.21.15.16:0/3106460659 learned_addr learned my addr 172.21.15.16:0/3106460659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.525+0000 7f9843fff640 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd73b2e0 0x560fbd7450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.525+0000 7f98437fe640 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73a9a0 0x560fbd744ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f9843fff640 1 -- 172.21.15.16:0/3106460659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73a9a0 msgr2=0x560fbd744ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f9843fff640 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73a9a0 0x560fbd744ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f9843fff640 1 -- 172.21.15.16:0/3106460659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 msgr2=0x560fbd74c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f9843fff640 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 0x560fbd74c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f9843fff640 1 -- 172.21.15.16:0/3106460659 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fbd3b1210 con 0x560fbd73b2e0 2024-09-19T09:29:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f98437fe640 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 0x560fbd74c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f9843fff640 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd73b2e0 0x560fbd7450e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9834009230 tx=0x7f98340044e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a70a6ebe1fb1f574 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f98237fe640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-09-19T09:29:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f98237fe640 1 -- 172.21.15.16:0/3106460659 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834035040 con 0x560fbd73b2e0 2024-09-19T09:29:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f984d20e740 1 -- 172.21.15.16:0/3106460659 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983800fc80 con 0x560fbd73b2e0 2024-09-19T09:29:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f98237fe640 1 -- 172.21.15.16:0/3106460659 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f9834005590 con 0x560fbd73b2e0 2024-09-19T09:29:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f98237fe640 1 -- 172.21.15.16:0/3106460659 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9834004770 con 0x560fbd73b2e0 2024-09-19T09:29:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f984d20e740 10 client.25480 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f984d20e740 1 -- 172.21.15.16:0/3106460659 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560fbd47d620 con 0x560fbd73b2e0 2024-09-19T09:29:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.526+0000 7f98237fe640 1 -- 172.21.15.16:0/3106460659 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983800fc80 con 0x560fbd73b2e0 2024-09-19T09:29:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.527+0000 7f98237fe640 1 -- 172.21.15.16:0/3106460659 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f98340035f0 con 0x560fbd73b2e0 2024-09-19T09:29:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.527+0000 7f984d20e740 10 client.25480 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.527+0000 7f984d20e740 10 client.25480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.527+0000 7f984d20e740 20 client.25480 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.527+0000 7f984d20e740 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fbd750860 0x560fbd752c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.527+0000 7f984d20e740 4 client.25480 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.527+0000 7f984d20e740 1 -- 172.21.15.16:0/3106460659 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fbd6aba20 con 0x560fbd750860 2024-09-19T09:29:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.527+0000 7f98437fe640 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fbd750860 0x560fbd752c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.527+0000 7f98237fe640 10 client.25480 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.527+0000 7f98437fe640 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fbd750860 0x560fbd752c70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdef828d970d8921 server_cookie=8e0089225256a8ad in_seq=0 out_seq=0 2024-09-19T09:29:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.528+0000 7f98237fe640 1 -- 172.21.15.16:0/3106460659 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fbd6aba20 con 0x560fbd750860 2024-09-19T09:29:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.528+0000 7f98237fe640 10 client.25480 handle_command_reply: tid=0 2024-09-19T09:29:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.528+0000 7f984d20e740 1 client.25480 shutdown 2024-09-19T09:29:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.528+0000 7f984d20e740 1 -- 172.21.15.16:0/3106460659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd73b2e0 msgr2=0x560fbd7450e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.528+0000 7f984d20e740 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd73b2e0 0x560fbd7450e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9834009230 tx=0x7f98340044e0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.528+0000 7f984d20e740 1 -- 172.21.15.16:0/3106460659 shutdown_connections 2024-09-19T09:29:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.528+0000 7f984d20e740 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fbd750860 0x560fbd752c70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.528+0000 7f984d20e740 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd73b2e0 0x560fbd7450e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.528+0000 7f984d20e740 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 0x560fbd74c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.528+0000 7f984d20e740 1 --2- 172.21.15.16:0/3106460659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73a9a0 0x560fbd744ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.528+0000 7f98237fe640 0 client.25480 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.528+0000 7f984d20e740 1 -- 172.21.15.16:0/3106460659 >> 172.21.15.16:0/3106460659 conn(0x560fbd736010 msgr2=0x560fbd737820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.528+0000 7f984d20e740 1 -- 172.21.15.16:0/3106460659 shutdown_connections 2024-09-19T09:29:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.529+0000 7f984d20e740 1 -- 172.21.15.16:0/3106460659 wait complete. 2024-09-19T09:29:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.529+0000 7f984d20e740 20 client.25480 trim_cache size 0 max 16384 2024-09-19T09:29:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.543+0000 7f984d20e740 1 Processor -- start 2024-09-19T09:29:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.543+0000 7f984d20e740 1 -- start start 2024-09-19T09:29:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f984d20e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 0x560fbd736f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f984d20e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73b2e0 0x560fbd737460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f984d20e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd740370 0x560fbd7379a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f984d20e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98441a8f40 con 0x560fbd73a9a0 2024-09-19T09:29:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f984d20e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f98441aaa80 con 0x560fbd73b2e0 2024-09-19T09:29:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f984d20e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98441a9610 con 0x560fbd740370 2024-09-19T09:29:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f98437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 0x560fbd736f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f98437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 0x560fbd736f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50892/0 (socket says 172.21.15.16:50892) 2024-09-19T09:29:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f98437fe640 1 -- 172.21.15.16:0/1853437903 learned_addr learned my addr 172.21.15.16:0/1853437903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f9843fff640 1 --2- 172.21.15.16:0/1853437903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73b2e0 0x560fbd737460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f98437fe640 1 -- 172.21.15.16:0/1853437903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd740370 msgr2=0x560fbd7379a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f98437fe640 1 --2- 172.21.15.16:0/1853437903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd740370 0x560fbd7379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f98437fe640 1 -- 172.21.15.16:0/1853437903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73b2e0 msgr2=0x560fbd737460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f98437fe640 1 --2- 172.21.15.16:0/1853437903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73b2e0 0x560fbd737460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f98437fe640 1 -- 172.21.15.16:0/1853437903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fbd7532f0 con 0x560fbd73a9a0 2024-09-19T09:29:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f9843fff640 1 --2- 172.21.15.16:0/1853437903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73b2e0 0x560fbd737460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f98437fe640 1 --2- 172.21.15.16:0/1853437903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 0x560fbd736f20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f98380058c0 tx=0x7f9838009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad52f3758cb76e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f9829ffb640 1 -- 172.21.15.16:0/1853437903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838028040 con 0x560fbd73a9a0 2024-09-19T09:29:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f9829ffb640 1 -- 172.21.15.16:0/1853437903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9838013190 con 0x560fbd73a9a0 2024-09-19T09:29:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.544+0000 7f9829ffb640 1 -- 172.21.15.16:0/1853437903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838010830 con 0x560fbd73a9a0 2024-09-19T09:29:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.545+0000 7f984d20e740 1 -- 172.21.15.16:0/1853437903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 msgr2=0x560fbd736f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.545+0000 7f984d20e740 1 --2- 172.21.15.16:0/1853437903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 0x560fbd736f20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f98380058c0 tx=0x7f9838009900 comp rx=0 tx=0).stop 2024-09-19T09:29:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.545+0000 7f984d20e740 1 -- 172.21.15.16:0/1853437903 shutdown_connections 2024-09-19T09:29:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.545+0000 7f984d20e740 1 --2- 172.21.15.16:0/1853437903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd740370 0x560fbd7379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.545+0000 7f984d20e740 1 --2- 172.21.15.16:0/1853437903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73b2e0 0x560fbd737460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.545+0000 7f984d20e740 1 --2- 172.21.15.16:0/1853437903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd73a9a0 0x560fbd736f20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.545+0000 7f984d20e740 1 -- 172.21.15.16:0/1853437903 >> 172.21.15.16:0/1853437903 conn(0x560fbd736010 msgr2=0x560fbd744ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.545+0000 7f984d20e740 1 -- 172.21.15.16:0/1853437903 shutdown_connections 2024-09-19T09:29:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.545+0000 7f984d20e740 1 -- 172.21.15.16:0/1853437903 wait complete. 2024-09-19T09:29:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.545+0000 7f984d20e740 1 -- start start 2024-09-19T09:29:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f984d20e740 1 Processor -- start 2024-09-19T09:29:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f984d20e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73a9a0 0x560fbd7b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f984d20e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd73b2e0 0x560fbd7b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f984d20e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 0x560fbd7bde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f984d20e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98441aa3b0 con 0x560fbd740370 2024-09-19T09:29:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f984d20e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f98441a9ce0 con 0x560fbd73a9a0 2024-09-19T09:29:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f984d20e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98441a6f00 con 0x560fbd73b2e0 2024-09-19T09:29:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f984d20e740 10 client.0 _ll_register_callbacks cb 0x7f9834009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f98437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 0x560fbd7bde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f98437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 0x560fbd7bde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50902/0 (socket says 172.21.15.16:50902) 2024-09-19T09:29:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f98437fe640 1 -- 172.21.15.16:0/3431648322 learned_addr learned my addr 172.21.15.16:0/3431648322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f9843fff640 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd73b2e0 0x560fbd7b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f98437fe640 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73a9a0 0x560fbd7b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f98437fe640 1 -- 172.21.15.16:0/3431648322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd73b2e0 msgr2=0x560fbd7b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f98437fe640 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd73b2e0 0x560fbd7b6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f98437fe640 1 -- 172.21.15.16:0/3431648322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73a9a0 msgr2=0x560fbd7b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.546+0000 7f98437fe640 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73a9a0 0x560fbd7b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f98437fe640 1 -- 172.21.15.16:0/3431648322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fbd7be480 con 0x560fbd740370 2024-09-19T09:29:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f9843fff640 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd73b2e0 0x560fbd7b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f98437fe640 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 0x560fbd7bde70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x560fbd7b5ec0 tx=0x7f98380097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720e110d2acc51fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f98237fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f98237fe640 1 -- 172.21.15.16:0/3431648322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983803e060 con 0x560fbd740370 2024-09-19T09:29:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f98237fe640 1 -- 172.21.15.16:0/3431648322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f983800f910 con 0x560fbd740370 2024-09-19T09:29:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f984d20e740 1 -- 172.21.15.16:0/3431648322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9838013190 con 0x560fbd740370 2024-09-19T09:29:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f98237fe640 1 -- 172.21.15.16:0/3431648322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f983800fb10 con 0x560fbd740370 2024-09-19T09:29:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f984d20e740 10 client.15996 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f984d20e740 1 -- 172.21.15.16:0/3431648322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560fbd7bf790 con 0x560fbd740370 2024-09-19T09:29:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f98237fe640 1 -- 172.21.15.16:0/3431648322 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9838013190 con 0x560fbd740370 2024-09-19T09:29:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f98237fe640 1 -- 172.21.15.16:0/3431648322 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9838003980 con 0x560fbd740370 2024-09-19T09:29:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f984d20e740 10 client.15996 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f984d20e740 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f984d20e740 20 client.15996 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f984d20e740 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fbd7bff00 0x560fbd7c2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f984d20e740 4 client.15996 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:29:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.547+0000 7f984d20e740 1 -- 172.21.15.16:0/3431648322 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fbd6a5d70 con 0x560fbd7bff00 2024-09-19T09:29:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.548+0000 7f9843fff640 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fbd7bff00 0x560fbd7c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.548+0000 7f9843fff640 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fbd7bff00 0x560fbd7c2320 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c527bff9d4bd8557 server_cookie=b00ff4938ec6eeec in_seq=0 out_seq=0 2024-09-19T09:29:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.548+0000 7f98237fe640 10 client.15996 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f98237fe640 1 -- 172.21.15.16:0/3431648322 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560fbd6a5d70 con 0x560fbd7bff00 2024-09-19T09:29:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f98237fe640 10 client.15996 handle_command_reply: tid=0 2024-09-19T09:29:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f984d20e740 1 client.15996 shutdown 2024-09-19T09:29:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f984d20e740 1 -- 172.21.15.16:0/3431648322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 msgr2=0x560fbd7bde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f984d20e740 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 0x560fbd7bde70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x560fbd7b5ec0 tx=0x7f98380097a0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f984d20e740 1 -- 172.21.15.16:0/3431648322 shutdown_connections 2024-09-19T09:29:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f984d20e740 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x560fbd7bff00 0x560fbd7c2320 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f984d20e740 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fbd73b2e0 0x560fbd7b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f98237fe640 0 client.15996 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f984d20e740 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560fbd740370 0x560fbd7bde70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f984d20e740 1 --2- 172.21.15.16:0/3431648322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fbd73a9a0 0x560fbd7b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f984d20e740 1 -- 172.21.15.16:0/3431648322 >> 172.21.15.16:0/3431648322 conn(0x560fbd736010 msgr2=0x560fbd7349a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f984d20e740 1 -- 172.21.15.16:0/3431648322 shutdown_connections 2024-09-19T09:29:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f984d20e740 1 -- 172.21.15.16:0/3431648322 wait complete. 2024-09-19T09:29:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.558+0000 7f984d20e740 20 client.15996 trim_cache size 0 max 16384 2024-09-19T09:29:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-19T09:29:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-19T09:29:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-19T09:29:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-19T09:29:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.559+0000 7f984a480640 1 -- 172.21.15.16:0/3214495631 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f982c0a32f0 msgr2=0x7f982c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.559+0000 7f984a480640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f982c0a32f0 0x7f982c0a57b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f983c00ccb0 tx=0x7f983c091a50 comp rx=0 tx=0).stop 2024-09-19T09:29:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.559+0000 7f984a480640 1 -- 172.21.15.16:0/3214495631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9844103850 msgr2=0x7f984419f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.559+0000 7f984a480640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9844103850 0x7f984419f320 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f983c00cce0 tx=0x7f983c00a620 comp rx=0 tx=0).stop 2024-09-19T09:29:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.559+0000 7f984a480640 1 -- 172.21.15.16:0/3214495631 shutdown_connections 2024-09-19T09:29:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.559+0000 7f984a480640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9844103850 0x7f984419f320 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.559+0000 7f984a480640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9844102e90 0x7f98441982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.559+0000 7f984a480640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f982c0a32f0 0x7f982c0a57b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.559+0000 7f984a480640 1 --2- 172.21.15.16:0/3214495631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98440ffe20 0x7f9844197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.559+0000 7f984a480640 1 -- 172.21.15.16:0/3214495631 >> 172.21.15.16:0/3214495631 conn(0x7f98440fb980 msgr2=0x7f9844106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.559+0000 7f984a480640 1 -- 172.21.15.16:0/3214495631 shutdown_connections 2024-09-19T09:29:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:24.559+0000 7f984a480640 1 -- 172.21.15.16:0/3214495631 wait complete. 2024-09-19T09:29:25.430 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:25 smithi142 ceph-mon[35032]: pgmap v392: 97 pgs: 97 active+clean; 791 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 20 op/s 2024-09-19T09:29:25.438 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:25 smithi097 ceph-mon[34864]: pgmap v392: 97 pgs: 97 active+clean; 791 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 20 op/s 2024-09-19T09:29:25.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:25 smithi016 ceph-mon[30979]: pgmap v392: 97 pgs: 97 active+clean; 791 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 20 op/s 2024-09-19T09:29:26.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:29:26.835 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:27.235 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:27 smithi016 ceph-mon[30979]: pgmap v393: 97 pgs: 97 active+clean; 791 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T09:29:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.318+0000 7f555e2b0640 1 -- 172.21.15.16:0/300611195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5558101480 msgr2=0x7f5558101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.318+0000 7f555e2b0640 1 --2- 172.21.15.16:0/300611195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5558101480 0x7f5558101860 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5548009900 tx=0x7f554802a410 comp rx=0 tx=0).stop 2024-09-19T09:29:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.318+0000 7f555e2b0640 1 -- 172.21.15.16:0/300611195 shutdown_connections 2024-09-19T09:29:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.318+0000 7f555e2b0640 1 --2- 172.21.15.16:0/300611195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5558105e80 0x7f555810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.318+0000 7f555e2b0640 1 --2- 172.21.15.16:0/300611195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5558101e30 0x7f5558105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.318+0000 7f555e2b0640 1 --2- 172.21.15.16:0/300611195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5558101480 0x7f5558101860 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.318+0000 7f555e2b0640 1 -- 172.21.15.16:0/300611195 >> 172.21.15.16:0/300611195 conn(0x7f5558077210 msgr2=0x7f55580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.318+0000 7f555e2b0640 1 -- 172.21.15.16:0/300611195 shutdown_connections 2024-09-19T09:29:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.318+0000 7f555e2b0640 1 -- 172.21.15.16:0/300611195 wait complete. 2024-09-19T09:29:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.319+0000 7f555e2b0640 1 Processor -- start 2024-09-19T09:29:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.319+0000 7f555e2b0640 1 -- start start 2024-09-19T09:29:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.319+0000 7f555e2b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5558101480 0x7f5558198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.319+0000 7f555e2b0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5558101e30 0x7f5558198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.319+0000 7f555e2b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5558105e80 0x7f555819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.319+0000 7f555e2b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5558110fc0 con 0x7f5558105e80 2024-09-19T09:29:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.319+0000 7f555e2b0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5558110e40 con 0x7f5558101480 2024-09-19T09:29:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.319+0000 7f555e2b0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5558111140 con 0x7f5558101e30 2024-09-19T09:29:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.319+0000 7f555c84d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5558105e80 0x7f555819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.319+0000 7f555c84d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5558105e80 0x7f555819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52696/0 (socket says 172.21.15.16:52696) 2024-09-19T09:29:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.319+0000 7f555c84d640 1 -- 172.21.15.16:0/686715952 learned_addr learned my addr 172.21.15.16:0/686715952 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f5557fff640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5558101480 0x7f5558198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f55577fe640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5558101e30 0x7f5558198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f555c84d640 1 -- 172.21.15.16:0/686715952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5558101e30 msgr2=0x7f5558198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f555c84d640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5558101e30 0x7f5558198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f555c84d640 1 -- 172.21.15.16:0/686715952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5558101480 msgr2=0x7f5558198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f555c84d640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5558101480 0x7f5558198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f555c84d640 1 -- 172.21.15.16:0/686715952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555819d510 con 0x7f5558105e80 2024-09-19T09:29:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f55577fe640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5558101e30 0x7f5558198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f555c84d640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5558105e80 0x7f555819ce30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f555000b970 tx=0x7f555000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f55557fa640 1 -- 172.21.15.16:0/686715952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550019070 con 0x7f5558105e80 2024-09-19T09:29:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f55557fa640 1 -- 172.21.15.16:0/686715952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555000cb40 con 0x7f5558105e80 2024-09-19T09:29:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f555e2b0640 1 -- 172.21.15.16:0/686715952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555819d800 con 0x7f5558105e80 2024-09-19T09:29:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f55557fa640 1 -- 172.21.15.16:0/686715952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550007650 con 0x7f5558105e80 2024-09-19T09:29:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.320+0000 7f555e2b0640 1 -- 172.21.15.16:0/686715952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55581a50e0 con 0x7f5558105e80 2024-09-19T09:29:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.321+0000 7f555e2b0640 1 -- 172.21.15.16:0/686715952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5558101860 con 0x7f5558105e80 2024-09-19T09:29:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.322+0000 7f55557fa640 1 -- 172.21.15.16:0/686715952 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f555001c070 con 0x7f5558105e80 2024-09-19T09:29:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.323+0000 7f55557fa640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f55400a3240 0x7f55400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.323+0000 7f5557fff640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f55400a3240 0x7f55400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.323+0000 7f55557fa640 1 -- 172.21.15.16:0/686715952 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f55500cefb0 con 0x7f5558105e80 2024-09-19T09:29:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.324+0000 7f5557fff640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f55400a3240 0x7f55400a5700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f55480096c0 tx=0x7f5548031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.328+0000 7f55557fa640 1 -- 172.21.15.16:0/686715952 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f555009bcd0 con 0x7f5558105e80 2024-09-19T09:29:27.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.540+0000 7f555e2b0640 1 -- 172.21.15.16:0/686715952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f555802d7c0 con 0x7f5558105e80 2024-09-19T09:29:27.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.541+0000 7f55557fa640 1 -- 172.21.15.16:0/686715952 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f555009b5d0 con 0x7f5558105e80 2024-09-19T09:29:27.542 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:29:27.542 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:29:27.543 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:29:27.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.542+0000 7f555e2b0640 1 -- 172.21.15.16:0/686715952 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f55400a3240 msgr2=0x7f55400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:27.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.542+0000 7f555e2b0640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f55400a3240 0x7f55400a5700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f55480096c0 tx=0x7f5548031490 comp rx=0 tx=0).stop 2024-09-19T09:29:27.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.543+0000 7f555e2b0640 1 -- 172.21.15.16:0/686715952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5558105e80 msgr2=0x7f555819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:27.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.543+0000 7f555e2b0640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5558105e80 0x7f555819ce30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f555000b970 tx=0x7f555000be30 comp rx=0 tx=0).stop 2024-09-19T09:29:27.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.543+0000 7f555e2b0640 1 -- 172.21.15.16:0/686715952 shutdown_connections 2024-09-19T09:29:27.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.543+0000 7f555e2b0640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f55400a3240 0x7f55400a5700 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:27.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.543+0000 7f555e2b0640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5558101e30 0x7f5558198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:27.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.543+0000 7f555e2b0640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5558105e80 0x7f555819ce30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:27.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.543+0000 7f555e2b0640 1 --2- 172.21.15.16:0/686715952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5558101480 0x7f5558198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:27.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.543+0000 7f555e2b0640 1 -- 172.21.15.16:0/686715952 >> 172.21.15.16:0/686715952 conn(0x7f5558077210 msgr2=0x7f5558100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:27.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.543+0000 7f555e2b0640 1 -- 172.21.15.16:0/686715952 shutdown_connections 2024-09-19T09:29:27.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:27.543+0000 7f555e2b0640 1 -- 172.21.15.16:0/686715952 wait complete. 2024-09-19T09:29:27.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:27 smithi142 ceph-mon[35032]: pgmap v393: 97 pgs: 97 active+clean; 791 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T09:29:27.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:27 smithi097 ceph-mon[34864]: pgmap v393: 97 pgs: 97 active+clean; 791 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-19T09:29:28.214 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 7b034bee --release --await' 2024-09-19T09:29:28.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 7b034bee --release --await 2024-09-19T09:29:28.238 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:28 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/686715952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:28.475 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:28.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:28 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/686715952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:28.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:28 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/686715952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:28.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.956+0000 7f4ab6deb640 1 -- 172.21.15.16:0/2832883160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab0103660 msgr2=0x7f4ab010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:28.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.956+0000 7f4ab6deb640 1 --2- 172.21.15.16:0/2832883160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab0103660 0x7f4ab010bb50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009900 tx=0x7f4a9802a480 comp rx=0 tx=0).stop 2024-09-19T09:29:28.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.956+0000 7f4ab6deb640 1 -- 172.21.15.16:0/2832883160 shutdown_connections 2024-09-19T09:29:28.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.956+0000 7f4ab6deb640 1 --2- 172.21.15.16:0/2832883160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ab010c170 0x7f4ab010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.956+0000 7f4ab6deb640 1 --2- 172.21.15.16:0/2832883160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab0103660 0x7f4ab010bb50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.956+0000 7f4ab6deb640 1 --2- 172.21.15.16:0/2832883160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab0102d40 0x7f4ab0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.956+0000 7f4ab6deb640 1 -- 172.21.15.16:0/2832883160 >> 172.21.15.16:0/2832883160 conn(0x7f4ab00fc7a0 msgr2=0x7f4ab00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.956+0000 7f4ab6deb640 1 -- 172.21.15.16:0/2832883160 shutdown_connections 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.956+0000 7f4ab6deb640 1 -- 172.21.15.16:0/2832883160 wait complete. 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.956+0000 7f4ab6deb640 1 Processor -- start 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab6deb640 1 -- start start 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab6deb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ab0102d40 0x7f4ab01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab6deb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab0103660 0x7f4ab0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab6deb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab010c170 0x7f4ab0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab6deb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0111f70 con 0x7f4ab010c170 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab6deb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0111df0 con 0x7f4ab0103660 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab6deb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01120f0 con 0x7f4ab0102d40 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4aaffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab0103660 0x7f4ab0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab5388640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab010c170 0x7f4ab0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4aaffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab0103660 0x7f4ab0100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:57396/0 (socket says 172.21.15.16:57396) 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4aaffff640 1 -- 172.21.15.16:0/2832642468 learned_addr learned my addr 172.21.15.16:0/2832642468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab4b87640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ab0102d40 0x7f4ab01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab5388640 1 -- 172.21.15.16:0/2832642468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ab0102d40 msgr2=0x7f4ab01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab5388640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ab0102d40 0x7f4ab01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab5388640 1 -- 172.21.15.16:0/2832642468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab0103660 msgr2=0x7f4ab0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab5388640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab0103660 0x7f4ab0100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4ab5388640 1 -- 172.21.15.16:0/2832642468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab01a6930 con 0x7f4ab010c170 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.957+0000 7f4aaffff640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab0103660 0x7f4ab0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.958+0000 7f4ab5388640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab010c170 0x7f4ab0101420 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800e940 tx=0x7f4aa800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.958+0000 7f4aadffb640 1 -- 172.21.15.16:0/2832642468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8004d20 con 0x7f4ab010c170 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.958+0000 7f4ab6deb640 1 -- 172.21.15.16:0/2832642468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab01a6c20 con 0x7f4ab010c170 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.958+0000 7f4aadffb640 1 -- 172.21.15.16:0/2832642468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa8005020 con 0x7f4ab010c170 2024-09-19T09:29:28.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.958+0000 7f4aadffb640 1 -- 172.21.15.16:0/2832642468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80116d0 con 0x7f4ab010c170 2024-09-19T09:29:28.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.958+0000 7f4ab6deb640 1 -- 172.21.15.16:0/2832642468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab01a7160 con 0x7f4ab010c170 2024-09-19T09:29:28.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.960+0000 7f4aadffb640 1 -- 172.21.15.16:0/2832642468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4aa8011930 con 0x7f4ab010c170 2024-09-19T09:29:28.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.960+0000 7f4ab91c4740 1 Processor -- start 2024-09-19T09:29:28.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab91c4740 1 -- start start 2024-09-19T09:29:28.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab91c4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 0x563d97ac3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab91c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 0x563d97acdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab91c4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ace240 0x563d97ad0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab91c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d97a28620 con 0x563d97ac3980 2024-09-19T09:29:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab91c4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563d979cfe50 con 0x563d97ace240 2024-09-19T09:29:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab91c4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563d97a245e0 con 0x563d97ac3060 2024-09-19T09:29:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab4b87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 0x563d97acdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab4b87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 0x563d97acdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52730/0 (socket says 172.21.15.16:52730) 2024-09-19T09:29:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab4b87640 1 -- 172.21.15.16:0/1487041855 learned_addr learned my addr 172.21.15.16:0/1487041855 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4aadffb640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4a9c0a3240 0x7f4a9c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4aaffff640 1 --2- 172.21.15.16:0/1487041855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 0x563d97ac3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4aaffff640 1 --2- 172.21.15.16:0/1487041855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ace240 0x563d97ad0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab5388640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4a9c0a3240 0x7f4a9c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab4b87640 1 -- 172.21.15.16:0/1487041855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 msgr2=0x563d97ac3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:28.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab4b87640 1 --2- 172.21.15.16:0/1487041855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 0x563d97ac3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab4b87640 1 -- 172.21.15.16:0/1487041855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ace240 msgr2=0x563d97ad0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab4b87640 1 --2- 172.21.15.16:0/1487041855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ace240 0x563d97ad0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4ab4b87640 1 -- 172.21.15.16:0/1487041855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d979c0d40 con 0x563d97ac3980 2024-09-19T09:29:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.961+0000 7f4aadffb640 1 -- 172.21.15.16:0/2832642468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4aa80cfc30 con 0x7f4ab010c170 2024-09-19T09:29:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.962+0000 7f4ab4b87640 1 --2- 172.21.15.16:0/1487041855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 0x563d97acdd00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000bd10 tx=0x7f4aa00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d297833867f688a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.962+0000 7f4ab5388640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4a9c0a3240 0x7f4a9c0a5700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4aa809c0e0 tx=0x7f4aa8068240 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.962+0000 7f4aa6ffd640 1 -- 172.21.15.16:0/1487041855 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0004030 con 0x563d97ac3980 2024-09-19T09:29:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.962+0000 7f4aa6ffd640 1 -- 172.21.15.16:0/1487041855 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa00041d0 con 0x563d97ac3980 2024-09-19T09:29:28.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.962+0000 7f4aa6ffd640 1 -- 172.21.15.16:0/1487041855 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0011730 con 0x563d97ac3980 2024-09-19T09:29:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.963+0000 7f4ab91c4740 1 -- 172.21.15.16:0/1487041855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 msgr2=0x563d97acdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.963+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/1487041855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 0x563d97acdd00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000bd10 tx=0x7f4aa00107a0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.963+0000 7f4ab91c4740 1 -- 172.21.15.16:0/1487041855 shutdown_connections 2024-09-19T09:29:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.963+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/1487041855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ace240 0x563d97ad0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.963+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/1487041855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 0x563d97acdd00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.963+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/1487041855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 0x563d97ac3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.963+0000 7f4ab91c4740 1 -- 172.21.15.16:0/1487041855 >> 172.21.15.16:0/1487041855 conn(0x563d97abf5f0 msgr2=0x563d97abf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:28.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.963+0000 7f4ab91c4740 1 -- 172.21.15.16:0/1487041855 shutdown_connections 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.963+0000 7f4ab91c4740 1 -- 172.21.15.16:0/1487041855 wait complete. 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.963+0000 7f4ab91c4740 1 -- start start 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.963+0000 7f4ab91c4740 1 Processor -- start 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4ab91c4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 0x563d97acb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4ab91c4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ac3980 0x563d97acb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4ab91c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ace240 0x563d97acbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4ab91c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d97ad0e10 con 0x563d97ace240 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4ab91c4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563d979b0f10 con 0x563d97ac3980 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4ab91c4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563d97ad0f90 con 0x563d97ac3060 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4ab91c4740 10 client.0 _ll_register_callbacks cb 0x563d979d4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4aaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ace240 0x563d97acbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4aaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ace240 0x563d97acbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52742/0 (socket says 172.21.15.16:52742) 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4aaffff640 1 -- 172.21.15.16:0/974021793 learned_addr learned my addr 172.21.15.16:0/974021793 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:28.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4ab4b87640 1 --2- 172.21.15.16:0/974021793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ac3980 0x563d97acb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4aaffff640 1 --2- 172.21.15.16:0/974021793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 0x563d97acb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4aaffff640 1 -- 172.21.15.16:0/974021793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 msgr2=0x563d97acb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4aaffff640 1 --2- 172.21.15.16:0/974021793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 0x563d97acb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4aaffff640 1 -- 172.21.15.16:0/974021793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ac3980 msgr2=0x563d97acb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4aaffff640 1 --2- 172.21.15.16:0/974021793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ac3980 0x563d97acb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4aaffff640 1 -- 172.21.15.16:0/974021793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d97737580 con 0x563d97ace240 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4aaffff640 1 --2- 172.21.15.16:0/974021793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ace240 0x563d97acbb60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009a20 tx=0x7f4a98005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3e445b44fcf36d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.964+0000 7f4a8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4a8ffff640 1 -- 172.21.15.16:0/974021793 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98035040 con 0x563d97ace240 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4a8ffff640 1 -- 172.21.15.16:0/974021793 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4a980055f0 con 0x563d97ace240 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4ab91c4740 1 -- 172.21.15.16:0/974021793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa00041d0 con 0x563d97ace240 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4a8ffff640 1 -- 172.21.15.16:0/974021793 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a98054050 con 0x563d97ace240 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4ab91c4740 10 client.16014 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:28.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4ab91c4740 1 -- 172.21.15.16:0/974021793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563d97801960 con 0x563d97ace240 2024-09-19T09:29:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4a8ffff640 1 -- 172.21.15.16:0/974021793 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa00041d0 con 0x563d97ace240 2024-09-19T09:29:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4a8ffff640 1 -- 172.21.15.16:0/974021793 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a9804b070 con 0x563d97ace240 2024-09-19T09:29:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4ab91c4740 10 client.16014 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4ab91c4740 10 client.16014 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:29:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4ab91c4740 20 client.16014 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/974021793 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563d97addcb0 0x563d97ae00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4ab91c4740 4 client.16014 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.965+0000 7f4ab91c4740 1 -- 172.21.15.16:0/974021793 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d97a35b90 con 0x563d97addcb0 2024-09-19T09:29:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.966+0000 7f4ab4b87640 1 --2- 172.21.15.16:0/974021793 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563d97addcb0 0x563d97ae00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.966+0000 7f4ab4b87640 1 --2- 172.21.15.16:0/974021793 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563d97addcb0 0x563d97ae00e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcdbf7556729b240 server_cookie=a3313a68dd21104a in_seq=0 out_seq=0 2024-09-19T09:29:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.966+0000 7f4a8ffff640 10 client.16014 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4a8ffff640 1 -- 172.21.15.16:0/974021793 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d97a35b90 con 0x563d97addcb0 2024-09-19T09:29:28.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4a8ffff640 10 client.16014 handle_command_reply: tid=0 2024-09-19T09:29:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4ab91c4740 1 client.16014 shutdown 2024-09-19T09:29:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4ab91c4740 1 -- 172.21.15.16:0/974021793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ace240 msgr2=0x563d97acbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/974021793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ace240 0x563d97acbb60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009a20 tx=0x7f4a98005d50 comp rx=0 tx=0).stop 2024-09-19T09:29:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4ab91c4740 1 -- 172.21.15.16:0/974021793 shutdown_connections 2024-09-19T09:29:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/974021793 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563d97addcb0 0x563d97ae00e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/974021793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ace240 0x563d97acbb60 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/974021793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ac3980 0x563d97acb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4a8ffff640 0 client.16014 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/974021793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 0x563d97acb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4ab91c4740 1 -- 172.21.15.16:0/974021793 >> 172.21.15.16:0/974021793 conn(0x563d97abf5f0 msgr2=0x563d97ac52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4ab91c4740 1 -- 172.21.15.16:0/974021793 shutdown_connections 2024-09-19T09:29:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4ab91c4740 1 -- 172.21.15.16:0/974021793 wait complete. 2024-09-19T09:29:28.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.967+0000 7f4ab91c4740 20 client.16014 trim_cache size 0 max 16384 2024-09-19T09:29:28.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.984+0000 7f4ab91c4740 1 Processor -- start 2024-09-19T09:29:28.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.984+0000 7f4ab91c4740 1 -- start start 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.984+0000 7f4ab91c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3060 0x563d97ac5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.984+0000 7f4ab91c4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ac3980 0x563d97abe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.984+0000 7f4ab91c4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ace240 0x563d97ae59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.984+0000 7f4ab91c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01af890 con 0x563d97ac3060 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.984+0000 7f4ab91c4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01b13d0 con 0x563d97ac3980 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.984+0000 7f4ab91c4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01aff60 con 0x563d97ace240 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.984+0000 7f4aaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3060 0x563d97ac5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.984+0000 7f4aaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3060 0x563d97ac5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52756/0 (socket says 172.21.15.16:52756) 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.984+0000 7f4aaffff640 1 -- 172.21.15.16:0/362271344 learned_addr learned my addr 172.21.15.16:0/362271344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.985+0000 7f4ab4b87640 1 --2- 172.21.15.16:0/362271344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ac3980 0x563d97abe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.985+0000 7f4aaffff640 1 -- 172.21.15.16:0/362271344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ace240 msgr2=0x563d97ae59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.985+0000 7f4aaffff640 1 --2- 172.21.15.16:0/362271344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ace240 0x563d97ae59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.985+0000 7f4aaffff640 1 -- 172.21.15.16:0/362271344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ac3980 msgr2=0x563d97abe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.985+0000 7f4aaffff640 1 --2- 172.21.15.16:0/362271344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ac3980 0x563d97abe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.985+0000 7f4aaffff640 1 -- 172.21.15.16:0/362271344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d97ae0760 con 0x563d97ac3060 2024-09-19T09:29:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.985+0000 7f4aaffff640 1 --2- 172.21.15.16:0/362271344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3060 0x563d97ac5ac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4a98006010 tx=0x7f4a98005110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c759ae9c8469f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.985+0000 7f4aa67fc640 1 -- 172.21.15.16:0/362271344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98035070 con 0x563d97ac3060 2024-09-19T09:29:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.985+0000 7f4aa67fc640 1 -- 172.21.15.16:0/362271344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a98004460 con 0x563d97ac3060 2024-09-19T09:29:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.985+0000 7f4aa67fc640 1 -- 172.21.15.16:0/362271344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98004ea0 con 0x563d97ac3060 2024-09-19T09:29:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.986+0000 7f4ab91c4740 1 -- 172.21.15.16:0/362271344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3060 msgr2=0x563d97ac5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.986+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/362271344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3060 0x563d97ac5ac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4a98006010 tx=0x7f4a98005110 comp rx=0 tx=0).stop 2024-09-19T09:29:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.986+0000 7f4ab91c4740 1 -- 172.21.15.16:0/362271344 shutdown_connections 2024-09-19T09:29:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.986+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/362271344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ace240 0x563d97ae59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.986+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/362271344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ac3980 0x563d97abe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.986+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/362271344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3060 0x563d97ac5ac0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.986+0000 7f4ab91c4740 1 -- 172.21.15.16:0/362271344 >> 172.21.15.16:0/362271344 conn(0x563d97abf5f0 msgr2=0x563d97abdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:28.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.986+0000 7f4ab91c4740 1 -- 172.21.15.16:0/362271344 shutdown_connections 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.986+0000 7f4ab91c4740 1 -- 172.21.15.16:0/362271344 wait complete. 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.986+0000 7f4ab91c4740 1 -- start start 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.986+0000 7f4ab91c4740 1 Processor -- start 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab91c4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 0x563d97ad2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab91c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 0x563d97ad2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab91c4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ace240 0x563d97b47980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab91c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01b0d00 con 0x563d97ac3980 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab91c4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01b0630 con 0x563d97ace240 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab91c4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01ad850 con 0x563d97ac3060 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab91c4740 10 client.0 _ll_register_callbacks cb 0x7f4a98005250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab4b87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 0x563d97ad2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab4b87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 0x563d97ad2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52768/0 (socket says 172.21.15.16:52768) 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab4b87640 1 -- 172.21.15.16:0/153578982 learned_addr learned my addr 172.21.15.16:0/153578982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:28.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4aaffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 0x563d97ad2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4aaffff640 1 --2- 172.21.15.16:0/153578982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ace240 0x563d97b47980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab4b87640 1 -- 172.21.15.16:0/153578982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 msgr2=0x563d97ad2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab4b87640 1 --2- 172.21.15.16:0/153578982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 0x563d97ad2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab4b87640 1 -- 172.21.15.16:0/153578982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ace240 msgr2=0x563d97b47980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab4b87640 1 --2- 172.21.15.16:0/153578982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ace240 0x563d97b47980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab4b87640 1 -- 172.21.15.16:0/153578982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d97b47ec0 con 0x563d97ac3980 2024-09-19T09:29:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4ab4b87640 1 --2- 172.21.15.16:0/153578982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 0x563d97ad2890 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000bd10 tx=0x7f4aa000b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe751e52fc631423 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.987+0000 7f4a8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4a8ffff640 1 -- 172.21.15.16:0/153578982 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0035060 con 0x563d97ac3980 2024-09-19T09:29:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4ab91c4740 1 -- 172.21.15.16:0/153578982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a98004460 con 0x563d97ac3980 2024-09-19T09:29:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4a8ffff640 1 -- 172.21.15.16:0/153578982 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4aa0005450 con 0x563d97ac3980 2024-09-19T09:29:28.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4a8ffff640 1 -- 172.21.15.16:0/153578982 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4aa0005650 con 0x563d97ac3980 2024-09-19T09:29:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4ab91c4740 10 client.16020 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4ab91c4740 1 -- 172.21.15.16:0/153578982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563d97b490f0 con 0x563d97ac3980 2024-09-19T09:29:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4a8ffff640 1 -- 172.21.15.16:0/153578982 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a98004460 con 0x563d97ac3980 2024-09-19T09:29:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4a8ffff640 1 -- 172.21.15.16:0/153578982 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4aa003a070 con 0x563d97ac3980 2024-09-19T09:29:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4ab91c4740 10 client.16020 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4ab91c4740 10 client.16020 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:29:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4ab91c4740 20 client.16020 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/153578982 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563d97b49590 0x563d97b4b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4ab91c4740 4 client.16020 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7b034bee", "release": true, "await": true}] 2024-09-19T09:29:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.988+0000 7f4ab91c4740 1 -- 172.21.15.16:0/153578982 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7b034bee", "release": true, "await": true}) -- 0x563d97aba950 con 0x563d97b49590 2024-09-19T09:29:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.989+0000 7f4aaffff640 1 --2- 172.21.15.16:0/153578982 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563d97b49590 0x563d97b4b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.989+0000 7f4a8ffff640 10 client.16020 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:28.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.989+0000 7f4aaffff640 1 --2- 172.21.15.16:0/153578982 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563d97b49590 0x563d97b4b980 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6699ee3c6d69847a server_cookie=27794470771405cf in_seq=0 out_seq=0 2024-09-19T09:29:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4a8ffff640 1 -- 172.21.15.16:0/153578982 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563d97aba950 con 0x563d97b49590 2024-09-19T09:29:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4a8ffff640 10 client.16020 handle_command_reply: tid=0 2024-09-19T09:29:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4ab91c4740 1 client.16020 shutdown 2024-09-19T09:29:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4ab91c4740 1 -- 172.21.15.16:0/153578982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 msgr2=0x563d97ad2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/153578982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 0x563d97ad2890 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000bd10 tx=0x7f4aa000b770 comp rx=0 tx=0).stop 2024-09-19T09:29:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4ab91c4740 1 -- 172.21.15.16:0/153578982 shutdown_connections 2024-09-19T09:29:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/153578982 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x563d97b49590 0x563d97b4b980 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/153578982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563d97ace240 0x563d97b47980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/153578982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d97ac3980 0x563d97ad2890 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4ab91c4740 1 --2- 172.21.15.16:0/153578982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d97ac3060 0x563d97ad2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4ab91c4740 1 -- 172.21.15.16:0/153578982 >> 172.21.15.16:0/153578982 conn(0x563d97abf5f0 msgr2=0x563d97acb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:28.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4a8ffff640 0 client.16020 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:28.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4ab91c4740 1 -- 172.21.15.16:0/153578982 shutdown_connections 2024-09-19T09:29:28.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4ab91c4740 1 -- 172.21.15.16:0/153578982 wait complete. 2024-09-19T09:29:28.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.992+0000 7f4ab91c4740 20 client.16020 trim_cache size 0 max 16384 2024-09-19T09:29:28.997 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:29:28.997 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-19T09:29:28.997 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24500, 2024-09-19T09:29:28.997 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 12, 2024-09-19T09:29:28.997 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-19T09:29:28.997 INFO:teuthology.orchestra.run.smithi016.stdout: "7b034bee": { 2024-09-19T09:29:28.997 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 12, 2024-09-19T09:29:28.998 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-19T09:29:28.998 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-19T09:29:28.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-19T09:29:28.998 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-19T09:29:28.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:28.998 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-19T09:29:28.998 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.3, 2024-09-19T09:29:28.998 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-19T09:29:28.998 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-19T09:29:28.998 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-19T09:29:28.998 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-19T09:29:28.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-19T09:29:28.998 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-19T09:29:28.999 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:28.999 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:28.999 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:28.999 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:28.999 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:28.999 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:29:29.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.993+0000 7f4ab6deb640 1 -- 172.21.15.16:0/2832642468 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4a9c0a3240 msgr2=0x7f4a9c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:29.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.993+0000 7f4ab6deb640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4a9c0a3240 0x7f4a9c0a5700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4aa809c0e0 tx=0x7f4aa8068240 comp rx=0 tx=0).stop 2024-09-19T09:29:29.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.993+0000 7f4ab6deb640 1 -- 172.21.15.16:0/2832642468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab010c170 msgr2=0x7f4ab0101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:29.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.993+0000 7f4ab6deb640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab010c170 0x7f4ab0101420 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800e940 tx=0x7f4aa800ee00 comp rx=0 tx=0).stop 2024-09-19T09:29:29.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.993+0000 7f4ab6deb640 1 -- 172.21.15.16:0/2832642468 shutdown_connections 2024-09-19T09:29:29.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.993+0000 7f4ab6deb640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab010c170 0x7f4ab0101420 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:29.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.993+0000 7f4ab6deb640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ab0103660 0x7f4ab0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:29.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.993+0000 7f4ab6deb640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4a9c0a3240 0x7f4a9c0a5700 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:29.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.993+0000 7f4ab6deb640 1 --2- 172.21.15.16:0/2832642468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ab0102d40 0x7f4ab01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:29.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.993+0000 7f4ab6deb640 1 -- 172.21.15.16:0/2832642468 >> 172.21.15.16:0/2832642468 conn(0x7f4ab00fc7a0 msgr2=0x7f4ab00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:29.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.993+0000 7f4ab6deb640 1 -- 172.21.15.16:0/2832642468 shutdown_connections 2024-09-19T09:29:29.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:28.993+0000 7f4ab6deb640 1 -- 172.21.15.16:0/2832642468 wait complete. 2024-09-19T09:29:29.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:29 smithi016 ceph-mon[30979]: pgmap v394: 97 pgs: 97 active+clean; 796 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-09-19T09:29:29.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:29 smithi142 ceph-mon[35032]: pgmap v394: 97 pgs: 97 active+clean; 796 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-09-19T09:29:29.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:29 smithi097 ceph-mon[34864]: pgmap v394: 97 pgs: 97 active+clean; 796 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-09-19T09:29:29.709 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7b034bee', total seconds elapsed: 23.5 2024-09-19T09:29:30.454 INFO:tasks.workunit.client.0.smithi016.stdout:writing pattern 2024-09-19T09:29:30.455 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-09-19T09:29:30.455 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-09-19T09:29:30.455 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-09-19T09:29:30.455 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-09-19T09:29:30.455 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-09-19T09:29:30.455 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-09-19T09:29:30.455 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-09-19T09:29:30.455 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-09-19T09:29:30.455 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-09-19T09:29:30.455 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-09-19T09:29:30.456 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-09-19T09:29:30.456 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-09-19T09:29:30.456 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-09-19T09:29:30.456 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-09-19T09:29:30.456 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-09-19T09:29:30.456 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-09-19T09:29:30.456 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-09-19T09:29:30.456 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-09-19T09:29:30.456 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-09-19T09:29:30.456 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-09-19T09:29:30.457 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-09-19T09:29:30.457 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-09-19T09:29:30.457 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-09-19T09:29:30.457 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-09-19T09:29:30.457 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-09-19T09:29:30.457 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-09-19T09:29:30.457 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-09-19T09:29:30.457 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-09-19T09:29:30.457 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-09-19T09:29:30.457 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-09-19T09:29:30.457 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-09-19T09:29:30.458 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-09-19T09:29:30.458 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-09-19T09:29:30.458 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-09-19T09:29:30.458 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-09-19T09:29:30.458 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-09-19T09:29:30.458 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-09-19T09:29:30.458 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-09-19T09:29:30.458 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-09-19T09:29:30.458 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-09-19T09:29:30.458 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-09-19T09:29:30.459 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-09-19T09:29:30.459 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-09-19T09:29:30.459 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-09-19T09:29:30.459 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-09-19T09:29:30.459 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-09-19T09:29:30.459 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-09-19T09:29:30.459 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-09-19T09:29:30.459 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-09-19T09:29:30.459 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-09-19T09:29:30.459 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-09-19T09:29:30.459 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-09-19T09:29:30.460 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-09-19T09:29:30.460 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-09-19T09:29:30.460 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-09-19T09:29:30.460 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-09-19T09:29:30.460 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-09-19T09:29:30.460 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-09-19T09:29:30.460 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-09-19T09:29:30.460 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-09-19T09:29:30.460 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-09-19T09:29:30.461 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-09-19T09:29:30.461 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-09-19T09:29:30.461 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-09-19T09:29:30.461 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-09-19T09:29:30.461 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-09-19T09:29:30.461 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-09-19T09:29:30.461 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-09-19T09:29:30.461 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-09-19T09:29:30.461 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-09-19T09:29:30.462 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-09-19T09:29:30.462 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-09-19T09:29:30.462 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-09-19T09:29:30.462 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-09-19T09:29:30.462 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-09-19T09:29:30.462 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-09-19T09:29:30.462 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-09-19T09:29:30.462 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-09-19T09:29:30.462 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-09-19T09:29:30.462 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-09-19T09:29:30.462 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-09-19T09:29:30.462 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-09-19T09:29:30.462 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-09-19T09:29:30.463 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-09-19T09:29:30.463 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-09-19T09:29:30.463 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-09-19T09:29:30.463 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-09-19T09:29:30.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-09-19T09:29:30.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-09-19T09:29:30.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-09-19T09:29:30.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-09-19T09:29:30.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-09-19T09:29:30.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-09-19T09:29:30.476 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-09-19T09:29:30.476 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-09-19T09:29:30.476 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-09-19T09:29:30.476 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-09-19T09:29:30.476 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-09-19T09:29:30.476 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-09-19T09:29:30.476 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-09-19T09:29:30.476 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-09-19T09:29:30.476 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-09-19T09:29:30.476 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-09-19T09:29:30.476 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-09-19T09:29:30.476 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-09-19T09:29:30.477 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-09-19T09:29:30.477 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-09-19T09:29:30.477 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-09-19T09:29:30.477 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-09-19T09:29:30.477 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-09-19T09:29:30.477 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-09-19T09:29:30.477 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-09-19T09:29:30.477 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-09-19T09:29:30.477 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-09-19T09:29:30.477 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-09-19T09:29:30.478 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-09-19T09:29:30.478 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-09-19T09:29:30.478 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-09-19T09:29:30.478 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-09-19T09:29:30.478 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-09-19T09:29:30.478 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-09-19T09:29:30.478 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-09-19T09:29:30.478 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-09-19T09:29:30.478 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-09-19T09:29:30.478 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-09-19T09:29:30.478 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-09-19T09:29:30.478 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-09-19T09:29:30.478 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-09-19T09:29:30.479 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-09-19T09:29:30.479 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-09-19T09:29:30.479 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-09-19T09:29:30.479 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-09-19T09:29:30.479 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-09-19T09:29:30.479 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-09-19T09:29:30.479 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-09-19T09:29:30.479 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-09-19T09:29:30.479 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-09-19T09:29:30.479 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-09-19T09:29:30.479 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-09-19T09:29:30.480 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-09-19T09:29:30.480 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-09-19T09:29:30.480 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-09-19T09:29:30.480 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-09-19T09:29:30.480 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-09-19T09:29:30.480 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-09-19T09:29:30.480 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-09-19T09:29:30.480 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-09-19T09:29:30.480 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-09-19T09:29:30.480 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-09-19T09:29:30.480 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-09-19T09:29:30.480 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-09-19T09:29:30.480 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-09-19T09:29:30.481 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-09-19T09:29:30.482 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-09-19T09:29:30.482 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-09-19T09:29:30.482 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-09-19T09:29:30.482 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-09-19T09:29:30.482 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-09-19T09:29:30.482 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-09-19T09:29:30.482 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-09-19T09:29:30.482 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-09-19T09:29:30.482 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-09-19T09:29:30.485 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-09-19T09:29:30.485 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-09-19T09:29:30.485 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-09-19T09:29:30.485 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-09-19T09:29:30.485 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-09-19T09:29:30.485 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-09-19T09:29:30.485 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-09-19T09:29:30.485 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-09-19T09:29:30.486 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-09-19T09:29:30.486 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-09-19T09:29:30.486 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-09-19T09:29:30.486 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-09-19T09:29:30.486 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-09-19T09:29:30.486 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-09-19T09:29:30.486 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-09-19T09:29:30.486 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-09-19T09:29:30.486 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-09-19T09:29:30.486 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-09-19T09:29:30.486 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-09-19T09:29:30.486 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-09-19T09:29:30.486 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-09-19T09:29:30.487 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-09-19T09:29:30.488 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-09-19T09:29:30.489 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-09-19T09:29:30.489 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-09-19T09:29:30.489 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-09-19T09:29:30.489 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-09-19T09:29:30.489 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-09-19T09:29:30.489 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-09-19T09:29:30.489 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-09-19T09:29:30.489 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-09-19T09:29:30.489 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-09-19T09:29:30.489 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-09-19T09:29:30.489 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-09-19T09:29:30.489 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-09-19T09:29:30.490 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-09-19T09:29:30.490 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-09-19T09:29:30.490 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-09-19T09:29:30.490 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-09-19T09:29:30.490 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-09-19T09:29:30.490 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-09-19T09:29:30.490 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-09-19T09:29:30.490 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-09-19T09:29:30.490 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-09-19T09:29:30.490 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-09-19T09:29:30.490 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-09-19T09:29:30.491 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-09-19T09:29:30.491 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-09-19T09:29:30.491 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-09-19T09:29:30.491 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-09-19T09:29:30.491 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-09-19T09:29:30.491 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-09-19T09:29:30.491 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-09-19T09:29:30.491 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-09-19T09:29:30.491 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-09-19T09:29:30.491 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-09-19T09:29:30.491 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-09-19T09:29:30.491 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-09-19T09:29:30.492 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-09-19T09:29:30.492 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-09-19T09:29:30.492 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-09-19T09:29:30.497 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-09-19T09:29:30.497 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-09-19T09:29:30.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-09-19T09:29:30.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-09-19T09:29:30.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-09-19T09:29:30.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-09-19T09:29:30.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-09-19T09:29:30.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-09-19T09:29:30.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-09-19T09:29:30.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-09-19T09:29:30.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-09-19T09:29:30.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-09-19T09:29:30.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-09-19T09:29:30.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-09-19T09:29:30.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-09-19T09:29:30.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-09-19T09:29:30.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-09-19T09:29:30.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-09-19T09:29:30.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-09-19T09:29:30.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-09-19T09:29:30.504 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-09-19T09:29:30.504 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-09-19T09:29:30.522 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-09-19T09:29:30.522 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-09-19T09:29:30.522 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-09-19T09:29:30.522 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-09-19T09:29:30.522 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-09-19T09:29:30.522 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-09-19T09:29:30.523 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-09-19T09:29:30.523 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-09-19T09:29:30.523 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-09-19T09:29:30.523 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-09-19T09:29:30.523 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-09-19T09:29:30.523 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-09-19T09:29:30.523 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-09-19T09:29:30.523 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-09-19T09:29:30.523 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-09-19T09:29:30.523 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-09-19T09:29:30.523 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-09-19T09:29:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-09-19T09:29:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-09-19T09:29:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-09-19T09:29:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-09-19T09:29:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-09-19T09:29:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-09-19T09:29:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-09-19T09:29:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-09-19T09:29:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-09-19T09:29:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-09-19T09:29:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-09-19T09:29:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-09-19T09:29:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-09-19T09:29:30.525 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-09-19T09:29:30.525 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-09-19T09:29:30.525 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-09-19T09:29:30.525 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-09-19T09:29:30.525 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-09-19T09:29:30.525 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-09-19T09:29:30.525 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-09-19T09:29:30.525 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-09-19T09:29:30.525 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-09-19T09:29:30.526 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-09-19T09:29:30.526 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-09-19T09:29:30.526 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-09-19T09:29:30.526 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-09-19T09:29:30.526 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-09-19T09:29:30.526 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-09-19T09:29:30.526 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-09-19T09:29:30.526 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-09-19T09:29:30.526 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-09-19T09:29:30.526 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-09-19T09:29:30.526 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-09-19T09:29:30.526 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-09-19T09:29:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-09-19T09:29:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-09-19T09:29:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-09-19T09:29:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-09-19T09:29:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-09-19T09:29:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-09-19T09:29:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-09-19T09:29:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-09-19T09:29:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-09-19T09:29:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-09-19T09:29:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-09-19T09:29:30.528 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-09-19T09:29:30.528 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-09-19T09:29:30.528 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-09-19T09:29:30.528 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-09-19T09:29:30.528 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-09-19T09:29:30.528 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-09-19T09:29:30.528 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-09-19T09:29:30.528 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-09-19T09:29:30.528 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-09-19T09:29:30.528 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-09-19T09:29:30.529 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-09-19T09:29:30.529 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-09-19T09:29:30.529 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-09-19T09:29:30.529 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-09-19T09:29:30.529 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-09-19T09:29:30.529 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-09-19T09:29:30.529 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-09-19T09:29:30.529 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-09-19T09:29:30.529 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-09-19T09:29:30.529 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-09-19T09:29:30.530 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-09-19T09:29:30.530 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-09-19T09:29:30.542 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-09-19T09:29:30.542 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-09-19T09:29:30.542 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-09-19T09:29:30.542 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-09-19T09:29:30.542 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-09-19T09:29:30.543 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-09-19T09:29:30.543 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-09-19T09:29:30.543 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-09-19T09:29:30.543 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-09-19T09:29:30.543 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-09-19T09:29:30.543 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-09-19T09:29:30.543 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-09-19T09:29:30.543 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-09-19T09:29:30.543 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-09-19T09:29:30.543 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-09-19T09:29:30.543 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-09-19T09:29:30.543 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-09-19T09:29:30.543 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-09-19T09:29:30.544 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-09-19T09:29:30.544 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-09-19T09:29:30.544 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-09-19T09:29:30.544 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-09-19T09:29:30.544 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-09-19T09:29:30.544 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-09-19T09:29:30.544 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-09-19T09:29:30.544 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-09-19T09:29:30.544 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-09-19T09:29:30.544 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-09-19T09:29:30.544 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-09-19T09:29:30.544 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-09-19T09:29:30.544 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-09-19T09:29:30.545 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-09-19T09:29:30.546 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-09-19T09:29:30.547 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-09-19T09:29:30.548 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-09-19T09:29:30.548 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-09-19T09:29:30.548 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-09-19T09:29:30.548 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-09-19T09:29:30.548 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-09-19T09:29:30.548 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-09-19T09:29:30.548 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-09-19T09:29:30.548 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-09-19T09:29:30.548 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-09-19T09:29:30.548 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-09-19T09:29:30.548 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-09-19T09:29:30.552 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-09-19T09:29:30.552 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-09-19T09:29:30.552 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-09-19T09:29:30.552 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-09-19T09:29:30.552 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-09-19T09:29:30.552 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-09-19T09:29:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-09-19T09:29:30.554 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-09-19T09:29:30.554 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-09-19T09:29:30.554 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-09-19T09:29:30.554 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-09-19T09:29:30.554 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-09-19T09:29:30.554 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-09-19T09:29:30.554 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-09-19T09:29:30.554 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-09-19T09:29:30.554 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-09-19T09:29:30.554 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-09-19T09:29:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-09-19T09:29:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-09-19T09:29:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-09-19T09:29:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-09-19T09:29:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-09-19T09:29:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-09-19T09:29:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-09-19T09:29:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-09-19T09:29:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-09-19T09:29:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-09-19T09:29:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-09-19T09:29:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-09-19T09:29:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-09-19T09:29:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-09-19T09:29:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-09-19T09:29:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-09-19T09:29:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-09-19T09:29:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-09-19T09:29:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-09-19T09:29:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-09-19T09:29:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-09-19T09:29:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-09-19T09:29:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-09-19T09:29:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-09-19T09:29:30.557 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-09-19T09:29:30.557 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-09-19T09:29:30.557 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-09-19T09:29:30.557 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-09-19T09:29:30.557 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-09-19T09:29:30.557 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-09-19T09:29:30.557 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-09-19T09:29:30.557 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-09-19T09:29:30.557 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-09-19T09:29:30.557 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-09-19T09:29:30.557 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-09-19T09:29:30.557 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-09-19T09:29:30.558 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-09-19T09:29:30.558 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-09-19T09:29:30.558 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-09-19T09:29:30.558 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-09-19T09:29:30.558 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-09-19T09:29:30.558 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-09-19T09:29:30.558 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-09-19T09:29:30.558 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-09-19T09:29:30.558 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-09-19T09:29:30.558 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-09-19T09:29:30.559 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-09-19T09:29:30.559 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-09-19T09:29:30.559 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-09-19T09:29:30.559 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-09-19T09:29:30.559 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-09-19T09:29:30.559 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-09-19T09:29:30.559 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-09-19T09:29:30.569 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-09-19T09:29:30.569 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-09-19T09:29:30.569 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-09-19T09:29:30.569 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-09-19T09:29:30.569 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-09-19T09:29:30.569 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-09-19T09:29:30.569 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-09-19T09:29:30.569 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-09-19T09:29:30.570 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-09-19T09:29:30.570 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-09-19T09:29:30.570 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-09-19T09:29:30.570 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-09-19T09:29:30.570 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-09-19T09:29:30.570 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-09-19T09:29:30.570 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-09-19T09:29:30.570 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-09-19T09:29:30.570 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-09-19T09:29:30.570 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-09-19T09:29:30.570 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-09-19T09:29:30.570 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-09-19T09:29:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-09-19T09:29:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-09-19T09:29:30.573 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-09-19T09:29:30.573 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-09-19T09:29:30.573 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-09-19T09:29:30.573 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-09-19T09:29:30.573 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-09-19T09:29:30.573 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-09-19T09:29:30.573 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-09-19T09:29:30.573 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-09-19T09:29:30.573 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-09-19T09:29:30.573 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-09-19T09:29:30.573 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-09-19T09:29:30.573 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-09-19T09:29:30.573 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-09-19T09:29:30.574 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-09-19T09:29:30.575 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-09-19T09:29:30.575 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-09-19T09:29:30.575 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-09-19T09:29:30.575 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-09-19T09:29:30.575 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-09-19T09:29:30.575 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-09-19T09:29:30.575 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-09-19T09:29:30.575 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-09-19T09:29:30.589 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-09-19T09:29:30.589 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-09-19T09:29:30.590 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-09-19T09:29:30.590 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-09-19T09:29:30.590 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-09-19T09:29:30.591 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-09-19T09:29:30.591 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-09-19T09:29:30.591 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-09-19T09:29:30.591 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-09-19T09:29:30.591 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-09-19T09:29:30.591 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-09-19T09:29:30.591 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-09-19T09:29:30.591 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-09-19T09:29:30.591 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-09-19T09:29:30.591 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-09-19T09:29:30.592 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-09-19T09:29:30.592 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-09-19T09:29:30.592 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-09-19T09:29:30.592 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-09-19T09:29:30.592 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-09-19T09:29:30.592 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-09-19T09:29:30.592 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-09-19T09:29:30.592 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-09-19T09:29:30.592 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-09-19T09:29:30.592 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-09-19T09:29:30.592 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-09-19T09:29:30.592 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-09-19T09:29:30.593 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-09-19T09:29:30.593 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-09-19T09:29:30.593 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-09-19T09:29:30.593 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-09-19T09:29:30.593 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-09-19T09:29:30.593 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-09-19T09:29:30.593 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-09-19T09:29:30.593 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-09-19T09:29:30.593 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-09-19T09:29:30.593 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-09-19T09:29:30.593 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-09-19T09:29:30.593 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-09-19T09:29:30.594 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-09-19T09:29:30.594 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-09-19T09:29:30.594 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-09-19T09:29:30.594 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-09-19T09:29:30.594 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-09-19T09:29:30.594 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-09-19T09:29:30.594 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-09-19T09:29:30.595 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-09-19T09:29:30.596 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-09-19T09:29:30.596 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-09-19T09:29:30.596 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-09-19T09:29:30.596 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-09-19T09:29:30.596 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-09-19T09:29:30.596 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-09-19T09:29:30.596 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-09-19T09:29:30.596 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-09-19T09:29:30.596 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-09-19T09:29:30.596 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-09-19T09:29:30.596 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-09-19T09:29:30.596 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-09-19T09:29:30.597 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-09-19T09:29:30.597 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-09-19T09:29:30.597 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-09-19T09:29:30.597 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-09-19T09:29:30.597 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-09-19T09:29:30.597 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-09-19T09:29:30.597 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-09-19T09:29:30.597 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-09-19T09:29:30.597 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-09-19T09:29:30.597 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-09-19T09:29:30.597 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-09-19T09:29:30.597 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-09-19T09:29:30.598 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-09-19T09:29:30.598 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-09-19T09:29:30.598 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-09-19T09:29:30.598 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-09-19T09:29:30.598 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-09-19T09:29:30.598 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-09-19T09:29:30.598 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-09-19T09:29:30.598 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-09-19T09:29:30.598 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-09-19T09:29:30.598 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-09-19T09:29:30.598 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-09-19T09:29:30.598 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-09-19T09:29:30.602 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-09-19T09:29:30.602 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-09-19T09:29:30.602 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-09-19T09:29:30.602 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-09-19T09:29:30.602 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-09-19T09:29:30.602 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-09-19T09:29:30.602 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-09-19T09:29:30.602 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-09-19T09:29:30.602 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-09-19T09:29:30.602 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-09-19T09:29:30.602 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-09-19T09:29:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-09-19T09:29:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-09-19T09:29:30.605 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-09-19T09:29:30.605 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-09-19T09:29:30.605 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-09-19T09:29:30.605 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-09-19T09:29:30.605 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-09-19T09:29:30.605 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-09-19T09:29:30.605 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-09-19T09:29:30.605 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-09-19T09:29:30.605 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-09-19T09:29:30.605 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-09-19T09:29:30.605 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-09-19T09:29:30.605 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-09-19T09:29:30.605 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-09-19T09:29:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-09-19T09:29:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-09-19T09:29:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-09-19T09:29:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-09-19T09:29:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-09-19T09:29:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-09-19T09:29:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-09-19T09:29:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-09-19T09:29:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-09-19T09:29:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-09-19T09:29:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-09-19T09:29:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-09-19T09:29:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-09-19T09:29:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-09-19T09:29:30.608 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-09-19T09:29:30.608 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-09-19T09:29:30.608 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-09-19T09:29:30.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-09-19T09:29:30.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-09-19T09:29:30.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-09-19T09:29:30.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-09-19T09:29:30.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-09-19T09:29:30.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-09-19T09:29:30.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-09-19T09:29:30.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-09-19T09:29:30.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-09-19T09:29:30.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-09-19T09:29:30.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-09-19T09:29:30.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-09-19T09:29:30.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-09-19T09:29:30.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-09-19T09:29:30.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-09-19T09:29:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-09-19T09:29:30.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-09-19T09:29:30.616 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-09-19T09:29:30.616 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-09-19T09:29:30.616 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-09-19T09:29:30.616 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-09-19T09:29:30.616 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-09-19T09:29:30.616 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-09-19T09:29:30.616 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-09-19T09:29:30.616 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-09-19T09:29:30.616 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-09-19T09:29:30.616 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-09-19T09:29:30.616 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-09-19T09:29:30.616 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-09-19T09:29:30.616 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-09-19T09:29:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-09-19T09:29:30.618 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-09-19T09:29:30.618 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-09-19T09:29:30.618 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-09-19T09:29:30.618 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-09-19T09:29:30.618 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-09-19T09:29:30.618 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-09-19T09:29:30.618 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-09-19T09:29:30.618 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-09-19T09:29:30.618 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-09-19T09:29:30.618 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-09-19T09:29:30.618 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-09-19T09:29:30.618 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-09-19T09:29:30.627 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-09-19T09:29:30.627 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-09-19T09:29:30.627 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-09-19T09:29:30.627 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-09-19T09:29:30.627 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-09-19T09:29:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-09-19T09:29:30.629 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-09-19T09:29:30.629 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-09-19T09:29:30.629 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-09-19T09:29:30.629 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-09-19T09:29:30.629 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-09-19T09:29:30.629 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-09-19T09:29:30.629 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-09-19T09:29:30.629 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-09-19T09:29:30.629 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-09-19T09:29:30.629 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-09-19T09:29:30.629 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-09-19T09:29:30.629 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-09-19T09:29:30.629 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-09-19T09:29:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-09-19T09:29:30.631 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-09-19T09:29:30.632 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-09-19T09:29:30.632 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-09-19T09:29:30.632 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-09-19T09:29:30.632 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-09-19T09:29:30.632 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-09-19T09:29:30.632 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-09-19T09:29:30.632 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-09-19T09:29:30.632 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-09-19T09:29:30.632 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-09-19T09:29:30.632 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-09-19T09:29:30.632 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-09-19T09:29:30.632 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-09-19T09:29:30.632 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-09-19T09:29:30.633 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-09-19T09:29:30.633 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-09-19T09:29:30.633 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-09-19T09:29:30.633 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-09-19T09:29:30.633 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-09-19T09:29:30.633 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-09-19T09:29:30.633 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-09-19T09:29:30.633 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-09-19T09:29:30.633 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-09-19T09:29:30.647 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-09-19T09:29:30.647 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-09-19T09:29:30.647 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-09-19T09:29:30.647 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-09-19T09:29:30.647 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-09-19T09:29:30.648 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-09-19T09:29:30.648 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-09-19T09:29:30.648 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-09-19T09:29:30.648 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-09-19T09:29:30.648 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-09-19T09:29:30.648 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-09-19T09:29:30.648 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-09-19T09:29:30.648 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-09-19T09:29:30.648 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-09-19T09:29:30.648 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-09-19T09:29:30.648 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-09-19T09:29:30.648 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-09-19T09:29:30.648 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-09-19T09:29:30.649 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-09-19T09:29:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-09-19T09:29:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-09-19T09:29:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-09-19T09:29:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-09-19T09:29:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-09-19T09:29:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-09-19T09:29:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-09-19T09:29:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-09-19T09:29:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-09-19T09:29:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-09-19T09:29:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-09-19T09:29:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-09-19T09:29:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-09-19T09:29:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-09-19T09:29:30.652 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-09-19T09:29:30.652 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-09-19T09:29:30.652 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-09-19T09:29:30.652 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-09-19T09:29:30.652 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-09-19T09:29:30.652 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-09-19T09:29:30.652 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-09-19T09:29:30.652 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-09-19T09:29:30.652 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-09-19T09:29:30.652 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-09-19T09:29:30.652 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-09-19T09:29:30.652 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-09-19T09:29:30.652 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-09-19T09:29:30.653 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-09-19T09:29:30.653 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-09-19T09:29:30.653 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-09-19T09:29:30.653 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-09-19T09:29:30.653 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-09-19T09:29:30.653 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-09-19T09:29:30.653 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-09-19T09:29:30.653 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-09-19T09:29:30.653 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-09-19T09:29:30.653 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-09-19T09:29:30.659 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-09-19T09:29:30.660 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-09-19T09:29:30.660 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-09-19T09:29:30.660 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-09-19T09:29:30.660 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-09-19T09:29:30.660 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-09-19T09:29:30.660 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-09-19T09:29:30.660 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-09-19T09:29:30.660 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-09-19T09:29:30.660 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-09-19T09:29:30.660 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-09-19T09:29:30.660 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-09-19T09:29:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-09-19T09:29:30.662 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-09-19T09:29:30.662 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-09-19T09:29:30.662 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-09-19T09:29:30.662 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-09-19T09:29:30.662 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-09-19T09:29:30.662 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-09-19T09:29:30.662 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-09-19T09:29:30.662 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-09-19T09:29:30.662 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-09-19T09:29:30.662 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-09-19T09:29:30.662 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-09-19T09:29:30.662 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-09-19T09:29:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-09-19T09:29:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-09-19T09:29:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-09-19T09:29:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-09-19T09:29:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-09-19T09:29:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-09-19T09:29:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-09-19T09:29:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-09-19T09:29:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-09-19T09:29:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-09-19T09:29:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-09-19T09:29:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-09-19T09:29:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-09-19T09:29:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-09-19T09:29:30.665 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-09-19T09:29:30.666 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-09-19T09:29:30.666 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-09-19T09:29:30.666 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-09-19T09:29:30.666 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-09-19T09:29:30.669 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-09-19T09:29:30.669 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-09-19T09:29:30.669 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-09-19T09:29:30.669 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-09-19T09:29:30.669 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-09-19T09:29:30.669 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-09-19T09:29:30.669 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-09-19T09:29:30.669 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-09-19T09:29:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-09-19T09:29:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-09-19T09:29:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-09-19T09:29:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-09-19T09:29:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-09-19T09:29:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-09-19T09:29:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-09-19T09:29:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-09-19T09:29:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-09-19T09:29:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-09-19T09:29:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-09-19T09:29:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-09-19T09:29:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-09-19T09:29:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-09-19T09:29:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-09-19T09:29:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-09-19T09:29:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-09-19T09:29:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-09-19T09:29:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-09-19T09:29:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-09-19T09:29:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-09-19T09:29:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-09-19T09:29:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-09-19T09:29:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-09-19T09:29:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-09-19T09:29:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-09-19T09:29:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-09-19T09:29:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-09-19T09:29:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-09-19T09:29:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-09-19T09:29:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-09-19T09:29:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-09-19T09:29:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-09-19T09:29:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-09-19T09:29:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-09-19T09:29:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-09-19T09:29:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-09-19T09:29:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-09-19T09:29:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-09-19T09:29:30.673 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-09-19T09:29:30.673 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-09-19T09:29:30.673 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-09-19T09:29:30.673 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-09-19T09:29:30.673 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-09-19T09:29:30.673 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-09-19T09:29:30.673 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-09-19T09:29:30.673 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-09-19T09:29:30.673 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-09-19T09:29:30.673 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-09-19T09:29:30.673 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-09-19T09:29:30.673 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-09-19T09:29:30.673 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-09-19T09:29:30.674 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-09-19T09:29:30.675 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-09-19T09:29:30.675 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-09-19T09:29:30.675 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-09-19T09:29:30.675 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-09-19T09:29:30.675 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-09-19T09:29:30.675 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-09-19T09:29:30.675 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-09-19T09:29:30.675 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-09-19T09:29:30.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-09-19T09:29:30.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-09-19T09:29:30.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-09-19T09:29:30.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-09-19T09:29:30.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-09-19T09:29:30.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-09-19T09:29:30.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-09-19T09:29:30.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-09-19T09:29:30.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-09-19T09:29:30.689 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-09-19T09:29:30.689 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-09-19T09:29:30.689 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-09-19T09:29:30.689 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-09-19T09:29:30.689 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-09-19T09:29:30.689 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-09-19T09:29:30.689 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-09-19T09:29:30.689 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-09-19T09:29:30.689 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-09-19T09:29:30.689 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-09-19T09:29:30.689 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-09-19T09:29:30.689 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-09-19T09:29:30.690 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-09-19T09:29:30.691 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-09-19T09:29:30.691 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-09-19T09:29:30.691 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-09-19T09:29:30.691 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-09-19T09:29:30.691 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-09-19T09:29:30.691 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-09-19T09:29:30.691 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-09-19T09:29:30.691 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-09-19T09:29:30.705 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-09-19T09:29:30.705 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-09-19T09:29:30.705 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-09-19T09:29:30.705 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-09-19T09:29:30.705 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-09-19T09:29:30.705 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-09-19T09:29:30.705 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-09-19T09:29:30.705 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-09-19T09:29:30.705 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-09-19T09:29:30.705 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-09-19T09:29:30.705 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-09-19T09:29:30.705 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-09-19T09:29:30.706 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-09-19T09:29:30.707 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-09-19T09:29:30.708 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-09-19T09:29:30.709 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-09-19T09:29:30.710 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-09-19T09:29:30.716 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-09-19T09:29:30.717 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-09-19T09:29:30.717 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-09-19T09:29:30.717 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-09-19T09:29:30.717 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-09-19T09:29:30.717 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-09-19T09:29:30.717 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-09-19T09:29:30.717 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-09-19T09:29:30.717 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-09-19T09:29:30.717 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-09-19T09:29:30.717 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-09-19T09:29:30.717 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-09-19T09:29:30.717 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-09-19T09:29:30.718 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-09-19T09:29:30.718 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-09-19T09:29:30.718 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-09-19T09:29:30.718 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-09-19T09:29:30.718 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-09-19T09:29:30.718 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-09-19T09:29:30.718 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-09-19T09:29:30.718 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-09-19T09:29:30.718 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-09-19T09:29:30.718 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-09-19T09:29:30.718 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-09-19T09:29:30.718 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-09-19T09:29:30.719 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-09-19T09:29:30.720 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-09-19T09:29:30.721 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-09-19T09:29:30.722 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-09-19T09:29:30.723 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-09-19T09:29:30.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-09-19T09:29:30.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-09-19T09:29:30.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-09-19T09:29:30.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-09-19T09:29:30.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-09-19T09:29:30.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-09-19T09:29:30.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-09-19T09:29:30.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-09-19T09:29:30.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-09-19T09:29:30.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-09-19T09:29:30.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-09-19T09:29:30.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-09-19T09:29:30.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-09-19T09:29:30.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-09-19T09:29:30.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-09-19T09:29:30.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-09-19T09:29:30.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-09-19T09:29:30.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-09-19T09:29:30.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-09-19T09:29:30.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-09-19T09:29:30.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-09-19T09:29:30.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-09-19T09:29:30.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-09-19T09:29:30.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-09-19T09:29:30.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-09-19T09:29:30.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-09-19T09:29:30.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-09-19T09:29:30.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-09-19T09:29:30.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-09-19T09:29:30.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-09-19T09:29:30.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-09-19T09:29:30.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-09-19T09:29:30.731 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-09-19T09:29:30.731 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-09-19T09:29:30.731 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-09-19T09:29:30.731 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-09-19T09:29:30.731 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-09-19T09:29:30.731 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-09-19T09:29:30.731 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-09-19T09:29:30.731 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-09-19T09:29:30.731 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-09-19T09:29:30.731 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-09-19T09:29:30.731 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-09-19T09:29:30.731 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-09-19T09:29:30.731 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-09-19T09:29:30.732 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-09-19T09:29:30.732 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-09-19T09:29:30.732 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-09-19T09:29:30.732 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-09-19T09:29:30.732 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-09-19T09:29:30.732 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-09-19T09:29:30.732 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-09-19T09:29:30.732 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-09-19T09:29:30.732 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-09-19T09:29:30.732 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-09-19T09:29:30.732 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-09-19T09:29:30.732 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-09-19T09:29:30.732 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-09-19T09:29:30.733 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-09-19T09:29:30.733 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-09-19T09:29:30.733 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-09-19T09:29:30.733 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-09-19T09:29:30.733 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-09-19T09:29:30.733 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-09-19T09:29:30.733 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-09-19T09:29:30.733 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-09-19T09:29:30.733 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-09-19T09:29:30.733 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-09-19T09:29:30.733 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-09-19T09:29:30.733 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-09-19T09:29:30.743 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-09-19T09:29:30.743 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-09-19T09:29:30.743 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-09-19T09:29:30.743 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-09-19T09:29:30.744 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-09-19T09:29:30.744 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-09-19T09:29:30.744 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-09-19T09:29:30.744 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-09-19T09:29:30.744 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-09-19T09:29:30.744 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-09-19T09:29:30.744 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-09-19T09:29:30.744 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-09-19T09:29:30.744 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-09-19T09:29:30.744 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-09-19T09:29:30.744 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-09-19T09:29:30.744 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-09-19T09:29:30.744 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-09-19T09:29:30.745 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-09-19T09:29:30.745 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-09-19T09:29:30.745 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-09-19T09:29:30.745 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-09-19T09:29:30.745 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-09-19T09:29:30.745 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-09-19T09:29:30.745 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-09-19T09:29:30.745 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-09-19T09:29:30.745 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-09-19T09:29:30.745 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-09-19T09:29:30.745 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-09-19T09:29:30.745 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-09-19T09:29:30.745 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-09-19T09:29:30.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-09-19T09:29:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-09-19T09:29:30.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-09-19T09:29:30.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-09-19T09:29:30.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-09-19T09:29:30.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-09-19T09:29:30.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-09-19T09:29:30.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-09-19T09:29:30.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-09-19T09:29:30.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-09-19T09:29:30.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-09-19T09:29:30.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-09-19T09:29:30.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-09-19T09:29:30.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-09-19T09:29:30.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-09-19T09:29:30.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-09-19T09:29:30.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-09-19T09:29:30.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-09-19T09:29:30.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-09-19T09:29:30.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-09-19T09:29:30.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-09-19T09:29:30.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-09-19T09:29:30.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-09-19T09:29:30.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-09-19T09:29:30.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-09-19T09:29:30.767 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-09-19T09:29:30.767 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-09-19T09:29:30.767 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-09-19T09:29:30.767 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-09-19T09:29:30.767 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-09-19T09:29:30.767 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-09-19T09:29:30.767 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-09-19T09:29:30.767 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-09-19T09:29:30.767 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-09-19T09:29:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-09-19T09:29:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-09-19T09:29:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-09-19T09:29:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-09-19T09:29:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-09-19T09:29:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-09-19T09:29:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-09-19T09:29:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-09-19T09:29:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-09-19T09:29:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-09-19T09:29:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-09-19T09:29:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-09-19T09:29:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-09-19T09:29:30.769 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-09-19T09:29:30.769 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-09-19T09:29:30.769 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-09-19T09:29:30.769 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-09-19T09:29:30.769 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-09-19T09:29:30.769 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-09-19T09:29:30.769 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-09-19T09:29:30.769 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-09-19T09:29:30.769 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-09-19T09:29:30.769 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-09-19T09:29:30.769 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-09-19T09:29:30.769 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-09-19T09:29:30.769 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-09-19T09:29:30.770 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-09-19T09:29:30.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-09-19T09:29:30.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-09-19T09:29:30.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-09-19T09:29:30.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-09-19T09:29:30.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-09-19T09:29:30.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-09-19T09:29:30.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-09-19T09:29:30.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-09-19T09:29:30.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-09-19T09:29:30.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-09-19T09:29:30.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-09-19T09:29:30.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-09-19T09:29:30.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-09-19T09:29:30.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-09-19T09:29:30.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-09-19T09:29:30.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-09-19T09:29:30.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-09-19T09:29:30.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-09-19T09:29:30.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-09-19T09:29:30.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-09-19T09:29:30.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-09-19T09:29:30.777 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-09-19T09:29:30.777 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-09-19T09:29:30.777 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-09-19T09:29:30.777 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-09-19T09:29:30.777 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-09-19T09:29:30.777 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-09-19T09:29:30.777 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-09-19T09:29:30.777 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-09-19T09:29:30.777 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-09-19T09:29:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-09-19T09:29:30.779 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-09-19T09:29:30.780 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-09-19T09:29:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-09-19T09:29:30.782 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-09-19T09:29:30.783 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-09-19T09:29:30.783 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-09-19T09:29:30.783 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-09-19T09:29:30.785 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-09-19T09:29:30.785 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-09-19T09:29:30.785 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-09-19T09:29:30.785 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-09-19T09:29:30.785 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-09-19T09:29:30.785 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-09-19T09:29:30.786 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-09-19T09:29:30.787 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-09-19T09:29:30.787 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-09-19T09:29:30.787 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-09-19T09:29:30.787 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-09-19T09:29:30.787 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-09-19T09:29:30.787 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-09-19T09:29:30.787 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-09-19T09:29:30.787 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-09-19T09:29:30.787 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-09-19T09:29:30.787 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-09-19T09:29:30.787 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-09-19T09:29:30.787 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-09-19T09:29:30.787 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-09-19T09:29:30.788 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-09-19T09:29:30.789 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-09-19T09:29:30.789 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-09-19T09:29:30.789 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-09-19T09:29:30.789 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-09-19T09:29:30.789 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-09-19T09:29:30.789 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-09-19T09:29:30.789 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-09-19T09:29:30.789 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-09-19T09:29:30.789 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-09-19T09:29:30.789 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-09-19T09:29:30.789 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-09-19T09:29:30.789 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-09-19T09:29:30.789 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-09-19T09:29:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-09-19T09:29:30.791 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-09-19T09:29:30.791 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-09-19T09:29:30.791 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-09-19T09:29:30.791 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-09-19T09:29:30.791 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-09-19T09:29:30.791 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-09-19T09:29:30.791 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-09-19T09:29:30.791 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-09-19T09:29:30.801 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-09-19T09:29:30.801 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-09-19T09:29:30.801 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-09-19T09:29:30.801 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-09-19T09:29:30.801 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-09-19T09:29:30.801 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-09-19T09:29:30.801 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-09-19T09:29:30.802 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-09-19T09:29:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-09-19T09:29:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-09-19T09:29:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-09-19T09:29:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-09-19T09:29:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-09-19T09:29:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-09-19T09:29:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-09-19T09:29:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-09-19T09:29:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-09-19T09:29:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-09-19T09:29:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-09-19T09:29:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-09-19T09:29:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-09-19T09:29:30.804 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-09-19T09:29:30.805 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-09-19T09:29:30.806 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-09-19T09:29:30.807 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-09-19T09:29:30.807 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-09-19T09:29:30.807 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-09-19T09:29:30.807 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-09-19T09:29:30.807 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-09-19T09:29:30.824 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-09-19T09:29:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-09-19T09:29:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-09-19T09:29:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-09-19T09:29:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-09-19T09:29:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-09-19T09:29:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-09-19T09:29:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-09-19T09:29:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-09-19T09:29:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-09-19T09:29:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-09-19T09:29:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-09-19T09:29:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-09-19T09:29:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-09-19T09:29:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-09-19T09:29:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-09-19T09:29:30.828 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-09-19T09:29:30.829 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-09-19T09:29:30.830 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-09-19T09:29:30.830 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-09-19T09:29:30.830 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-09-19T09:29:30.830 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-09-19T09:29:30.830 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-09-19T09:29:30.830 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-09-19T09:29:30.830 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-09-19T09:29:30.830 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-09-19T09:29:30.830 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-09-19T09:29:30.830 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-09-19T09:29:30.830 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-09-19T09:29:30.830 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-09-19T09:29:30.835 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-09-19T09:29:30.835 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-09-19T09:29:30.835 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-09-19T09:29:30.835 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-09-19T09:29:30.835 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-09-19T09:29:30.836 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-09-19T09:29:30.836 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-09-19T09:29:30.836 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-09-19T09:29:30.836 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-09-19T09:29:30.836 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-09-19T09:29:30.836 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-09-19T09:29:30.836 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-09-19T09:29:30.836 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-09-19T09:29:30.836 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-09-19T09:29:30.836 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-09-19T09:29:30.836 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-09-19T09:29:30.836 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-09-19T09:29:30.837 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-09-19T09:29:30.837 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-09-19T09:29:30.837 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-09-19T09:29:30.837 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-09-19T09:29:30.837 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-09-19T09:29:30.837 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-09-19T09:29:30.837 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-09-19T09:29:30.837 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-09-19T09:29:30.837 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-09-19T09:29:30.837 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-09-19T09:29:30.837 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-09-19T09:29:30.837 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-09-19T09:29:30.837 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-09-19T09:29:30.838 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-09-19T09:29:30.838 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-09-19T09:29:30.838 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-09-19T09:29:30.838 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-09-19T09:29:30.838 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-09-19T09:29:30.838 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-09-19T09:29:30.838 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-09-19T09:29:30.838 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-09-19T09:29:30.838 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-09-19T09:29:30.838 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-09-19T09:29:30.838 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-09-19T09:29:30.839 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-09-19T09:29:30.839 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-09-19T09:29:30.839 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-09-19T09:29:30.839 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-09-19T09:29:30.839 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-09-19T09:29:30.839 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-09-19T09:29:30.839 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-09-19T09:29:30.839 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-09-19T09:29:30.839 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-09-19T09:29:30.840 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-09-19T09:29:30.840 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-09-19T09:29:30.840 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-09-19T09:29:30.840 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-09-19T09:29:30.840 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-09-19T09:29:30.840 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-09-19T09:29:30.840 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-09-19T09:29:30.840 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-09-19T09:29:30.840 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-09-19T09:29:30.840 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-09-19T09:29:30.840 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-09-19T09:29:30.840 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-09-19T09:29:30.841 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-09-19T09:29:30.841 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-09-19T09:29:30.841 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-09-19T09:29:30.841 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-09-19T09:29:30.841 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-09-19T09:29:30.841 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-09-19T09:29:30.841 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-09-19T09:29:30.841 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-09-19T09:29:30.841 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-09-19T09:29:30.842 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-09-19T09:29:30.842 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-09-19T09:29:30.842 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-09-19T09:29:30.842 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-09-19T09:29:30.842 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-09-19T09:29:30.842 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-09-19T09:29:30.842 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-09-19T09:29:30.842 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-09-19T09:29:30.842 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-09-19T09:29:30.842 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-09-19T09:29:30.842 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-09-19T09:29:30.842 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-09-19T09:29:30.848 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-09-19T09:29:30.848 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-09-19T09:29:30.849 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-09-19T09:29:30.849 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-09-19T09:29:30.849 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-09-19T09:29:30.849 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-09-19T09:29:30.849 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-09-19T09:29:30.849 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-09-19T09:29:30.849 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-09-19T09:29:30.849 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-09-19T09:29:30.849 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-09-19T09:29:30.849 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-09-19T09:29:30.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-09-19T09:29:30.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-09-19T09:29:30.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-09-19T09:29:30.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-09-19T09:29:30.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-09-19T09:29:30.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-09-19T09:29:30.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-09-19T09:29:30.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-09-19T09:29:30.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-09-19T09:29:30.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-09-19T09:29:30.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-09-19T09:29:30.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-09-19T09:29:30.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-09-19T09:29:30.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-09-19T09:29:30.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-09-19T09:29:30.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-09-19T09:29:30.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-09-19T09:29:30.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-09-19T09:29:30.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-09-19T09:29:30.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-09-19T09:29:30.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-09-19T09:29:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-09-19T09:29:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-09-19T09:29:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-09-19T09:29:30.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-09-19T09:29:30.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-09-19T09:29:30.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-09-19T09:29:30.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-09-19T09:29:30.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-09-19T09:29:30.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-09-19T09:29:30.871 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-09-19T09:29:30.871 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-09-19T09:29:30.871 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-09-19T09:29:30.871 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-09-19T09:29:30.871 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-09-19T09:29:30.871 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-09-19T09:29:30.871 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-09-19T09:29:30.871 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-09-19T09:29:30.871 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-09-19T09:29:30.872 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-09-19T09:29:30.872 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-09-19T09:29:30.872 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-09-19T09:29:30.872 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-09-19T09:29:30.872 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-09-19T09:29:30.872 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-09-19T09:29:30.872 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-09-19T09:29:30.872 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-09-19T09:29:30.872 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-09-19T09:29:30.872 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-09-19T09:29:30.872 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-09-19T09:29:30.872 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-09-19T09:29:30.872 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-09-19T09:29:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-09-19T09:29:30.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-09-19T09:29:30.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-09-19T09:29:30.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-09-19T09:29:30.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-09-19T09:29:30.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-09-19T09:29:30.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-09-19T09:29:30.895 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-09-19T09:29:30.895 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-09-19T09:29:30.895 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-09-19T09:29:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-09-19T09:29:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-09-19T09:29:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-09-19T09:29:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-09-19T09:29:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-09-19T09:29:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-09-19T09:29:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-09-19T09:29:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-09-19T09:29:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-09-19T09:29:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-09-19T09:29:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-09-19T09:29:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-09-19T09:29:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-09-19T09:29:30.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-09-19T09:29:30.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-09-19T09:29:30.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-09-19T09:29:30.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-09-19T09:29:30.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-09-19T09:29:30.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-09-19T09:29:30.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-09-19T09:29:30.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-09-19T09:29:30.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-09-19T09:29:30.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-09-19T09:29:30.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-09-19T09:29:30.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-09-19T09:29:30.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-09-19T09:29:30.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-09-19T09:29:30.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-09-19T09:29:30.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-09-19T09:29:30.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-09-19T09:29:30.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-09-19T09:29:30.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-09-19T09:29:30.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-09-19T09:29:30.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-09-19T09:29:30.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-09-19T09:29:30.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-09-19T09:29:30.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-09-19T09:29:30.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-09-19T09:29:30.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-09-19T09:29:30.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-09-19T09:29:30.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-09-19T09:29:30.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-09-19T09:29:30.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-09-19T09:29:30.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-09-19T09:29:30.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-09-19T09:29:30.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-09-19T09:29:30.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-09-19T09:29:30.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-09-19T09:29:30.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-09-19T09:29:30.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-09-19T09:29:30.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-09-19T09:29:30.906 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-09-19T09:29:30.906 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-09-19T09:29:30.906 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-09-19T09:29:30.906 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-09-19T09:29:30.906 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-09-19T09:29:30.906 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-09-19T09:29:30.906 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-09-19T09:29:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-09-19T09:29:30.908 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-09-19T09:29:30.908 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-09-19T09:29:30.908 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-09-19T09:29:30.908 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-09-19T09:29:30.908 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-09-19T09:29:30.908 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-09-19T09:29:30.908 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-09-19T09:29:30.908 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-09-19T09:29:30.908 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-09-19T09:29:30.908 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-09-19T09:29:30.908 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-09-19T09:29:30.908 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-09-19T09:29:30.908 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-09-19T09:29:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-09-19T09:29:30.910 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-09-19T09:29:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-09-19T09:29:30.912 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-09-19T09:29:30.912 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-09-19T09:29:30.912 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-09-19T09:29:30.912 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-09-19T09:29:31.049 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-09-19T09:29:31.049 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-09-19T09:29:31.049 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-09-19T09:29:31.049 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-09-19T09:29:31.049 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-09-19T09:29:31.049 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-09-19T09:29:31.049 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-09-19T09:29:31.049 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-09-19T09:29:31.049 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-09-19T09:29:31.049 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-09-19T09:29:31.049 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-09-19T09:29:31.050 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-09-19T09:29:31.051 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-09-19T09:29:31.051 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-09-19T09:29:31.051 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-09-19T09:29:31.051 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-09-19T09:29:31.051 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-09-19T09:29:31.051 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-09-19T09:29:31.051 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-09-19T09:29:31.051 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-09-19T09:29:31.051 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-09-19T09:29:31.051 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-09-19T09:29:31.051 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-09-19T09:29:31.051 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-09-19T09:29:31.051 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-09-19T09:29:31.052 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-09-19T09:29:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-09-19T09:29:31.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-09-19T09:29:31.055 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-09-19T09:29:31.055 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-09-19T09:29:31.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:31 smithi016 ceph-mon[30979]: pgmap v395: 97 pgs: 97 active+clean; 796 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-19T09:29:31.551 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-09-19T09:29:31.552 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-09-19T09:29:31.552 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-09-19T09:29:31.552 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-09-19T09:29:31.552 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-09-19T09:29:31.552 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-09-19T09:29:31.552 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-09-19T09:29:31.552 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-09-19T09:29:31.552 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-09-19T09:29:31.552 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-09-19T09:29:31.552 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-09-19T09:29:31.552 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-09-19T09:29:31.552 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-09-19T09:29:31.552 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-09-19T09:29:31.553 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-09-19T09:29:31.554 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-09-19T09:29:31.555 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-09-19T09:29:31.556 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-09-19T09:29:31.556 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-09-19T09:29:31.556 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-09-19T09:29:31.556 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-09-19T09:29:31.556 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-09-19T09:29:31.556 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-09-19T09:29:31.556 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-09-19T09:29:31.556 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-09-19T09:29:31.556 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-09-19T09:29:31.556 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-09-19T09:29:31.556 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-09-19T09:29:31.556 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-09-19T09:29:31.556 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-09-19T09:29:31.557 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-09-19T09:29:31.558 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-09-19T09:29:31.558 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-09-19T09:29:31.558 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-09-19T09:29:31.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:31 smithi142 ceph-mon[35032]: pgmap v395: 97 pgs: 97 active+clean; 796 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-19T09:29:31.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:31 smithi097 ceph-mon[34864]: pgmap v395: 97 pgs: 97 active+clean; 796 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-19T09:29:31.992 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-09-19T09:29:31.992 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-09-19T09:29:31.992 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-09-19T09:29:31.992 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-09-19T09:29:31.992 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-09-19T09:29:31.992 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-09-19T09:29:31.992 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-09-19T09:29:31.992 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-09-19T09:29:31.992 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-09-19T09:29:31.992 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-09-19T09:29:31.992 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-09-19T09:29:31.993 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-09-19T09:29:31.993 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-09-19T09:29:31.993 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-09-19T09:29:31.993 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-09-19T09:29:31.993 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-09-19T09:29:31.993 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-09-19T09:29:31.993 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-09-19T09:29:31.993 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-09-19T09:29:31.993 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-09-19T09:29:31.996 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-09-19T09:29:31.997 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-09-19T09:29:31.997 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-09-19T09:29:31.997 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-09-19T09:29:31.997 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-09-19T09:29:31.997 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-09-19T09:29:31.997 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-09-19T09:29:31.997 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-09-19T09:29:31.997 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-09-19T09:29:31.997 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-09-19T09:29:31.997 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-09-19T09:29:31.997 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-09-19T09:29:31.997 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-09-19T09:29:31.998 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-09-19T09:29:31.999 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-09-19T09:29:32.000 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-09-19T09:29:32.001 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-09-19T09:29:32.001 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-09-19T09:29:32.001 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-09-19T09:29:32.001 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-09-19T09:29:32.001 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-09-19T09:29:32.001 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-09-19T09:29:32.001 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-09-19T09:29:32.001 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-09-19T09:29:32.001 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-09-19T09:29:32.001 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-09-19T09:29:32.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-09-19T09:29:32.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-09-19T09:29:32.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-09-19T09:29:32.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-09-19T09:29:32.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-09-19T09:29:32.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-09-19T09:29:32.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-09-19T09:29:32.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-09-19T09:29:32.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-09-19T09:29:32.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-09-19T09:29:32.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-09-19T09:29:32.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-09-19T09:29:32.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-09-19T09:29:32.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-09-19T09:29:32.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-09-19T09:29:32.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-09-19T09:29:32.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-09-19T09:29:32.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-09-19T09:29:32.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-09-19T09:29:32.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-09-19T09:29:32.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-09-19T09:29:32.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-09-19T09:29:32.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-09-19T09:29:32.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-09-19T09:29:32.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-09-19T09:29:32.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-09-19T09:29:32.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-09-19T09:29:32.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-09-19T09:29:32.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-09-19T09:29:32.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-09-19T09:29:32.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-09-19T09:29:32.322 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-09-19T09:29:32.322 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-09-19T09:29:32.322 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-09-19T09:29:32.322 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-09-19T09:29:32.322 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-09-19T09:29:32.322 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-09-19T09:29:32.322 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-09-19T09:29:32.322 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-09-19T09:29:32.322 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-09-19T09:29:32.322 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-09-19T09:29:32.322 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-09-19T09:29:32.322 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-09-19T09:29:32.322 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-09-19T09:29:32.323 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-09-19T09:29:32.324 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-09-19T09:29:32.324 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-09-19T09:29:32.324 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-09-19T09:29:32.324 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-09-19T09:29:32.324 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-09-19T09:29:32.324 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-09-19T09:29:32.324 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-09-19T09:29:32.324 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-09-19T09:29:32.324 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-09-19T09:29:32.324 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-09-19T09:29:32.324 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-09-19T09:29:32.324 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-09-19T09:29:32.324 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-09-19T09:29:32.325 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-09-19T09:29:32.325 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-09-19T09:29:32.325 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-09-19T09:29:32.325 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-09-19T09:29:32.325 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-09-19T09:29:32.325 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-09-19T09:29:32.325 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-09-19T09:29:32.325 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-09-19T09:29:32.325 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-09-19T09:29:32.325 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-09-19T09:29:32.325 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-09-19T09:29:32.325 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-09-19T09:29:32.326 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-09-19T09:29:32.326 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-09-19T09:29:32.326 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-09-19T09:29:32.781 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-09-19T09:29:32.781 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-09-19T09:29:32.781 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-09-19T09:29:32.781 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-09-19T09:29:32.781 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-09-19T09:29:32.781 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-09-19T09:29:32.781 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-09-19T09:29:32.781 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-09-19T09:29:32.782 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-09-19T09:29:32.782 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-09-19T09:29:32.782 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-09-19T09:29:32.782 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-09-19T09:29:32.782 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-09-19T09:29:32.782 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-09-19T09:29:32.782 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-09-19T09:29:32.782 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-09-19T09:29:32.782 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-09-19T09:29:32.782 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-09-19T09:29:32.782 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-09-19T09:29:32.782 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-09-19T09:29:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-09-19T09:29:32.784 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-09-19T09:29:32.784 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-09-19T09:29:32.784 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-09-19T09:29:32.784 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-09-19T09:29:32.784 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-09-19T09:29:32.784 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-09-19T09:29:32.784 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-09-19T09:29:32.784 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-09-19T09:29:32.784 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-09-19T09:29:32.784 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-09-19T09:29:32.784 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-09-19T09:29:32.784 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-09-19T09:29:32.784 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-09-19T09:29:32.785 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-09-19T09:29:32.785 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-09-19T09:29:32.785 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-09-19T09:29:32.785 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-09-19T09:29:32.785 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-09-19T09:29:32.785 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-09-19T09:29:32.785 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-09-19T09:29:32.785 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-09-19T09:29:32.785 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-09-19T09:29:32.785 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-09-19T09:29:32.785 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-09-19T09:29:32.785 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-09-19T09:29:32.786 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-09-19T09:29:32.787 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-09-19T09:29:32.787 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-09-19T09:29:32.787 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-09-19T09:29:32.787 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-09-19T09:29:32.787 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-09-19T09:29:32.787 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-09-19T09:29:32.787 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-09-19T09:29:32.787 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-09-19T09:29:32.787 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-09-19T09:29:32.787 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-09-19T09:29:33.298 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-09-19T09:29:33.298 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-09-19T09:29:33.298 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-09-19T09:29:33.298 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-09-19T09:29:33.298 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-09-19T09:29:33.298 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-09-19T09:29:33.298 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-09-19T09:29:33.298 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-09-19T09:29:33.299 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-09-19T09:29:33.300 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-09-19T09:29:33.300 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-09-19T09:29:33.300 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-09-19T09:29:33.300 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-09-19T09:29:33.300 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-09-19T09:29:33.300 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-09-19T09:29:33.300 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-09-19T09:29:33.300 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-09-19T09:29:33.300 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-09-19T09:29:33.300 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-09-19T09:29:33.300 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-09-19T09:29:33.300 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-09-19T09:29:33.300 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-09-19T09:29:33.301 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-09-19T09:29:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-09-19T09:29:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-09-19T09:29:33.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-09-19T09:29:33.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-09-19T09:29:33.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-09-19T09:29:33.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-09-19T09:29:33.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:33 smithi016 ceph-mon[30979]: pgmap v396: 97 pgs: 97 active+clean; 13 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 343 KiB/s wr, 64 op/s 2024-09-19T09:29:33.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:33 smithi142 ceph-mon[35032]: pgmap v396: 97 pgs: 97 active+clean; 13 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 343 KiB/s wr, 64 op/s 2024-09-19T09:29:33.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:33 smithi097 ceph-mon[34864]: pgmap v396: 97 pgs: 97 active+clean; 13 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 343 KiB/s wr, 64 op/s 2024-09-19T09:29:33.683 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-09-19T09:29:33.683 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-09-19T09:29:33.684 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-09-19T09:29:33.684 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-09-19T09:29:33.684 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-09-19T09:29:33.684 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-09-19T09:29:33.684 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-09-19T09:29:33.684 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-09-19T09:29:33.684 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-09-19T09:29:33.684 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-09-19T09:29:33.684 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-09-19T09:29:33.684 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-09-19T09:29:33.684 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-09-19T09:29:33.684 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-09-19T09:29:33.684 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-09-19T09:29:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-09-19T09:29:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-09-19T09:29:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-09-19T09:29:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-09-19T09:29:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-09-19T09:29:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-09-19T09:29:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-09-19T09:29:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-09-19T09:29:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-09-19T09:29:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-09-19T09:29:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-09-19T09:29:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-09-19T09:29:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-09-19T09:29:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-09-19T09:29:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-09-19T09:29:33.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-09-19T09:29:33.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-09-19T09:29:33.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-09-19T09:29:33.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-09-19T09:29:33.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-09-19T09:29:33.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-09-19T09:29:33.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-09-19T09:29:33.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-09-19T09:29:33.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-09-19T09:29:33.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-09-19T09:29:33.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-09-19T09:29:33.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-09-19T09:29:33.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-09-19T09:29:34.005 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-09-19T09:29:34.005 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-09-19T09:29:34.006 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-09-19T09:29:34.007 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-09-19T09:29:34.008 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-09-19T09:29:34.008 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-09-19T09:29:34.008 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-09-19T09:29:34.008 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-09-19T09:29:34.008 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-09-19T09:29:34.008 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-09-19T09:29:34.008 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-09-19T09:29:34.008 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-09-19T09:29:34.008 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-09-19T09:29:34.008 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-09-19T09:29:34.008 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-09-19T09:29:34.008 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-09-19T09:29:34.008 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-09-19T09:29:34.009 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-09-19T09:29:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-09-19T09:29:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-09-19T09:29:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-09-19T09:29:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-09-19T09:29:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-09-19T09:29:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-09-19T09:29:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-09-19T09:29:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-09-19T09:29:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-09-19T09:29:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-09-19T09:29:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-09-19T09:29:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-09-19T09:29:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-09-19T09:29:34.011 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-09-19T09:29:34.011 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-09-19T09:29:34.011 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-09-19T09:29:34.011 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-09-19T09:29:34.011 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-09-19T09:29:34.011 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-09-19T09:29:34.011 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-09-19T09:29:34.011 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-09-19T09:29:34.011 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-09-19T09:29:34.011 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-09-19T09:29:34.011 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-09-19T09:29:34.011 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-09-19T09:29:34.494 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-09-19T09:29:34.494 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-09-19T09:29:34.495 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-09-19T09:29:34.495 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-09-19T09:29:34.495 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-09-19T09:29:34.495 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-09-19T09:29:34.495 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-09-19T09:29:34.495 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-09-19T09:29:34.495 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-09-19T09:29:34.495 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-09-19T09:29:34.495 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-09-19T09:29:34.495 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-09-19T09:29:34.495 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-09-19T09:29:34.495 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-09-19T09:29:34.495 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-09-19T09:29:34.496 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-09-19T09:29:34.496 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-09-19T09:29:34.496 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-09-19T09:29:34.496 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-09-19T09:29:34.496 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-09-19T09:29:34.496 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-09-19T09:29:34.496 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-09-19T09:29:34.496 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-09-19T09:29:34.496 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-09-19T09:29:34.496 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-09-19T09:29:34.496 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-09-19T09:29:34.496 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-09-19T09:29:34.496 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-09-19T09:29:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-09-19T09:29:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-09-19T09:29:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-09-19T09:29:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-09-19T09:29:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-09-19T09:29:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-09-19T09:29:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-09-19T09:29:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-09-19T09:29:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-09-19T09:29:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-09-19T09:29:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-09-19T09:29:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-09-19T09:29:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-09-19T09:29:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-09-19T09:29:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-09-19T09:29:34.500 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-09-19T09:29:34.500 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-09-19T09:29:34.500 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-09-19T09:29:34.500 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-09-19T09:29:34.500 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-09-19T09:29:34.500 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-09-19T09:29:34.500 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-09-19T09:29:34.500 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-09-19T09:29:34.500 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-09-19T09:29:34.500 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-09-19T09:29:34.939 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-09-19T09:29:34.939 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-09-19T09:29:34.939 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-09-19T09:29:34.940 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-09-19T09:29:34.941 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-09-19T09:29:34.942 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-09-19T09:29:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-09-19T09:29:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-09-19T09:29:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-09-19T09:29:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-09-19T09:29:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-09-19T09:29:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-09-19T09:29:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-09-19T09:29:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-09-19T09:29:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-09-19T09:29:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-09-19T09:29:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-09-19T09:29:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-09-19T09:29:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-09-19T09:29:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-09-19T09:29:34.945 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-09-19T09:29:34.945 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-09-19T09:29:34.945 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-09-19T09:29:34.945 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-09-19T09:29:34.945 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-09-19T09:29:34.945 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-09-19T09:29:34.945 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-09-19T09:29:34.945 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-09-19T09:29:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-09-19T09:29:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-09-19T09:29:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-09-19T09:29:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-09-19T09:29:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-09-19T09:29:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-09-19T09:29:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-09-19T09:29:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-09-19T09:29:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-09-19T09:29:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-09-19T09:29:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-09-19T09:29:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-09-19T09:29:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-09-19T09:29:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-09-19T09:29:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-09-19T09:29:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-09-19T09:29:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-09-19T09:29:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-09-19T09:29:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-09-19T09:29:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-09-19T09:29:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-09-19T09:29:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-09-19T09:29:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-09-19T09:29:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-09-19T09:29:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-09-19T09:29:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-09-19T09:29:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-09-19T09:29:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-09-19T09:29:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-09-19T09:29:35.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:35 smithi016 ceph-mon[30979]: pgmap v397: 97 pgs: 97 active+clean; 318 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 812 KiB/s wr, 200 op/s 2024-09-19T09:29:35.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:35 smithi142 ceph-mon[35032]: pgmap v397: 97 pgs: 97 active+clean; 318 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 812 KiB/s wr, 200 op/s 2024-09-19T09:29:35.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:35 smithi097 ceph-mon[34864]: pgmap v397: 97 pgs: 97 active+clean; 318 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 812 KiB/s wr, 200 op/s 2024-09-19T09:29:35.662 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-09-19T09:29:35.662 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-09-19T09:29:35.662 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-09-19T09:29:35.662 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-09-19T09:29:35.662 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-09-19T09:29:35.662 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-09-19T09:29:35.662 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-09-19T09:29:35.662 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-09-19T09:29:35.662 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-09-19T09:29:35.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-09-19T09:29:35.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-09-19T09:29:35.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-09-19T09:29:35.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-09-19T09:29:35.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-09-19T09:29:35.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-09-19T09:29:35.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-09-19T09:29:35.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-09-19T09:29:35.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-09-19T09:29:35.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-09-19T09:29:35.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-09-19T09:29:35.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-09-19T09:29:35.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-09-19T09:29:35.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-09-19T09:29:35.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-09-19T09:29:35.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-09-19T09:29:35.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-09-19T09:29:35.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-09-19T09:29:35.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-09-19T09:29:35.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-09-19T09:29:35.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-09-19T09:29:35.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-09-19T09:29:35.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-09-19T09:29:35.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-09-19T09:29:35.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-09-19T09:29:35.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-09-19T09:29:35.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-09-19T09:29:35.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-09-19T09:29:35.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-09-19T09:29:35.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-09-19T09:29:35.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-09-19T09:29:35.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-09-19T09:29:35.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-09-19T09:29:35.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-09-19T09:29:35.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-09-19T09:29:35.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-09-19T09:29:35.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-09-19T09:29:35.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-09-19T09:29:35.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-09-19T09:29:35.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-09-19T09:29:35.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-09-19T09:29:35.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-09-19T09:29:35.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-09-19T09:29:35.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-09-19T09:29:35.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-09-19T09:29:35.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-09-19T09:29:35.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-09-19T09:29:35.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-09-19T09:29:35.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-09-19T09:29:35.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-09-19T09:29:35.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-09-19T09:29:35.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-09-19T09:29:35.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-09-19T09:29:35.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-09-19T09:29:35.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-09-19T09:29:35.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-09-19T09:29:35.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-09-19T09:29:35.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-09-19T09:29:35.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-09-19T09:29:36.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-09-19T09:29:36.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-09-19T09:29:36.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-09-19T09:29:36.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-09-19T09:29:36.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-09-19T09:29:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-09-19T09:29:36.152 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-09-19T09:29:36.152 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-09-19T09:29:36.152 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-09-19T09:29:36.152 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-09-19T09:29:36.152 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-09-19T09:29:36.152 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-09-19T09:29:36.152 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-09-19T09:29:36.152 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-09-19T09:29:36.152 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-09-19T09:29:36.152 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-09-19T09:29:36.152 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-09-19T09:29:36.152 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-09-19T09:29:36.152 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-09-19T09:29:36.153 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-09-19T09:29:36.154 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-09-19T09:29:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-09-19T09:29:36.156 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-09-19T09:29:36.156 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-09-19T09:29:36.156 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-09-19T09:29:36.156 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-09-19T09:29:36.156 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-09-19T09:29:36.156 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-09-19T09:29:36.578 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-09-19T09:29:36.578 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-09-19T09:29:36.578 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-09-19T09:29:36.578 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-09-19T09:29:36.578 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-09-19T09:29:36.578 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-09-19T09:29:36.578 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-09-19T09:29:36.578 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-09-19T09:29:36.578 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-09-19T09:29:36.578 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-09-19T09:29:36.578 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-09-19T09:29:36.579 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-09-19T09:29:36.580 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-09-19T09:29:36.580 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-09-19T09:29:36.580 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-09-19T09:29:36.580 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-09-19T09:29:36.580 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-09-19T09:29:36.580 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-09-19T09:29:36.580 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-09-19T09:29:36.580 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-09-19T09:29:36.580 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-09-19T09:29:36.580 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-09-19T09:29:36.580 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-09-19T09:29:36.580 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-09-19T09:29:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-09-19T09:29:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-09-19T09:29:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-09-19T09:29:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-09-19T09:29:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-09-19T09:29:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-09-19T09:29:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-09-19T09:29:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-09-19T09:29:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-09-19T09:29:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-09-19T09:29:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-09-19T09:29:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-09-19T09:29:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-09-19T09:29:36.582 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-09-19T09:29:36.582 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-09-19T09:29:36.582 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-09-19T09:29:36.582 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-09-19T09:29:36.582 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-09-19T09:29:36.582 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-09-19T09:29:36.582 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-09-19T09:29:36.582 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-09-19T09:29:36.582 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-09-19T09:29:36.582 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-09-19T09:29:36.583 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-09-19T09:29:36.583 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-09-19T09:29:36.583 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-09-19T09:29:36.583 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-09-19T09:29:36.583 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-09-19T09:29:36.583 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-09-19T09:29:36.584 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-09-19T09:29:36.584 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-09-19T09:29:36.584 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-09-19T09:29:36.584 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-09-19T09:29:36.585 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-09-19T09:29:36.585 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-09-19T09:29:36.585 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-09-19T09:29:36.585 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-09-19T09:29:36.585 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-09-19T09:29:36.586 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-09-19T09:29:36.586 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-09-19T09:29:36.586 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-09-19T09:29:36.586 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-09-19T09:29:36.586 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-09-19T09:29:36.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-09-19T09:29:36.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-09-19T09:29:36.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-09-19T09:29:36.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-09-19T09:29:36.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-09-19T09:29:36.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-09-19T09:29:36.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-09-19T09:29:36.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-09-19T09:29:36.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-09-19T09:29:36.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-09-19T09:29:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-09-19T09:29:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-09-19T09:29:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-09-19T09:29:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-09-19T09:29:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-09-19T09:29:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-09-19T09:29:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-09-19T09:29:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-09-19T09:29:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-09-19T09:29:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-09-19T09:29:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-09-19T09:29:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-09-19T09:29:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-09-19T09:29:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-09-19T09:29:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-09-19T09:29:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-09-19T09:29:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-09-19T09:29:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-09-19T09:29:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-09-19T09:29:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-09-19T09:29:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-09-19T09:29:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-09-19T09:29:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-09-19T09:29:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-09-19T09:29:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-09-19T09:29:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-09-19T09:29:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-09-19T09:29:36.918 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-09-19T09:29:36.919 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-09-19T09:29:36.919 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-09-19T09:29:37.346 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-09-19T09:29:37.346 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-09-19T09:29:37.346 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-09-19T09:29:37.346 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-09-19T09:29:37.346 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-09-19T09:29:37.346 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-09-19T09:29:37.346 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-09-19T09:29:37.346 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-09-19T09:29:37.346 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-09-19T09:29:37.346 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-09-19T09:29:37.346 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-09-19T09:29:37.347 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-09-19T09:29:37.348 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-09-19T09:29:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-09-19T09:29:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-09-19T09:29:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-09-19T09:29:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-09-19T09:29:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-09-19T09:29:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-09-19T09:29:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-09-19T09:29:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-09-19T09:29:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-09-19T09:29:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-09-19T09:29:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-09-19T09:29:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-09-19T09:29:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-09-19T09:29:37.350 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-09-19T09:29:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-09-19T09:29:37.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:37 smithi016 ceph-mon[30979]: pgmap v398: 97 pgs: 97 active+clean; 318 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 812 KiB/s wr, 196 op/s 2024-09-19T09:29:37.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:37 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:37.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:37 smithi142 ceph-mon[35032]: pgmap v398: 97 pgs: 97 active+clean; 318 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 812 KiB/s wr, 196 op/s 2024-09-19T09:29:37.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:37 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:37.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:37 smithi097 ceph-mon[34864]: pgmap v398: 97 pgs: 97 active+clean; 318 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 812 KiB/s wr, 196 op/s 2024-09-19T09:29:37.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:37 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T09:29:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.659+0000 7f2e0f16d640 1 -- 172.21.15.16:0/3012797020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08100a50 msgr2=0x7f2e08100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.659+0000 7f2e0f16d640 1 --2- 172.21.15.16:0/3012797020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08100a50 0x7f2e08100e50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009900 tx=0x7f2e0402a410 comp rx=0 tx=0).stop 2024-09-19T09:29:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.659+0000 7f2e0f16d640 1 -- 172.21.15.16:0/3012797020 shutdown_connections 2024-09-19T09:29:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.659+0000 7f2e0f16d640 1 --2- 172.21.15.16:0/3012797020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e08102600 0x7f2e08108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.659+0000 7f2e0f16d640 1 --2- 172.21.15.16:0/3012797020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e08101c40 0x7f2e081020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.659+0000 7f2e0f16d640 1 --2- 172.21.15.16:0/3012797020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08100a50 0x7f2e08100e50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.659+0000 7f2e0f16d640 1 -- 172.21.15.16:0/3012797020 >> 172.21.15.16:0/3012797020 conn(0x7f2e080fc1b0 msgr2=0x7f2e080fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.659+0000 7f2e0f16d640 1 -- 172.21.15.16:0/3012797020 shutdown_connections 2024-09-19T09:29:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.659+0000 7f2e0f16d640 1 -- 172.21.15.16:0/3012797020 wait complete. 2024-09-19T09:29:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.659+0000 7f2e0f16d640 1 Processor -- start 2024-09-19T09:29:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.659+0000 7f2e0f16d640 1 -- start start 2024-09-19T09:29:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0f16d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08100a50 0x7f2e08197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0f16d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e08101c40 0x7f2e08198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0f16d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e08102600 0x7f2e0819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0f16d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0810c730 con 0x7f2e08100a50 2024-09-19T09:29:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0f16d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0810c5b0 con 0x7f2e08101c40 2024-09-19T09:29:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0f16d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0810c8b0 con 0x7f2e08102600 2024-09-19T09:29:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0cf09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08100a50 0x7f2e08197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0cf09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08100a50 0x7f2e08197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53372/0 (socket says 172.21.15.16:53372) 2024-09-19T09:29:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0cf09640 1 -- 172.21.15.16:0/1489660922 learned_addr learned my addr 172.21.15.16:0/1489660922 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0d70a640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e08102600 0x7f2e0819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2dfffff640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e08101c40 0x7f2e08198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0cf09640 1 -- 172.21.15.16:0/1489660922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e08102600 msgr2=0x7f2e0819f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e08102600 0x7f2e0819f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0cf09640 1 -- 172.21.15.16:0/1489660922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e08101c40 msgr2=0x7f2e08198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e08101c40 0x7f2e08198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0cf09640 1 -- 172.21.15.16:0/1489660922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0819fa70 con 0x7f2e08100a50 2024-09-19T09:29:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08100a50 0x7f2e08197e50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2e04031610 tx=0x7f2e04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.660+0000 7f2dfdffb640 1 -- 172.21.15.16:0/1489660922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0403f030 con 0x7f2e08100a50 2024-09-19T09:29:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.661+0000 7f2e0f16d640 1 -- 172.21.15.16:0/1489660922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0819fd00 con 0x7f2e08100a50 2024-09-19T09:29:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.661+0000 7f2dfdffb640 1 -- 172.21.15.16:0/1489660922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e04003ad0 con 0x7f2e08100a50 2024-09-19T09:29:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.661+0000 7f2dfdffb640 1 -- 172.21.15.16:0/1489660922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04035070 con 0x7f2e08100a50 2024-09-19T09:29:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.661+0000 7f2e0f16d640 1 -- 172.21.15.16:0/1489660922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e081a0190 con 0x7f2e08100a50 2024-09-19T09:29:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.663+0000 7f2dfdffb640 1 -- 172.21.15.16:0/1489660922 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f2e04042030 con 0x7f2e08100a50 2024-09-19T09:29:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2e11fe9740 1 Processor -- start 2024-09-19T09:29:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2e11fe9740 1 -- start start 2024-09-19T09:29:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2e11fe9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 0x55a162f05da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2e11fe9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f062e0 0x55a162f0ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2dfdffb640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2df00a32f0 0x7f2df00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2e11fe9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f0b370 0x55a162f0d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2e11fe9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a162e692f0 con 0x55a162f059a0 2024-09-19T09:29:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2e11fe9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a162e11270 con 0x55a162f0b370 2024-09-19T09:29:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2e11fe9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a162e652b0 con 0x55a162f062e0 2024-09-19T09:29:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2e0cf09640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f062e0 0x55a162f0ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2e0d70a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 0x55a162f05da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2dfffff640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2df00a32f0 0x7f2df00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2e0d70a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 0x55a162f05da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53378/0 (socket says 172.21.15.16:53378) 2024-09-19T09:29:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2e0d70a640 1 -- 172.21.15.16:0/2388969350 learned_addr learned my addr 172.21.15.16:0/2388969350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2dfdffb640 1 -- 172.21.15.16:0/1489660922 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2e040effe0 con 0x7f2e08100a50 2024-09-19T09:29:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.664+0000 7f2e0d70a640 1 --2- 172.21.15.16:0/2388969350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f0b370 0x55a162f0d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.665+0000 7f2e0cf09640 1 -- 172.21.15.16:0/2388969350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f0b370 msgr2=0x55a162f0d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.665+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/2388969350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f0b370 0x55a162f0d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.665+0000 7f2e0cf09640 1 -- 172.21.15.16:0/2388969350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 msgr2=0x55a162f05da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.665+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/2388969350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 0x55a162f05da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.665+0000 7f2e0cf09640 1 -- 172.21.15.16:0/2388969350 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a162e10340 con 0x55a162f062e0 2024-09-19T09:29:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.665+0000 7f2e0d70a640 1 --2- 172.21.15.16:0/2388969350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 0x55a162f05da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.665+0000 7f2dfffff640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2df00a32f0 0x7f2df00a57b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2df80095f0 tx=0x7f2df8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.665+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/2388969350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f062e0 0x55a162f0ada0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e040c4d40 tx=0x7f2e04040fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ff92fc9417d55ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.665+0000 7f2deeffd640 1 -- 172.21.15.16:0/2388969350 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04049040 con 0x55a162f062e0 2024-09-19T09:29:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.665+0000 7f2deeffd640 1 -- 172.21.15.16:0/2388969350 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e040bc6b0 con 0x55a162f062e0 2024-09-19T09:29:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.665+0000 7f2deeffd640 1 -- 172.21.15.16:0/2388969350 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04041e70 con 0x55a162f062e0 2024-09-19T09:29:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.666+0000 7f2e11fe9740 1 -- 172.21.15.16:0/2388969350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f062e0 msgr2=0x55a162f0ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.666+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/2388969350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f062e0 0x55a162f0ada0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e040c4d40 tx=0x7f2e04040fb0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.666+0000 7f2e11fe9740 1 -- 172.21.15.16:0/2388969350 shutdown_connections 2024-09-19T09:29:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.666+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/2388969350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f062e0 0x55a162f0ada0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.666+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/2388969350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f0b370 0x55a162f0d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.666+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/2388969350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 0x55a162f05da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.666+0000 7f2e11fe9740 1 -- 172.21.15.16:0/2388969350 >> 172.21.15.16:0/2388969350 conn(0x55a162f01010 msgr2=0x55a162f01410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.666+0000 7f2e11fe9740 1 -- 172.21.15.16:0/2388969350 shutdown_connections 2024-09-19T09:29:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.666+0000 7f2e11fe9740 1 -- 172.21.15.16:0/2388969350 wait complete. 2024-09-19T09:29:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.666+0000 7f2e11fe9740 1 -- start start 2024-09-19T09:29:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.666+0000 7f2e11fe9740 1 Processor -- start 2024-09-19T09:29:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e11fe9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 0x55a162f086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e11fe9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f062e0 0x55a162f08bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e11fe9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f0b370 0x55a162f09c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e11fe9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a162f0e020 con 0x55a162f059a0 2024-09-19T09:29:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e11fe9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a162df2290 con 0x55a162f062e0 2024-09-19T09:29:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e11fe9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a162f0e1a0 con 0x55a162f0b370 2024-09-19T09:29:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e11fe9740 10 client.0 _ll_register_callbacks cb 0x55a162e23140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e0cf09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 0x55a162f086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e0d70a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f0b370 0x55a162f09c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2dfffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f062e0 0x55a162f08bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e0cf09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 0x55a162f086b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53386/0 (socket says 172.21.15.16:53386) 2024-09-19T09:29:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e0d70a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f0b370 0x55a162f09c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:49358/0 (socket says 172.21.15.16:49358) 2024-09-19T09:29:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e0cf09640 1 -- 172.21.15.16:0/4004532761 learned_addr learned my addr 172.21.15.16:0/4004532761 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2dfffff640 1 --2- 172.21.15.16:0/4004532761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f062e0 0x55a162f08bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:55020/0 (socket says 172.21.15.16:55020) 2024-09-19T09:29:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e0cf09640 1 -- 172.21.15.16:0/4004532761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f0b370 msgr2=0x55a162f09c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/4004532761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f0b370 0x55a162f09c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e0cf09640 1 -- 172.21.15.16:0/4004532761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f062e0 msgr2=0x55a162f08bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/4004532761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f062e0 0x55a162f08bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.667+0000 7f2e0cf09640 1 -- 172.21.15.16:0/4004532761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a162b7c210 con 0x55a162f059a0 2024-09-19T09:29:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/4004532761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 0x55a162f086b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2e04000c00 tx=0x7f2e040edc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2329482849dbfaed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2de7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2de7fff640 1 -- 172.21.15.16:0/4004532761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e040d5020 con 0x55a162f059a0 2024-09-19T09:29:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4004532761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e040bc6b0 con 0x55a162f059a0 2024-09-19T09:29:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2de7fff640 1 -- 172.21.15.16:0/4004532761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2e040b8d20 con 0x55a162f059a0 2024-09-19T09:29:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2de7fff640 1 -- 172.21.15.16:0/4004532761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e04049070 con 0x55a162f059a0 2024-09-19T09:29:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2e11fe9740 10 client.16032 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4004532761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a162c48620 con 0x55a162f059a0 2024-09-19T09:29:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2de7fff640 1 -- 172.21.15.16:0/4004532761 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e040bc6b0 con 0x55a162f059a0 2024-09-19T09:29:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2de7fff640 1 -- 172.21.15.16:0/4004532761 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2e04049c40 con 0x55a162f059a0 2024-09-19T09:29:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2e11fe9740 10 client.16032 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2e11fe9740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2e11fe9740 20 client.16032 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/4004532761 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a162f1b4d0 0x55a162f1d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2e11fe9740 4 client.16032 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.668+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4004532761 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a162e76a20 con 0x55a162f1b4d0 2024-09-19T09:29:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.669+0000 7f2dfffff640 1 --2- 172.21.15.16:0/4004532761 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a162f1b4d0 0x55a162f1d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.669+0000 7f2de7fff640 10 client.16032 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.669+0000 7f2dfffff640 1 --2- 172.21.15.16:0/4004532761 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a162f1b4d0 0x55a162f1d8c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f89cebfe2a2c7f9 server_cookie=504325808f9af192 in_seq=0 out_seq=0 2024-09-19T09:29:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2de7fff640 1 -- 172.21.15.16:0/4004532761 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a162e76a20 con 0x55a162f1b4d0 2024-09-19T09:29:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2de7fff640 10 client.16032 handle_command_reply: tid=0 2024-09-19T09:29:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2e11fe9740 1 client.16032 shutdown 2024-09-19T09:29:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4004532761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 msgr2=0x55a162f086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/4004532761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 0x55a162f086b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2e04000c00 tx=0x7f2e040edc70 comp rx=0 tx=0).stop 2024-09-19T09:29:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4004532761 shutdown_connections 2024-09-19T09:29:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/4004532761 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a162f1b4d0 0x55a162f1d8c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/4004532761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f0b370 0x55a162f09c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2de7fff640 0 client.16032 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/4004532761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f062e0 0x55a162f08bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/4004532761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f059a0 0x55a162f086b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4004532761 >> 172.21.15.16:0/4004532761 conn(0x55a162f01010 msgr2=0x55a162f02820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4004532761 shutdown_connections 2024-09-19T09:29:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4004532761 wait complete. 2024-09-19T09:29:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.670+0000 7f2e11fe9740 20 client.16032 trim_cache size 0 max 16384 2024-09-19T09:29:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.687+0000 7f2e11fe9740 1 Processor -- start 2024-09-19T09:29:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.687+0000 7f2e11fe9740 1 -- start start 2024-09-19T09:29:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.687+0000 7f2e11fe9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f059a0 0x55a162f08c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.687+0000 7f2e11fe9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f062e0 0x55a162f07420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.687+0000 7f2e11fe9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f0b370 0x55a162f01ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.687+0000 7f2e11fe9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081a6ec0 con 0x55a162f0b370 2024-09-19T09:29:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.687+0000 7f2e11fe9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081a8a00 con 0x55a162f059a0 2024-09-19T09:29:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.687+0000 7f2e11fe9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081a7590 con 0x55a162f062e0 2024-09-19T09:29:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.687+0000 7f2e0d70a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f0b370 0x55a162f01ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.687+0000 7f2e0cf09640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f059a0 0x55a162f08c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.687+0000 7f2e0cf09640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f059a0 0x55a162f08c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:55028/0 (socket says 172.21.15.16:55028) 2024-09-19T09:29:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.687+0000 7f2e0cf09640 1 -- 172.21.15.16:0/1301793408 learned_addr learned my addr 172.21.15.16:0/1301793408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.687+0000 7f2dfffff640 1 --2- 172.21.15.16:0/1301793408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f062e0 0x55a162f07420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.688+0000 7f2e0cf09640 1 -- 172.21.15.16:0/1301793408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f062e0 msgr2=0x55a162f07420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.688+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/1301793408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f062e0 0x55a162f07420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.688+0000 7f2e0cf09640 1 -- 172.21.15.16:0/1301793408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f0b370 msgr2=0x55a162f01ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.688+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/1301793408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f0b370 0x55a162f01ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.688+0000 7f2e0cf09640 1 -- 172.21.15.16:0/1301793408 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a162f1df40 con 0x55a162f059a0 2024-09-19T09:29:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.688+0000 7f2e0d70a640 1 --2- 172.21.15.16:0/1301793408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f0b370 0x55a162f01ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.688+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/1301793408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f059a0 0x55a162f08c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2e040ba960 tx=0x7f2e040c4e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945c33d5112f5e4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.688+0000 7f2dee7fc640 1 -- 172.21.15.16:0/1301793408 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e040d5040 con 0x55a162f059a0 2024-09-19T09:29:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.688+0000 7f2dee7fc640 1 -- 172.21.15.16:0/1301793408 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e040bc6b0 con 0x55a162f059a0 2024-09-19T09:29:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.688+0000 7f2dee7fc640 1 -- 172.21.15.16:0/1301793408 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e040c4ed0 con 0x55a162f059a0 2024-09-19T09:29:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.689+0000 7f2e11fe9740 1 -- 172.21.15.16:0/1301793408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f059a0 msgr2=0x55a162f08c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.689+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/1301793408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f059a0 0x55a162f08c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2e040ba960 tx=0x7f2e040c4e60 comp rx=0 tx=0).stop 2024-09-19T09:29:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.689+0000 7f2e11fe9740 1 -- 172.21.15.16:0/1301793408 shutdown_connections 2024-09-19T09:29:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.689+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/1301793408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f062e0 0x55a162f07420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.689+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/1301793408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f0b370 0x55a162f01ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.689+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/1301793408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f059a0 0x55a162f08c80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.697 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.689+0000 7f2e11fe9740 1 -- 172.21.15.16:0/1301793408 >> 172.21.15.16:0/1301793408 conn(0x55a162f01010 msgr2=0x55a162f06d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.689+0000 7f2e11fe9740 1 -- 172.21.15.16:0/1301793408 shutdown_connections 2024-09-19T09:29:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.689+0000 7f2e11fe9740 1 -- 172.21.15.16:0/1301793408 wait complete. 2024-09-19T09:29:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.689+0000 7f2e11fe9740 1 -- start start 2024-09-19T09:29:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.689+0000 7f2e11fe9740 1 Processor -- start 2024-09-19T09:29:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2e11fe9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f059a0 0x55a162f15570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2e11fe9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f062e0 0x55a162f15ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2e11fe9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f0b370 0x55a162f16ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2e11fe9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081a8330 con 0x55a162f0b370 2024-09-19T09:29:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2e11fe9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081a7c60 con 0x55a162f062e0 2024-09-19T09:29:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2e11fe9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081a4e80 con 0x55a162f059a0 2024-09-19T09:29:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2e11fe9740 10 client.0 _ll_register_callbacks cb 0x7f2e00009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2dfffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f062e0 0x55a162f15ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2e0d70a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f0b370 0x55a162f16ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2e0cf09640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f059a0 0x55a162f15570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2dfffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f062e0 0x55a162f15ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:55042/0 (socket says 172.21.15.16:55042) 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2dfffff640 1 -- 172.21.15.16:0/4186687762 learned_addr learned my addr 172.21.15.16:0/4186687762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2dfffff640 1 -- 172.21.15.16:0/4186687762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f059a0 msgr2=0x55a162f15570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2dfffff640 1 --2- 172.21.15.16:0/4186687762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f059a0 0x55a162f15570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2dfffff640 1 -- 172.21.15.16:0/4186687762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f0b370 msgr2=0x55a162f16ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2dfffff640 1 --2- 172.21.15.16:0/4186687762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f0b370 0x55a162f16ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2dfffff640 1 -- 172.21.15.16:0/4186687762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a162f170d0 con 0x55a162f062e0 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.690+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/4186687762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f059a0 0x55a162f15570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2dfffff640 1 --2- 172.21.15.16:0/4186687762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f062e0 0x55a162f15ab0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55a162f01e90 tx=0x7f2df8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da895325f141eba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2de7fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2de7fff640 1 -- 172.21.15.16:0/4186687762 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80275c0 con 0x55a162f062e0 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4186687762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e040bc6b0 con 0x55a162f062e0 2024-09-19T09:29:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2de7fff640 1 -- 172.21.15.16:0/4186687762 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f2df8027760 con 0x55a162f062e0 2024-09-19T09:29:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2de7fff640 1 -- 172.21.15.16:0/4186687762 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2df8027960 con 0x55a162f062e0 2024-09-19T09:29:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2e11fe9740 10 client.25601 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4186687762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a162f86200 con 0x55a162f062e0 2024-09-19T09:29:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2de7fff640 1 -- 172.21.15.16:0/4186687762 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e040bc6b0 con 0x55a162f062e0 2024-09-19T09:29:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2de7fff640 1 -- 172.21.15.16:0/4186687762 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2df80393b0 con 0x55a162f062e0 2024-09-19T09:29:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2e11fe9740 10 client.25601 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2e11fe9740 10 client.25601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2e11fe9740 20 client.25601 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/4186687762 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a162f86b90 0x55a162f88fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2e11fe9740 4 client.25601 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T09:29:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.691+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4186687762 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a162e70d70 con 0x55a162f86b90 2024-09-19T09:29:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.692+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/4186687762 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a162f86b90 0x55a162f88fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.692+0000 7f2e0cf09640 1 --2- 172.21.15.16:0/4186687762 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a162f86b90 0x55a162f88fb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1aff8a7691f60f7 server_cookie=630d821f8b7f6974 in_seq=0 out_seq=0 2024-09-19T09:29:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.692+0000 7f2de7fff640 10 client.25601 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2de7fff640 1 -- 172.21.15.16:0/4186687762 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a162e70d70 con 0x55a162f86b90 2024-09-19T09:29:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2de7fff640 10 client.25601 handle_command_reply: tid=0 2024-09-19T09:29:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2e11fe9740 1 client.25601 shutdown 2024-09-19T09:29:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4186687762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f062e0 msgr2=0x55a162f15ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/4186687762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f062e0 0x55a162f15ab0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55a162f01e90 tx=0x7f2df8004730 comp rx=0 tx=0).stop 2024-09-19T09:29:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4186687762 shutdown_connections 2024-09-19T09:29:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/4186687762 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55a162f86b90 0x55a162f88fb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/4186687762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a162f0b370 0x55a162f16ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2de7fff640 0 client.25601 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/4186687762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a162f062e0 0x55a162f15ab0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2e11fe9740 1 --2- 172.21.15.16:0/4186687762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a162f059a0 0x55a162f15570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4186687762 >> 172.21.15.16:0/4186687762 conn(0x55a162f01010 msgr2=0x55a162f073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4186687762 shutdown_connections 2024-09-19T09:29:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2e11fe9740 1 -- 172.21.15.16:0/4186687762 wait complete. 2024-09-19T09:29:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.741+0000 7f2e11fe9740 20 client.25601 trim_cache size 0 max 16384 2024-09-19T09:29:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:{ 2024-09-19T09:29:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "message": "", 2024-09-19T09:29:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout: "return_code": 0 2024-09-19T09:29:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stdout:} 2024-09-19T09:29:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.742+0000 7f2e0f16d640 1 -- 172.21.15.16:0/1489660922 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2df00a32f0 msgr2=0x7f2df00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.742+0000 7f2e0f16d640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2df00a32f0 0x7f2df00a57b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2df80095f0 tx=0x7f2df8005f90 comp rx=0 tx=0).stop 2024-09-19T09:29:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.742+0000 7f2e0f16d640 1 -- 172.21.15.16:0/1489660922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08100a50 msgr2=0x7f2e08197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.742+0000 7f2e0f16d640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08100a50 0x7f2e08197e50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2e04031610 tx=0x7f2e04004270 comp rx=0 tx=0).stop 2024-09-19T09:29:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.742+0000 7f2e0f16d640 1 -- 172.21.15.16:0/1489660922 shutdown_connections 2024-09-19T09:29:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.742+0000 7f2e0f16d640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f2df00a32f0 0x7f2df00a57b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.742+0000 7f2e0f16d640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e08102600 0x7f2e0819f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.742+0000 7f2e0f16d640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e08101c40 0x7f2e08198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.742+0000 7f2e0f16d640 1 --2- 172.21.15.16:0/1489660922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e08100a50 0x7f2e08197e50 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.742+0000 7f2e0f16d640 1 -- 172.21.15.16:0/1489660922 >> 172.21.15.16:0/1489660922 conn(0x7f2e080fc1b0 msgr2=0x7f2e0806a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.742+0000 7f2e0f16d640 1 -- 172.21.15.16:0/1489660922 shutdown_connections 2024-09-19T09:29:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi016.front.sepia.ceph.com.smithi016.stderr:2024-09-19T09:29:37.742+0000 7f2e0f16d640 1 -- 172.21.15.16:0/1489660922 wait complete. 2024-09-19T09:29:37.909 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-09-19T09:29:37.910 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-09-19T09:29:37.910 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-09-19T09:29:37.910 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-09-19T09:29:37.910 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-09-19T09:29:37.910 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-09-19T09:29:37.910 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-09-19T09:29:37.910 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-09-19T09:29:37.910 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-09-19T09:29:37.910 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-09-19T09:29:37.910 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-09-19T09:29:37.910 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-09-19T09:29:37.910 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-09-19T09:29:37.911 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-09-19T09:29:37.911 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-09-19T09:29:37.911 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-09-19T09:29:37.911 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-09-19T09:29:37.911 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-09-19T09:29:37.911 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-09-19T09:29:37.911 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-09-19T09:29:37.911 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-09-19T09:29:37.911 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-09-19T09:29:37.911 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-09-19T09:29:37.911 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-09-19T09:29:37.911 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-09-19T09:29:37.911 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-09-19T09:29:37.912 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-09-19T09:29:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-09-19T09:29:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-09-19T09:29:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-09-19T09:29:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-09-19T09:29:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-09-19T09:29:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-09-19T09:29:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-09-19T09:29:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-09-19T09:29:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-09-19T09:29:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-09-19T09:29:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-09-19T09:29:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-09-19T09:29:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-09-19T09:29:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-09-19T09:29:37.915 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-09-19T09:29:38.233 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-09-19T09:29:38.233 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-09-19T09:29:38.233 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-09-19T09:29:38.233 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-09-19T09:29:38.233 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-09-19T09:29:38.233 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-09-19T09:29:38.233 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-09-19T09:29:38.233 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-09-19T09:29:38.234 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-09-19T09:29:38.234 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-09-19T09:29:38.234 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-09-19T09:29:38.234 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-09-19T09:29:38.234 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-09-19T09:29:38.234 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-09-19T09:29:38.234 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-09-19T09:29:38.234 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-09-19T09:29:38.234 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-09-19T09:29:38.235 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-09-19T09:29:38.235 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-09-19T09:29:38.235 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-09-19T09:29:38.235 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-09-19T09:29:38.235 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-09-19T09:29:38.235 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-09-19T09:29:38.235 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-09-19T09:29:38.235 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-09-19T09:29:38.235 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-09-19T09:29:38.235 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-09-19T09:29:38.235 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-09-19T09:29:38.235 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-09-19T09:29:38.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-09-19T09:29:38.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-09-19T09:29:38.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-09-19T09:29:38.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-09-19T09:29:38.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-09-19T09:29:38.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-09-19T09:29:38.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-09-19T09:29:38.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-09-19T09:29:38.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-09-19T09:29:38.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-09-19T09:29:38.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-09-19T09:29:38.236 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-09-19T09:29:38.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-09-19T09:29:38.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-09-19T09:29:38.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-09-19T09:29:38.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-09-19T09:29:38.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-09-19T09:29:38.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-09-19T09:29:38.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-09-19T09:29:38.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-09-19T09:29:38.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-09-19T09:29:38.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-09-19T09:29:38.237 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-09-19T09:29:38.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-09-19T09:29:38.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-09-19T09:29:38.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-09-19T09:29:38.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-09-19T09:29:38.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-09-19T09:29:38.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-09-19T09:29:38.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-09-19T09:29:38.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-09-19T09:29:38.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-09-19T09:29:38.238 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-09-19T09:29:38.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-09-19T09:29:38.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-09-19T09:29:38.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-09-19T09:29:38.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-09-19T09:29:38.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-09-19T09:29:38.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-09-19T09:29:38.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-09-19T09:29:38.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-09-19T09:29:38.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-09-19T09:29:38.239 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-09-19T09:29:38.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-09-19T09:29:38.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-09-19T09:29:38.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-09-19T09:29:38.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-09-19T09:29:38.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-09-19T09:29:38.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-09-19T09:29:38.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-09-19T09:29:38.240 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-09-19T09:29:38.602 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-09-19T09:29:38.602 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-09-19T09:29:38.602 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-09-19T09:29:38.602 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-09-19T09:29:38.602 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-09-19T09:29:38.602 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-09-19T09:29:38.602 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-09-19T09:29:38.602 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-09-19T09:29:38.602 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-09-19T09:29:38.602 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-09-19T09:29:38.603 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-09-19T09:29:38.603 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-09-19T09:29:38.603 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-09-19T09:29:38.603 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-09-19T09:29:38.603 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-09-19T09:29:38.603 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-09-19T09:29:38.603 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-09-19T09:29:38.603 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-09-19T09:29:38.603 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-09-19T09:29:38.603 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-09-19T09:29:38.603 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-09-19T09:29:38.603 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-09-19T09:29:38.604 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-09-19T09:29:38.604 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-09-19T09:29:38.604 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-09-19T09:29:38.604 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-09-19T09:29:38.604 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-09-19T09:29:38.604 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-09-19T09:29:38.604 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-09-19T09:29:38.604 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-09-19T09:29:38.604 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-09-19T09:29:38.604 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-09-19T09:29:38.604 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-09-19T09:29:38.605 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-09-19T09:29:38.605 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-09-19T09:29:38.605 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-09-19T09:29:38.605 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-09-19T09:29:38.605 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-09-19T09:29:38.605 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-09-19T09:29:38.605 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-09-19T09:29:38.605 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-09-19T09:29:38.605 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-09-19T09:29:38.605 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-09-19T09:29:38.605 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-09-19T09:29:38.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-09-19T09:29:38.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-09-19T09:29:38.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-09-19T09:29:38.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-09-19T09:29:38.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-09-19T09:29:38.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-09-19T09:29:38.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-09-19T09:29:38.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-09-19T09:29:38.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-09-19T09:29:38.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-09-19T09:29:38.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-09-19T09:29:38.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-09-19T09:29:38.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-09-19T09:29:38.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-09-19T09:29:38.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-09-19T09:29:38.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-09-19T09:29:38.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-09-19T09:29:38.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-09-19T09:29:38.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-09-19T09:29:38.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-09-19T09:29:38.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-09-19T09:29:38.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-09-19T09:29:38.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-09-19T09:29:38.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-09-19T09:29:38.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-09-19T09:29:38.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-09-19T09:29:38.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-09-19T09:29:38.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-09-19T09:29:38.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-09-19T09:29:38.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-09-19T09:29:38.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-09-19T09:29:38.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-09-19T09:29:38.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-09-19T09:29:38.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-09-19T09:29:38.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-09-19T09:29:38.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-09-19T09:29:38.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-09-19T09:29:38.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:38 smithi016 ceph-mon[30979]: pgmap v399: 97 pgs: 97 active+clean; 318 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s rd, 1.3 MiB/s wr, 368 op/s 2024-09-19T09:29:38.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:38.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:38.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:38 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:38 smithi142 ceph-mon[35032]: pgmap v399: 97 pgs: 97 active+clean; 318 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s rd, 1.3 MiB/s wr, 368 op/s 2024-09-19T09:29:38.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:38.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:38.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:38 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:38 smithi097 ceph-mon[34864]: pgmap v399: 97 pgs: 97 active+clean; 318 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s rd, 1.3 MiB/s wr, 368 op/s 2024-09-19T09:29:38.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:38.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T09:29:38.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T09:29:38.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:38 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' 2024-09-19T09:29:39.064 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-09-19T09:29:39.065 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-09-19T09:29:39.065 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-09-19T09:29:39.065 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-09-19T09:29:39.065 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-09-19T09:29:39.065 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-09-19T09:29:39.065 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-09-19T09:29:39.065 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-09-19T09:29:39.065 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-09-19T09:29:39.065 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-09-19T09:29:39.065 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-09-19T09:29:39.066 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-09-19T09:29:39.066 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-09-19T09:29:39.066 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-09-19T09:29:39.066 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-09-19T09:29:39.066 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-09-19T09:29:39.066 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-09-19T09:29:39.066 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-09-19T09:29:39.066 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-09-19T09:29:39.066 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-09-19T09:29:39.066 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-09-19T09:29:39.066 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-09-19T09:29:39.067 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-09-19T09:29:39.067 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-09-19T09:29:39.067 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-09-19T09:29:39.067 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-09-19T09:29:39.067 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-09-19T09:29:39.067 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-09-19T09:29:39.067 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-09-19T09:29:39.067 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-09-19T09:29:39.067 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-09-19T09:29:39.067 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-09-19T09:29:39.067 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-09-19T09:29:39.067 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-09-19T09:29:39.068 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-09-19T09:29:39.068 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-09-19T09:29:39.068 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-09-19T09:29:39.068 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-09-19T09:29:39.068 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-09-19T09:29:39.068 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-09-19T09:29:39.068 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-09-19T09:29:39.068 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-09-19T09:29:39.068 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-09-19T09:29:39.068 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-09-19T09:29:39.068 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-09-19T09:29:39.069 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-09-19T09:29:39.069 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-09-19T09:29:39.069 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-09-19T09:29:39.069 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-09-19T09:29:39.069 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-09-19T09:29:39.069 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-09-19T09:29:39.069 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-09-19T09:29:39.069 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-09-19T09:29:39.069 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-09-19T09:29:39.069 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-09-19T09:29:39.069 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-09-19T09:29:39.069 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-09-19T09:29:39.069 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-09-19T09:29:39.070 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-09-19T09:29:39.070 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-09-19T09:29:39.070 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-09-19T09:29:39.070 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-09-19T09:29:39.070 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-09-19T09:29:39.070 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-09-19T09:29:39.070 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-09-19T09:29:39.070 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-09-19T09:29:39.070 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-09-19T09:29:39.070 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-09-19T09:29:39.070 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-09-19T09:29:39.070 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-09-19T09:29:39.070 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-09-19T09:29:39.071 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-09-19T09:29:39.071 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-09-19T09:29:39.071 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-09-19T09:29:39.071 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-09-19T09:29:39.071 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-09-19T09:29:39.071 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-09-19T09:29:39.071 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-09-19T09:29:39.071 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-09-19T09:29:39.071 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-09-19T09:29:39.071 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-09-19T09:29:39.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-09-19T09:29:39.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-09-19T09:29:39.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-09-19T09:29:39.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-09-19T09:29:39.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-09-19T09:29:39.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-09-19T09:29:39.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-09-19T09:29:39.663 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-09-19T09:29:39.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-09-19T09:29:39.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-09-19T09:29:39.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-09-19T09:29:39.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-09-19T09:29:39.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-09-19T09:29:39.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-09-19T09:29:39.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-09-19T09:29:39.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-09-19T09:29:39.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-09-19T09:29:39.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-09-19T09:29:39.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-09-19T09:29:39.664 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-09-19T09:29:39.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-09-19T09:29:39.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-09-19T09:29:39.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-09-19T09:29:39.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-09-19T09:29:39.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-09-19T09:29:39.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-09-19T09:29:39.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-09-19T09:29:39.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-09-19T09:29:39.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-09-19T09:29:39.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-09-19T09:29:39.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-09-19T09:29:39.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-09-19T09:29:39.665 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-09-19T09:29:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-09-19T09:29:39.667 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-09-19T09:29:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-09-19T09:29:39.669 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-09-19T09:29:39.669 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-09-19T09:29:39.669 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-09-19T09:29:39.669 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-09-19T09:29:39.669 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-09-19T09:29:39.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T09:29:39.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-09-19T09:29:39.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-09-19T09:29:39.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-09-19T09:29:39.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-09-19T09:29:39.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-09-19T09:29:39.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-09-19T09:29:39.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-09-19T09:29:39.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-09-19T09:29:39.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-09-19T09:29:39.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-09-19T09:29:39.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-09-19T09:29:39.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-09-19T09:29:39.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-09-19T09:29:39.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-09-19T09:29:39.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-09-19T09:29:39.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-09-19T09:29:39.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-09-19T09:29:39.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-09-19T09:29:39.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-09-19T09:29:39.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-09-19T09:29:39.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-09-19T09:29:39.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-09-19T09:29:39.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-09-19T09:29:39.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-09-19T09:29:39.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-09-19T09:29:39.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-09-19T09:29:39.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-09-19T09:29:39.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-09-19T09:29:39.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-09-19T09:29:39.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-09-19T09:29:39.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-09-19T09:29:39.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-09-19T09:29:39.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-09-19T09:29:39.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-09-19T09:29:39.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-09-19T09:29:39.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-09-19T09:29:39.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-09-19T09:29:39.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-09-19T09:29:39.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-09-19T09:29:39.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-09-19T09:29:39.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-09-19T09:29:39.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-09-19T09:29:39.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-09-19T09:29:39.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-09-19T09:29:39.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-09-19T09:29:39.974 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-09-19T09:29:39.974 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-09-19T09:29:39.974 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-09-19T09:29:39.974 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-09-19T09:29:39.974 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-09-19T09:29:39.974 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-09-19T09:29:39.975 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-09-19T09:29:39.975 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-09-19T09:29:39.975 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-09-19T09:29:39.975 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-09-19T09:29:39.975 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-09-19T09:29:39.975 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-09-19T09:29:39.975 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-09-19T09:29:39.975 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-09-19T09:29:39.975 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-09-19T09:29:39.975 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-09-19T09:29:39.975 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-09-19T09:29:39.975 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-09-19T09:29:39.976 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-09-19T09:29:39.976 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-09-19T09:29:39.976 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-09-19T09:29:39.976 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-09-19T09:29:39.976 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-09-19T09:29:39.976 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-09-19T09:29:39.976 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-09-19T09:29:39.976 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-09-19T09:29:39.976 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-09-19T09:29:39.976 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-09-19T09:29:39.976 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-09-19T09:29:39.977 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-09-19T09:29:39.977 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-09-19T09:29:39.977 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-09-19T09:29:39.977 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-09-19T09:29:39.977 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-09-19T09:29:39.977 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-09-19T09:29:39.977 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-09-19T09:29:40.275 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:40.329 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-09-19T09:29:40.329 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-09-19T09:29:40.329 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-09-19T09:29:40.329 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-09-19T09:29:40.329 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-09-19T09:29:40.329 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-09-19T09:29:40.329 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-09-19T09:29:40.329 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-09-19T09:29:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-09-19T09:29:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-09-19T09:29:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-09-19T09:29:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-09-19T09:29:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-09-19T09:29:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-09-19T09:29:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-09-19T09:29:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-09-19T09:29:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-09-19T09:29:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-09-19T09:29:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-09-19T09:29:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-09-19T09:29:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-09-19T09:29:40.331 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-09-19T09:29:40.332 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-09-19T09:29:40.332 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-09-19T09:29:40.332 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-09-19T09:29:40.332 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-09-19T09:29:40.332 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-09-19T09:29:40.332 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-09-19T09:29:40.332 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-09-19T09:29:40.332 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-09-19T09:29:40.332 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-09-19T09:29:40.332 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-09-19T09:29:40.332 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-09-19T09:29:40.333 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-09-19T09:29:40.333 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-09-19T09:29:40.333 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-09-19T09:29:40.333 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-09-19T09:29:40.333 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-09-19T09:29:40.333 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-09-19T09:29:40.333 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-09-19T09:29:40.333 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-09-19T09:29:40.333 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-09-19T09:29:40.333 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-09-19T09:29:40.333 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-09-19T09:29:40.333 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-09-19T09:29:40.334 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-09-19T09:29:40.334 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-09-19T09:29:40.334 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-09-19T09:29:40.334 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-09-19T09:29:40.334 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-09-19T09:29:40.334 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-09-19T09:29:40.334 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-09-19T09:29:40.334 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-09-19T09:29:40.334 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-09-19T09:29:40.334 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-09-19T09:29:40.334 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-09-19T09:29:40.335 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-09-19T09:29:40.335 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-09-19T09:29:40.335 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-09-19T09:29:40.335 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-09-19T09:29:40.335 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-09-19T09:29:40.335 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-09-19T09:29:40.335 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-09-19T09:29:40.335 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-09-19T09:29:40.335 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-09-19T09:29:40.335 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-09-19T09:29:40.335 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-09-19T09:29:40.676 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:40 smithi016 ceph-mon[30979]: pgmap v400: 97 pgs: 97 active+clean; 392 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 106 MiB/s rd, 1.6 MiB/s wr, 469 op/s 2024-09-19T09:29:40.867 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-09-19T09:29:40.867 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-09-19T09:29:40.867 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-09-19T09:29:40.867 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-09-19T09:29:40.867 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-09-19T09:29:40.868 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-09-19T09:29:40.868 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-09-19T09:29:40.868 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-09-19T09:29:40.868 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-09-19T09:29:40.868 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-09-19T09:29:40.868 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-09-19T09:29:40.868 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-09-19T09:29:40.868 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-09-19T09:29:40.868 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-09-19T09:29:40.868 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-09-19T09:29:40.868 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-09-19T09:29:40.868 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-09-19T09:29:40.868 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-09-19T09:29:40.869 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-09-19T09:29:40.869 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-09-19T09:29:40.869 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-09-19T09:29:40.869 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-09-19T09:29:40.869 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-09-19T09:29:40.869 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-09-19T09:29:40.869 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-09-19T09:29:40.869 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-09-19T09:29:40.869 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-09-19T09:29:40.869 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-09-19T09:29:40.869 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-09-19T09:29:40.869 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-09-19T09:29:40.869 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-09-19T09:29:40.870 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-09-19T09:29:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-09-19T09:29:40.872 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-09-19T09:29:40.873 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-09-19T09:29:40.873 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-09-19T09:29:40.873 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-09-19T09:29:40.873 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-09-19T09:29:40.873 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-09-19T09:29:40.873 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-09-19T09:29:40.873 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-09-19T09:29:40.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:40 smithi142 ceph-mon[35032]: pgmap v400: 97 pgs: 97 active+clean; 392 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 106 MiB/s rd, 1.6 MiB/s wr, 469 op/s 2024-09-19T09:29:40.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:40 smithi097 ceph-mon[34864]: pgmap v400: 97 pgs: 97 active+clean; 392 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 106 MiB/s rd, 1.6 MiB/s wr, 469 op/s 2024-09-19T09:29:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.969+0000 7f30a8a41640 1 -- 172.21.15.16:0/3762261564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a4108300 msgr2=0x7f30a410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.969+0000 7f30a8a41640 1 --2- 172.21.15.16:0/3762261564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a4108300 0x7f30a410ec70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f309c009cd0 tx=0x7f309c02a8d0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.969+0000 7f30a8a41640 1 -- 172.21.15.16:0/3762261564 shutdown_connections 2024-09-19T09:29:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.969+0000 7f30a8a41640 1 --2- 172.21.15.16:0/3762261564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30a4103780 0x7f30a4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.969+0000 7f30a8a41640 1 --2- 172.21.15.16:0/3762261564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a4108300 0x7f30a410ec70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.969+0000 7f30a8a41640 1 --2- 172.21.15.16:0/3762261564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30a4102dd0 0x7f30a41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.969+0000 7f30a8a41640 1 -- 172.21.15.16:0/3762261564 >> 172.21.15.16:0/3762261564 conn(0x7f30a40fc7d0 msgr2=0x7f30a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.969+0000 7f30a8a41640 1 -- 172.21.15.16:0/3762261564 shutdown_connections 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.969+0000 7f30a8a41640 1 -- 172.21.15.16:0/3762261564 wait complete. 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.970+0000 7f30a8a41640 1 Processor -- start 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.970+0000 7f30a8a41640 1 -- start start 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.970+0000 7f30a8a41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a4102dd0 0x7f30a4100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.970+0000 7f30a8a41640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30a4103780 0x7f30a4100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.970+0000 7f30a8a41640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30a4108300 0x7f30a41014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.970+0000 7f30a8a41640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41120d0 con 0x7f30a4102dd0 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.970+0000 7f30a8a41640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4111f50 con 0x7f30a4108300 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.970+0000 7f30a8a41640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4112250 con 0x7f30a4103780 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.970+0000 7f30a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a4102dd0 0x7f30a4100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.970+0000 7f30a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a4102dd0 0x7f30a4100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53434/0 (socket says 172.21.15.16:53434) 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.970+0000 7f30a259c640 1 -- 172.21.15.16:0/4158711787 learned_addr learned my addr 172.21.15.16:0/4158711787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.970+0000 7f30a1d9b640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30a4103780 0x7f30a4100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.970+0000 7f30a2d9d640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30a4108300 0x7f30a41014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.971+0000 7f30a259c640 1 -- 172.21.15.16:0/4158711787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30a4103780 msgr2=0x7f30a4100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.971+0000 7f30a259c640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30a4103780 0x7f30a4100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.971+0000 7f30a259c640 1 -- 172.21.15.16:0/4158711787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30a4108300 msgr2=0x7f30a41014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.971+0000 7f30a259c640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30a4108300 0x7f30a41014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.971+0000 7f30a259c640 1 -- 172.21.15.16:0/4158711787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a41a6a50 con 0x7f30a4102dd0 2024-09-19T09:29:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.971+0000 7f30a2d9d640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30a4108300 0x7f30a41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.971+0000 7f30a259c640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a4102dd0 0x7f30a4100a30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f309400e9f0 tx=0x7f309400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.971+0000 7f30937fe640 1 -- 172.21.15.16:0/4158711787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309400cd80 con 0x7f30a4102dd0 2024-09-19T09:29:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.971+0000 7f30937fe640 1 -- 172.21.15.16:0/4158711787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094010910 con 0x7f30a4102dd0 2024-09-19T09:29:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.971+0000 7f30a8a41640 1 -- 172.21.15.16:0/4158711787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a41a6d40 con 0x7f30a4102dd0 2024-09-19T09:29:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.971+0000 7f30937fe640 1 -- 172.21.15.16:0/4158711787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094018990 con 0x7f30a4102dd0 2024-09-19T09:29:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.971+0000 7f30a8a41640 1 -- 172.21.15.16:0/4158711787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a41a7270 con 0x7f30a4102dd0 2024-09-19T09:29:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.973+0000 7f30937fe640 1 -- 172.21.15.16:0/4158711787 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3094010ac0 con 0x7f30a4102dd0 2024-09-19T09:29:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.974+0000 7f30aae1a740 1 Processor -- start 2024-09-19T09:29:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.974+0000 7f30aae1a740 1 -- start start 2024-09-19T09:29:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.974+0000 7f30aae1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475ca4a0 0x55cc475ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.974+0000 7f30aae1a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475cadc0 0x55cc475d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.974+0000 7f30aae1a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475d5680 0x55cc475d7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.974+0000 7f30937fe640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f308c0a32c0 0x7f308c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.974+0000 7f30aae1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cc475302e0 con 0x55cc475ca4a0 2024-09-19T09:29:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.974+0000 7f30aae1a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc474d7ce0 con 0x55cc475cadc0 2024-09-19T09:29:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.974+0000 7f30aae1a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cc4752c2a0 con 0x55cc475d5680 2024-09-19T09:29:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475cadc0 0x55cc475d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475cadc0 0x55cc475d5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:55066/0 (socket says 172.21.15.16:55066) 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a259c640 1 -- 172.21.15.16:0/2085095339 learned_addr learned my addr 172.21.15.16:0/2085095339 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a1d9b640 1 --2- 172.21.15.16:0/2085095339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475d5680 0x55cc475d7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30937fe640 1 -- 172.21.15.16:0/4158711787 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f30940cfc30 con 0x7f30a4102dd0 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a2d9d640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f308c0a32c0 0x7f308c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a2d9d640 1 --2- 172.21.15.16:0/2085095339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475ca4a0 0x55cc475ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a1d9b640 1 -- 172.21.15.16:0/2085095339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475cadc0 msgr2=0x55cc475d5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a1d9b640 1 --2- 172.21.15.16:0/2085095339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475cadc0 0x55cc475d5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a1d9b640 1 -- 172.21.15.16:0/2085095339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475ca4a0 msgr2=0x55cc475ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a1d9b640 1 --2- 172.21.15.16:0/2085095339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475ca4a0 0x55cc475ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a1d9b640 1 -- 172.21.15.16:0/2085095339 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc474bc440 con 0x55cc475d5680 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a259c640 1 --2- 172.21.15.16:0/2085095339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475cadc0 0x55cc475d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a2d9d640 1 --2- 172.21.15.16:0/2085095339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475ca4a0 0x55cc475ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.975+0000 7f30a1d9b640 1 --2- 172.21.15.16:0/2085095339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475d5680 0x55cc475d7a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f30a406a2d0 tx=0x7f3098011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7a79436d74bc679 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:41.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.976+0000 7f3090ff9640 1 -- 172.21.15.16:0/2085095339 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098004050 con 0x55cc475d5680 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.976+0000 7f30a2d9d640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f308c0a32c0 0x7f308c0a5780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f309c02f4d0 tx=0x7f309c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.976+0000 7f3090ff9640 1 -- 172.21.15.16:0/2085095339 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3098005500 con 0x55cc475d5680 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.976+0000 7f3090ff9640 1 -- 172.21.15.16:0/2085095339 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098004cd0 con 0x55cc475d5680 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.977+0000 7f30aae1a740 1 -- 172.21.15.16:0/2085095339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475d5680 msgr2=0x55cc475d7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.977+0000 7f30aae1a740 1 --2- 172.21.15.16:0/2085095339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475d5680 0x55cc475d7a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f30a406a2d0 tx=0x7f3098011810 comp rx=0 tx=0).stop 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.977+0000 7f30aae1a740 1 -- 172.21.15.16:0/2085095339 shutdown_connections 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.977+0000 7f30aae1a740 1 --2- 172.21.15.16:0/2085095339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475d5680 0x55cc475d7a70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.977+0000 7f30aae1a740 1 --2- 172.21.15.16:0/2085095339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475cadc0 0x55cc475d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.977+0000 7f30aae1a740 1 --2- 172.21.15.16:0/2085095339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475ca4a0 0x55cc475ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.977+0000 7f30aae1a740 1 -- 172.21.15.16:0/2085095339 >> 172.21.15.16:0/2085095339 conn(0x55cc475c6a30 msgr2=0x55cc475c6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.977+0000 7f30aae1a740 1 -- 172.21.15.16:0/2085095339 shutdown_connections 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.977+0000 7f30aae1a740 1 -- 172.21.15.16:0/2085095339 wait complete. 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.977+0000 7f30aae1a740 1 -- start start 2024-09-19T09:29:41.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.977+0000 7f30aae1a740 1 Processor -- start 2024-09-19T09:29:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30aae1a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475ca4a0 0x55cc475d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30aae1a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 0x55cc475d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30aae1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475d5680 0x55cc475d2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30aae1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cc475d8250 con 0x55cc475d5680 2024-09-19T09:29:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30aae1a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc474eb050 con 0x55cc475ca4a0 2024-09-19T09:29:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30aae1a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cc475d83d0 con 0x55cc475cadc0 2024-09-19T09:29:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30aae1a740 10 client.0 _ll_register_callbacks cb 0x55cc4752b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30a2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475d5680 0x55cc475d2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30a2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475d5680 0x55cc475d2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53456/0 (socket says 172.21.15.16:53456) 2024-09-19T09:29:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30a2d9d640 1 -- 172.21.15.16:0/4083909927 learned_addr learned my addr 172.21.15.16:0/4083909927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30a259c640 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475ca4a0 0x55cc475d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30a1d9b640 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 0x55cc475d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30a2d9d640 1 -- 172.21.15.16:0/4083909927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 msgr2=0x55cc475d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30a2d9d640 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 0x55cc475d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30a2d9d640 1 -- 172.21.15.16:0/4083909927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475ca4a0 msgr2=0x55cc475d4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30a2d9d640 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475ca4a0 0x55cc475d4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30a2d9d640 1 -- 172.21.15.16:0/4083909927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc472a13c0 con 0x55cc475d5680 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.978+0000 7f30a259c640 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475ca4a0 0x55cc475d4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.979+0000 7f30a2d9d640 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475d5680 0x55cc475d2740 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f309c031ce0 tx=0x7f309c02a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c01e5ccf13bb6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.979+0000 7f3089ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.979+0000 7f3089ffb640 1 -- 172.21.15.16:0/4083909927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c031830 con 0x55cc475d5680 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.979+0000 7f30aae1a740 1 -- 172.21.15.16:0/4083909927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3098005500 con 0x55cc475d5680 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.979+0000 7f3089ffb640 1 -- 172.21.15.16:0/4083909927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f309c003970 con 0x55cc475d5680 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.979+0000 7f3089ffb640 1 -- 172.21.15.16:0/4083909927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f309c003b70 con 0x55cc475d5680 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.979+0000 7f30aae1a740 10 client.16047 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.979+0000 7f30aae1a740 1 -- 172.21.15.16:0/4083909927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc475e3b40 con 0x55cc475d5680 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.979+0000 7f3089ffb640 1 -- 172.21.15.16:0/4083909927 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3098005500 con 0x55cc475d5680 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.979+0000 7f3089ffb640 1 -- 172.21.15.16:0/4083909927 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f309c02f750 con 0x55cc475d5680 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.980+0000 7f30aae1a740 10 client.16047 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.980+0000 7f30aae1a740 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.980+0000 7f30aae1a740 20 client.16047 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.980+0000 7f30aae1a740 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cc475e5000 0x55cc475e7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.980+0000 7f30aae1a740 4 client.16047 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.980+0000 7f30aae1a740 1 -- 172.21.15.16:0/4083909927 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc4753d850 con 0x55cc475e5000 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.980+0000 7f30a259c640 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cc475e5000 0x55cc475e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.980+0000 7f30a259c640 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cc475e5000 0x55cc475e7470 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77cd75cb44acf275 server_cookie=1b086b7d1f70a829 in_seq=0 out_seq=0 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.980+0000 7f3089ffb640 10 client.16047 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f3089ffb640 1 -- 172.21.15.16:0/4083909927 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc4753d850 con 0x55cc475e5000 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f3089ffb640 10 client.16047 handle_command_reply: tid=0 2024-09-19T09:29:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f30aae1a740 1 client.16047 shutdown 2024-09-19T09:29:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f30aae1a740 1 -- 172.21.15.16:0/4083909927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475d5680 msgr2=0x55cc475d2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f30aae1a740 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475d5680 0x55cc475d2740 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f309c031ce0 tx=0x7f309c02a8d0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f30aae1a740 1 -- 172.21.15.16:0/4083909927 shutdown_connections 2024-09-19T09:29:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f30aae1a740 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cc475e5000 0x55cc475e7470 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f30aae1a740 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 0x55cc475d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f30aae1a740 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475d5680 0x55cc475d2740 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f3089ffb640 0 client.16047 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f30aae1a740 1 --2- 172.21.15.16:0/4083909927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475ca4a0 0x55cc475d4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f30aae1a740 1 -- 172.21.15.16:0/4083909927 >> 172.21.15.16:0/4083909927 conn(0x55cc475c6a30 msgr2=0x55cc475cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f30aae1a740 1 -- 172.21.15.16:0/4083909927 shutdown_connections 2024-09-19T09:29:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f30aae1a740 1 -- 172.21.15.16:0/4083909927 wait complete. 2024-09-19T09:29:41.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:40.981+0000 7f30aae1a740 20 client.16047 trim_cache size 0 max 16384 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.000+0000 7f30aae1a740 1 Processor -- start 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30aae1a740 1 -- start start 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30aae1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475c6a30 0x55cc475e14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30aae1a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475ca4a0 0x55cc475cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30aae1a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 0x55cc475ecd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30aae1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41af7e0 con 0x55cc475c6a30 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30aae1a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41b1320 con 0x55cc475ca4a0 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30aae1a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41afeb0 con 0x55cc475cadc0 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475c6a30 0x55cc475e14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475c6a30 0x55cc475e14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53470/0 (socket says 172.21.15.16:53470) 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30a259c640 1 -- 172.21.15.16:0/1969440873 learned_addr learned my addr 172.21.15.16:0/1969440873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30a1d9b640 1 --2- 172.21.15.16:0/1969440873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475ca4a0 0x55cc475cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30a2d9d640 1 --2- 172.21.15.16:0/1969440873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 0x55cc475ecd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30a259c640 1 -- 172.21.15.16:0/1969440873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 msgr2=0x55cc475ecd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30a259c640 1 --2- 172.21.15.16:0/1969440873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 0x55cc475ecd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30a259c640 1 -- 172.21.15.16:0/1969440873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475ca4a0 msgr2=0x55cc475cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30a259c640 1 --2- 172.21.15.16:0/1969440873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475ca4a0 0x55cc475cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.001+0000 7f30a259c640 1 -- 172.21.15.16:0/1969440873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc475cabf0 con 0x55cc475c6a30 2024-09-19T09:29:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.002+0000 7f30a259c640 1 --2- 172.21.15.16:0/1969440873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475c6a30 0x55cc475e14e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3094098bd0 tx=0x7f30940cd600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63588ab906d12c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.002+0000 7f308bfff640 1 -- 172.21.15.16:0/1969440873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309401de70 con 0x55cc475c6a30 2024-09-19T09:29:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.002+0000 7f308bfff640 1 -- 172.21.15.16:0/1969440873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f309409c950 con 0x55cc475c6a30 2024-09-19T09:29:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.002+0000 7f308bfff640 1 -- 172.21.15.16:0/1969440873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30940a4a30 con 0x55cc475c6a30 2024-09-19T09:29:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.003+0000 7f30aae1a740 1 -- 172.21.15.16:0/1969440873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475c6a30 msgr2=0x55cc475e14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.003+0000 7f30aae1a740 1 --2- 172.21.15.16:0/1969440873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475c6a30 0x55cc475e14e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3094098bd0 tx=0x7f30940cd600 comp rx=0 tx=0).stop 2024-09-19T09:29:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.003+0000 7f30aae1a740 1 -- 172.21.15.16:0/1969440873 shutdown_connections 2024-09-19T09:29:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.003+0000 7f30aae1a740 1 --2- 172.21.15.16:0/1969440873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 0x55cc475ecd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.003+0000 7f30aae1a740 1 --2- 172.21.15.16:0/1969440873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475ca4a0 0x55cc475cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.003+0000 7f30aae1a740 1 --2- 172.21.15.16:0/1969440873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475c6a30 0x55cc475e14e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.003+0000 7f30aae1a740 1 -- 172.21.15.16:0/1969440873 >> 172.21.15.16:0/1969440873 conn(0x55cc475d28a0 msgr2=0x55cc475cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.003+0000 7f30aae1a740 1 -- 172.21.15.16:0/1969440873 shutdown_connections 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.003+0000 7f30aae1a740 1 -- 172.21.15.16:0/1969440873 wait complete. 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.003+0000 7f30aae1a740 1 -- start start 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.003+0000 7f30aae1a740 1 Processor -- start 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30aae1a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475c6a30 0x55cc475dde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30aae1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475ca4a0 0x55cc475cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30aae1a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 0x55cc475cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30aae1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41b0c50 con 0x55cc475ca4a0 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30aae1a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41b0580 con 0x55cc475c6a30 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30aae1a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41ad7a0 con 0x55cc475cadc0 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30aae1a740 10 client.0 _ll_register_callbacks cb 0x7f3094005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30a259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475c6a30 0x55cc475dde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30a1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475ca4a0 0x55cc475cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30a259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475c6a30 0x55cc475dde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.16:55088/0 (socket says 172.21.15.16:55088) 2024-09-19T09:29:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30a2d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 0x55cc475cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30a259c640 1 -- 172.21.15.16:0/3342599317 learned_addr learned my addr 172.21.15.16:0/3342599317 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30a259c640 1 -- 172.21.15.16:0/3342599317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 msgr2=0x55cc475cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30a259c640 1 --2- 172.21.15.16:0/3342599317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 0x55cc475cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30a259c640 1 -- 172.21.15.16:0/3342599317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475ca4a0 msgr2=0x55cc475cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30a259c640 1 --2- 172.21.15.16:0/3342599317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475ca4a0 0x55cc475cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30a259c640 1 -- 172.21.15.16:0/3342599317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc475cfe00 con 0x55cc475c6a30 2024-09-19T09:29:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f30a259c640 1 --2- 172.21.15.16:0/3342599317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475c6a30 0x55cc475dde90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3094098c20 tx=0x7f309401de70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e655f7489268598 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.004+0000 7f3089ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-19T09:29:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f3089ffb640 1 -- 172.21.15.16:0/3342599317 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30940a4ec0 con 0x55cc475c6a30 2024-09-19T09:29:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f3089ffb640 1 -- 172.21.15.16:0/3342599317 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f30940b5190 con 0x55cc475c6a30 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f30aae1a740 1 -- 172.21.15.16:0/3342599317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f309409c950 con 0x55cc475c6a30 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f3089ffb640 1 -- 172.21.15.16:0/3342599317 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f30940a0600 con 0x55cc475c6a30 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f30aae1a740 10 client.25619 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f30aae1a740 1 -- 172.21.15.16:0/3342599317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc4764fe00 con 0x55cc475c6a30 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f3089ffb640 1 -- 172.21.15.16:0/3342599317 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f309409c950 con 0x55cc475c6a30 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f3089ffb640 1 -- 172.21.15.16:0/3342599317 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3094018b30 con 0x55cc475c6a30 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f30aae1a740 10 client.25619 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f30aae1a740 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f30aae1a740 20 client.25619 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f30aae1a740 1 --2- 172.21.15.16:0/3342599317 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cc475d5680 0x55cc47652470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f30aae1a740 4 client.25619 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.005+0000 7f30aae1a740 1 -- 172.21.15.16:0/3342599317 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cc47537d60 con 0x55cc475d5680 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.006+0000 7f30a1d9b640 1 --2- 172.21.15.16:0/3342599317 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cc475d5680 0x55cc47652470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.006+0000 7f3089ffb640 10 client.25619 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.006+0000 7f30a1d9b640 1 --2- 172.21.15.16:0/3342599317 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cc475d5680 0x55cc47652470 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d9f2a2982c1232 server_cookie=bddae0f1e263abe3 in_seq=0 out_seq=0 2024-09-19T09:29:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f3089ffb640 1 -- 172.21.15.16:0/3342599317 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cc47537d60 con 0x55cc475d5680 2024-09-19T09:29:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f3089ffb640 10 client.25619 handle_command_reply: tid=0 2024-09-19T09:29:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f30aae1a740 1 client.25619 shutdown 2024-09-19T09:29:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f30aae1a740 1 -- 172.21.15.16:0/3342599317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475c6a30 msgr2=0x55cc475dde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f30aae1a740 1 --2- 172.21.15.16:0/3342599317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475c6a30 0x55cc475dde90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3094098c20 tx=0x7f309401de70 comp rx=0 tx=0).stop 2024-09-19T09:29:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f30aae1a740 1 -- 172.21.15.16:0/3342599317 shutdown_connections 2024-09-19T09:29:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f30aae1a740 1 --2- 172.21.15.16:0/3342599317 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x55cc475d5680 0x55cc47652470 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f30aae1a740 1 --2- 172.21.15.16:0/3342599317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc475cadc0 0x55cc475cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.079 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:29:41.079 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-19T09:29:41.079 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-19T09:29:41.079 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:29:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f30aae1a740 1 --2- 172.21.15.16:0/3342599317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc475ca4a0 0x55cc475cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f3089ffb640 0 client.25619 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f30aae1a740 1 --2- 172.21.15.16:0/3342599317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc475c6a30 0x55cc475dde90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f30aae1a740 1 -- 172.21.15.16:0/3342599317 >> 172.21.15.16:0/3342599317 conn(0x55cc475d28a0 msgr2=0x55cc475d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f30aae1a740 1 -- 172.21.15.16:0/3342599317 shutdown_connections 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f30aae1a740 1 -- 172.21.15.16:0/3342599317 wait complete. 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.008+0000 7f30aae1a740 20 client.25619 trim_cache size 0 max 16384 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.009+0000 7f30a8a41640 1 -- 172.21.15.16:0/4158711787 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f308c0a32c0 msgr2=0x7f308c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.009+0000 7f30a8a41640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f308c0a32c0 0x7f308c0a5780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f309c02f4d0 tx=0x7f309c0095c0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.009+0000 7f30a8a41640 1 -- 172.21.15.16:0/4158711787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a4102dd0 msgr2=0x7f30a4100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.009+0000 7f30a8a41640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a4102dd0 0x7f30a4100a30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f309400e9f0 tx=0x7f309400eeb0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.009+0000 7f30a8a41640 1 -- 172.21.15.16:0/4158711787 shutdown_connections 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.009+0000 7f30a8a41640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f308c0a32c0 0x7f308c0a5780 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.009+0000 7f30a8a41640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30a4103780 0x7f30a4100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.009+0000 7f30a8a41640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30a4108300 0x7f30a41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.009+0000 7f30a8a41640 1 --2- 172.21.15.16:0/4158711787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a4102dd0 0x7f30a4100a30 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.009+0000 7f30a8a41640 1 -- 172.21.15.16:0/4158711787 >> 172.21.15.16:0/4158711787 conn(0x7f30a40fc7d0 msgr2=0x7f30a4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.009+0000 7f30a8a41640 1 -- 172.21.15.16:0/4158711787 shutdown_connections 2024-09-19T09:29:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:41.009+0000 7f30a8a41640 1 -- 172.21.15.16:0/4158711787 wait complete. 2024-09-19T09:29:41.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-09-19T09:29:41.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-09-19T09:29:41.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-09-19T09:29:41.303 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-09-19T09:29:41.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-09-19T09:29:41.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-09-19T09:29:41.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-09-19T09:29:41.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-09-19T09:29:41.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-09-19T09:29:41.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-09-19T09:29:41.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-09-19T09:29:41.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-09-19T09:29:41.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-09-19T09:29:41.304 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-09-19T09:29:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-09-19T09:29:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-09-19T09:29:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-09-19T09:29:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-09-19T09:29:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-09-19T09:29:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-09-19T09:29:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-09-19T09:29:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-09-19T09:29:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-09-19T09:29:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-09-19T09:29:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-09-19T09:29:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-09-19T09:29:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-09-19T09:29:41.306 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-09-19T09:29:41.306 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-09-19T09:29:41.306 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-09-19T09:29:41.306 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-09-19T09:29:41.306 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-09-19T09:29:41.306 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-09-19T09:29:41.306 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-09-19T09:29:41.306 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-09-19T09:29:41.306 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-09-19T09:29:41.306 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-09-19T09:29:41.306 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-09-19T09:29:41.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-09-19T09:29:41.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-09-19T09:29:41.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-09-19T09:29:41.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-09-19T09:29:41.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-09-19T09:29:41.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-09-19T09:29:41.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-09-19T09:29:41.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-09-19T09:29:41.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-09-19T09:29:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-09-19T09:29:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-09-19T09:29:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-09-19T09:29:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-09-19T09:29:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-09-19T09:29:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-09-19T09:29:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-09-19T09:29:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-09-19T09:29:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-09-19T09:29:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-09-19T09:29:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-09-19T09:29:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-09-19T09:29:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-09-19T09:29:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-09-19T09:29:41.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-09-19T09:29:41.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-09-19T09:29:41.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-09-19T09:29:41.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-09-19T09:29:41.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-09-19T09:29:41.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-09-19T09:29:41.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-09-19T09:29:41.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-09-19T09:29:41.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-09-19T09:29:41.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-09-19T09:29:41.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-09-19T09:29:41.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-09-19T09:29:41.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-09-19T09:29:41.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-09-19T09:29:41.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-09-19T09:29:41.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-09-19T09:29:41.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-09-19T09:29:41.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-09-19T09:29:41.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-09-19T09:29:41.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-09-19T09:29:41.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-09-19T09:29:41.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-09-19T09:29:41.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-09-19T09:29:41.821 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T09:29:41.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:29:42.094 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:42.145 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-09-19T09:29:42.145 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-09-19T09:29:42.145 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-09-19T09:29:42.145 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-09-19T09:29:42.145 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-09-19T09:29:42.145 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-09-19T09:29:42.145 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-09-19T09:29:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-09-19T09:29:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-09-19T09:29:42.148 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-09-19T09:29:42.149 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-09-19T09:29:42.149 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-09-19T09:29:42.149 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-09-19T09:29:42.149 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-09-19T09:29:42.149 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-09-19T09:29:42.149 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-09-19T09:29:42.149 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-09-19T09:29:42.149 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-09-19T09:29:42.149 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-09-19T09:29:42.149 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-09-19T09:29:42.149 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-09-19T09:29:42.149 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-09-19T09:29:42.149 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-09-19T09:29:42.150 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-09-19T09:29:42.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-09-19T09:29:42.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-09-19T09:29:42.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-09-19T09:29:42.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-09-19T09:29:42.151 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-09-19T09:29:42.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.610+0000 7f122a940640 1 -- 172.21.15.16:0/11968012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f122410c170 msgr2=0x7f122410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:42.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.610+0000 7f122a940640 1 --2- 172.21.15.16:0/11968012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f122410c170 0x7f122410e610 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f121c009ca0 tx=0x7f121c02a950 comp rx=0 tx=0).stop 2024-09-19T09:29:42.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.610+0000 7f122a940640 1 -- 172.21.15.16:0/11968012 shutdown_connections 2024-09-19T09:29:42.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.610+0000 7f122a940640 1 --2- 172.21.15.16:0/11968012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f122410c170 0x7f122410e610 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.610+0000 7f122a940640 1 --2- 172.21.15.16:0/11968012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1224103660 0x7f122410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.610+0000 7f122a940640 1 --2- 172.21.15.16:0/11968012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224102d40 0x7f1224103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.610+0000 7f122a940640 1 -- 172.21.15.16:0/11968012 >> 172.21.15.16:0/11968012 conn(0x7f12240fc7a0 msgr2=0x7f12240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.610+0000 7f122a940640 1 -- 172.21.15.16:0/11968012 shutdown_connections 2024-09-19T09:29:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.610+0000 7f122a940640 1 -- 172.21.15.16:0/11968012 wait complete. 2024-09-19T09:29:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.611+0000 7f122a940640 1 Processor -- start 2024-09-19T09:29:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.611+0000 7f122a940640 1 -- start start 2024-09-19T09:29:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.611+0000 7f122a940640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224102d40 0x7f1224198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.611+0000 7f122a940640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224103660 0x7f1224198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.611+0000 7f122a940640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f122410c170 0x7f122419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.611+0000 7f122a940640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1224111f70 con 0x7f1224103660 2024-09-19T09:29:42.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.611+0000 7f122a940640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1224111df0 con 0x7f122410c170 2024-09-19T09:29:42.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.611+0000 7f122a940640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f12241120f0 con 0x7f1224102d40 2024-09-19T09:29:42.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.611+0000 7f1223fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224102d40 0x7f1224198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:42.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.611+0000 7f121bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224103660 0x7f1224198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:42.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.611+0000 7f1223fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224102d40 0x7f1224198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:49460/0 (socket says 172.21.15.16:49460) 2024-09-19T09:29:42.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.611+0000 7f121bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224103660 0x7f1224198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53486/0 (socket says 172.21.15.16:53486) 2024-09-19T09:29:42.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.611+0000 7f1223fff640 1 -- 172.21.15.16:0/111848220 learned_addr learned my addr 172.21.15.16:0/111848220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:42.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.612+0000 7f1228edd640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f122410c170 0x7f122419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:42.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.612+0000 7f1223fff640 1 -- 172.21.15.16:0/111848220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f122410c170 msgr2=0x7f122419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:42.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.612+0000 7f1223fff640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f122410c170 0x7f122419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:42.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.612+0000 7f1223fff640 1 -- 172.21.15.16:0/111848220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224103660 msgr2=0x7f1224198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:42.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.612+0000 7f1223fff640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224103660 0x7f1224198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:42.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.612+0000 7f1223fff640 1 -- 172.21.15.16:0/111848220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122419d3c0 con 0x7f1224102d40 2024-09-19T09:29:42.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.612+0000 7f1228edd640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f122410c170 0x7f122419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:42.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.612+0000 7f121bfff640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224103660 0x7f1224198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:42.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.612+0000 7f1223fff640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224102d40 0x7f1224198410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f121400e9f0 tx=0x7f121400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:42.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.612+0000 7f1221ffb640 1 -- 172.21.15.16:0/111848220 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121400cd80 con 0x7f1224102d40 2024-09-19T09:29:42.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.612+0000 7f1221ffb640 1 -- 172.21.15.16:0/111848220 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1214005500 con 0x7f1224102d40 2024-09-19T09:29:42.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.613+0000 7f122a940640 1 -- 172.21.15.16:0/111848220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122419d6b0 con 0x7f1224102d40 2024-09-19T09:29:42.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.613+0000 7f1221ffb640 1 -- 172.21.15.16:0/111848220 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214010690 con 0x7f1224102d40 2024-09-19T09:29:42.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.613+0000 7f122a940640 1 -- 172.21.15.16:0/111848220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12241a4f90 con 0x7f1224102d40 2024-09-19T09:29:42.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.614+0000 7f122a940640 1 -- 172.21.15.16:0/111848220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1224103120 con 0x7f1224102d40 2024-09-19T09:29:42.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.616+0000 7f1221ffb640 1 -- 172.21.15.16:0/111848220 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f12140046c0 con 0x7f1224102d40 2024-09-19T09:29:42.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.617+0000 7f1221ffb640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f12100a32c0 0x7f12100a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:42.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.617+0000 7f1221ffb640 1 -- 172.21.15.16:0/111848220 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f12140cfd80 con 0x7f1224102d40 2024-09-19T09:29:42.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.617+0000 7f121bfff640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f12100a32c0 0x7f12100a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:42.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.617+0000 7f121bfff640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f12100a32c0 0x7f12100a5780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f122410d490 tx=0x7f120c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:42.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.622+0000 7f1221ffb640 1 -- 172.21.15.16:0/111848220 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f121409ca60 con 0x7f1224102d40 2024-09-19T09:29:42.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-09-19T09:29:42.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-09-19T09:29:42.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-09-19T09:29:42.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-09-19T09:29:42.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-09-19T09:29:42.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-09-19T09:29:42.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-09-19T09:29:42.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-09-19T09:29:42.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-09-19T09:29:42.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-09-19T09:29:42.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-09-19T09:29:42.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-09-19T09:29:42.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-09-19T09:29:42.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-09-19T09:29:42.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-09-19T09:29:42.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-09-19T09:29:42.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-09-19T09:29:42.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-09-19T09:29:42.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-09-19T09:29:42.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-09-19T09:29:42.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-09-19T09:29:42.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-09-19T09:29:42.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-09-19T09:29:42.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-09-19T09:29:42.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-09-19T09:29:42.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-09-19T09:29:42.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-09-19T09:29:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-09-19T09:29:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-09-19T09:29:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-09-19T09:29:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-09-19T09:29:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-09-19T09:29:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-09-19T09:29:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-09-19T09:29:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-09-19T09:29:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-09-19T09:29:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-09-19T09:29:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-09-19T09:29:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-09-19T09:29:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-09-19T09:29:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-09-19T09:29:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-09-19T09:29:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-09-19T09:29:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-09-19T09:29:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-09-19T09:29:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-09-19T09:29:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-09-19T09:29:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-09-19T09:29:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-09-19T09:29:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-09-19T09:29:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-09-19T09:29:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-09-19T09:29:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-09-19T09:29:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-09-19T09:29:42.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:42 smithi016 ceph-mon[30979]: pgmap v401: 97 pgs: 97 active+clean; 380 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 113 MiB/s rd, 1.7 MiB/s wr, 485 op/s 2024-09-19T09:29:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.844+0000 7f122a940640 1 -- 172.21.15.16:0/111848220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12241a51f0 con 0x7f1224102d40 2024-09-19T09:29:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.845+0000 7f1221ffb640 1 -- 172.21.15.16:0/111848220 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f121409c360 con 0x7f1224102d40 2024-09-19T09:29:42.846 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:29:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:29:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:29:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.847+0000 7f122a940640 1 -- 172.21.15.16:0/111848220 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f12100a32c0 msgr2=0x7f12100a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.847+0000 7f122a940640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f12100a32c0 0x7f12100a5780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f122410d490 tx=0x7f120c005f90 comp rx=0 tx=0).stop 2024-09-19T09:29:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.847+0000 7f122a940640 1 -- 172.21.15.16:0/111848220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224102d40 msgr2=0x7f1224198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.847+0000 7f122a940640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224102d40 0x7f1224198410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f121400e9f0 tx=0x7f121400edf0 comp rx=0 tx=0).stop 2024-09-19T09:29:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.847+0000 7f122a940640 1 -- 172.21.15.16:0/111848220 shutdown_connections 2024-09-19T09:29:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.847+0000 7f122a940640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f122410c170 0x7f122419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.847+0000 7f122a940640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224103660 0x7f1224198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.847+0000 7f122a940640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f12100a32c0 0x7f12100a5780 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.847+0000 7f122a940640 1 --2- 172.21.15.16:0/111848220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224102d40 0x7f1224198410 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.847+0000 7f122a940640 1 -- 172.21.15.16:0/111848220 >> 172.21.15.16:0/111848220 conn(0x7f12240fc7a0 msgr2=0x7f12240fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.848+0000 7f122a940640 1 -- 172.21.15.16:0/111848220 shutdown_connections 2024-09-19T09:29:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:42.848+0000 7f122a940640 1 -- 172.21.15.16:0/111848220 wait complete. 2024-09-19T09:29:42.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:42 smithi142 ceph-mon[35032]: pgmap v401: 97 pgs: 97 active+clean; 380 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 113 MiB/s rd, 1.7 MiB/s wr, 485 op/s 2024-09-19T09:29:42.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:42 smithi097 ceph-mon[34864]: pgmap v401: 97 pgs: 97 active+clean; 380 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 113 MiB/s rd, 1.7 MiB/s wr, 485 op/s 2024-09-19T09:29:43.076 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-09-19T09:29:43.076 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-09-19T09:29:43.077 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-09-19T09:29:43.077 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-09-19T09:29:43.077 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-09-19T09:29:43.077 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-09-19T09:29:43.077 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-09-19T09:29:43.077 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-09-19T09:29:43.077 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-09-19T09:29:43.077 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-09-19T09:29:43.077 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-09-19T09:29:43.077 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-09-19T09:29:43.077 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-09-19T09:29:43.077 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-09-19T09:29:43.078 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-09-19T09:29:43.078 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-09-19T09:29:43.078 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-09-19T09:29:43.078 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-09-19T09:29:43.078 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-09-19T09:29:43.078 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-09-19T09:29:43.078 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-09-19T09:29:43.078 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-09-19T09:29:43.078 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-09-19T09:29:43.078 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-09-19T09:29:43.078 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-09-19T09:29:43.078 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-09-19T09:29:43.078 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-09-19T09:29:43.079 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-09-19T09:29:43.080 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-09-19T09:29:43.080 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-09-19T09:29:43.080 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-09-19T09:29:43.080 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-09-19T09:29:43.080 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-09-19T09:29:43.080 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-09-19T09:29:43.080 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-09-19T09:29:43.080 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-09-19T09:29:43.080 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-09-19T09:29:43.080 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-09-19T09:29:43.080 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-09-19T09:29:43.080 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-09-19T09:29:43.080 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-09-19T09:29:43.081 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-09-19T09:29:43.082 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-09-19T09:29:43.082 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-09-19T09:29:43.082 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-09-19T09:29:43.082 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-09-19T09:29:43.082 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-09-19T09:29:43.082 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-09-19T09:29:43.082 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-09-19T09:29:43.082 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-09-19T09:29:43.082 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-09-19T09:29:43.082 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-09-19T09:29:43.082 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-09-19T09:29:43.416 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-09-19T09:29:43.416 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-09-19T09:29:43.416 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-09-19T09:29:43.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-09-19T09:29:43.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-09-19T09:29:43.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-09-19T09:29:43.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-09-19T09:29:43.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-09-19T09:29:43.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-09-19T09:29:43.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-09-19T09:29:43.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-09-19T09:29:43.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-09-19T09:29:43.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-09-19T09:29:43.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-09-19T09:29:43.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-09-19T09:29:43.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-09-19T09:29:43.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-09-19T09:29:43.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-09-19T09:29:43.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-09-19T09:29:43.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-09-19T09:29:43.422 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-09-19T09:29:43.422 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-09-19T09:29:43.422 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-09-19T09:29:43.422 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-09-19T09:29:43.422 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-09-19T09:29:43.485 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-09-19T09:29:43.485 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-09-19T09:29:43.485 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-09-19T09:29:43.485 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-09-19T09:29:43.486 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-09-19T09:29:43.486 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-09-19T09:29:43.486 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-09-19T09:29:43.486 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-09-19T09:29:43.486 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-09-19T09:29:43.486 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-09-19T09:29:43.486 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-09-19T09:29:43.486 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-09-19T09:29:43.486 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-09-19T09:29:43.486 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-09-19T09:29:43.486 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T09:29:43.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T09:29:43.521 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-09-19T09:29:43.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T09:29:43.585 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:43 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/111848220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:43.794 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T09:29:43.878 DEBUG:teuthology.parallel:result is None 2024-09-19T09:29:43.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T09:29:43.880 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:43 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/111848220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:43.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:43 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/111848220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:43.953 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T09:29:43.953 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-19T09:29:44.023 INFO:tasks.quiescer:joining Quiescers 2024-09-19T09:29:44.023 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-19T09:29:44.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:29:44.094 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.596+0000 7f3de599e640 1 -- 172.21.15.16:0/1655704947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3de01034b0 msgr2=0x7f3de010b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.596+0000 7f3de599e640 1 --2- 172.21.15.16:0/1655704947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3de01034b0 0x7f3de010b9a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009900 tx=0x7f3dcc02a480 comp rx=0 tx=0).stop 2024-09-19T09:29:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.596+0000 7f3de599e640 1 -- 172.21.15.16:0/1655704947 shutdown_connections 2024-09-19T09:29:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.596+0000 7f3de599e640 1 --2- 172.21.15.16:0/1655704947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de010bfc0 0x7f3de010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.596+0000 7f3de599e640 1 --2- 172.21.15.16:0/1655704947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3de01034b0 0x7f3de010b9a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.596+0000 7f3de599e640 1 --2- 172.21.15.16:0/1655704947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3de0102b90 0x7f3de0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.596+0000 7f3de599e640 1 -- 172.21.15.16:0/1655704947 >> 172.21.15.16:0/1655704947 conn(0x7f3de00fc610 msgr2=0x7f3de00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.596+0000 7f3de599e640 1 -- 172.21.15.16:0/1655704947 shutdown_connections 2024-09-19T09:29:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.596+0000 7f3de599e640 1 -- 172.21.15.16:0/1655704947 wait complete. 2024-09-19T09:29:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.597+0000 7f3de599e640 1 Processor -- start 2024-09-19T09:29:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.597+0000 7f3de599e640 1 -- start start 2024-09-19T09:29:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.597+0000 7f3de599e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3de0102b90 0x7f3de01982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3de599e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3de01034b0 0x7f3de0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3de599e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de010bfc0 0x7f3de019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3de599e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3de0111dc0 con 0x7f3de01034b0 2024-09-19T09:29:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3de599e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3de0111c40 con 0x7f3de010bfc0 2024-09-19T09:29:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3de599e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3de0111f40 con 0x7f3de0102b90 2024-09-19T09:29:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3de01034b0 0x7f3de0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3de499c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3de0102b90 0x7f3de01982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3de519d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de010bfc0 0x7f3de019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3de01034b0 0x7f3de0198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53498/0 (socket says 172.21.15.16:53498) 2024-09-19T09:29:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3ddffff640 1 -- 172.21.15.16:0/2490691913 learned_addr learned my addr 172.21.15.16:0/2490691913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3de519d640 1 -- 172.21.15.16:0/2490691913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3de0102b90 msgr2=0x7f3de01982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3de519d640 1 --2- 172.21.15.16:0/2490691913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3de0102b90 0x7f3de01982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3de519d640 1 -- 172.21.15.16:0/2490691913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3de01034b0 msgr2=0x7f3de0198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3de519d640 1 --2- 172.21.15.16:0/2490691913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3de01034b0 0x7f3de0198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3de519d640 1 -- 172.21.15.16:0/2490691913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de019d280 con 0x7f3de010bfc0 2024-09-19T09:29:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.598+0000 7f3de519d640 1 --2- 172.21.15.16:0/2490691913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de010bfc0 0x7f3de019cba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009fd0 tx=0x7f3dd800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.599+0000 7f3dddffb640 1 -- 172.21.15.16:0/2490691913 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800ce20 con 0x7f3de010bfc0 2024-09-19T09:29:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.599+0000 7f3dddffb640 1 -- 172.21.15.16:0/2490691913 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dd8005020 con 0x7f3de010bfc0 2024-09-19T09:29:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.599+0000 7f3dddffb640 1 -- 172.21.15.16:0/2490691913 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8010720 con 0x7f3de010bfc0 2024-09-19T09:29:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.599+0000 7f3de599e640 1 -- 172.21.15.16:0/2490691913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de019d570 con 0x7f3de010bfc0 2024-09-19T09:29:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.599+0000 7f3de599e640 1 -- 172.21.15.16:0/2490691913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de0100720 con 0x7f3de010bfc0 2024-09-19T09:29:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.602+0000 7f3de8fd9740 1 Processor -- start 2024-09-19T09:29:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.602+0000 7f3dddffb640 1 -- 172.21.15.16:0/2490691913 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3dd8005540 con 0x7f3de010bfc0 2024-09-19T09:29:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.602+0000 7f3de8fd9740 1 -- start start 2024-09-19T09:29:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.602+0000 7f3de8fd9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073dd20 0x564d2073c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.602+0000 7f3de8fd9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d2073c7b0 0x564d2073cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.602+0000 7f3de8fd9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d20747550 0x564d20749940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.602+0000 7f3de8fd9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564d206a22f0 con 0x564d20747550 2024-09-19T09:29:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.602+0000 7f3de8fd9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564d20649cf0 con 0x564d2073dd20 2024-09-19T09:29:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.602+0000 7f3de8fd9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564d2069e2b0 con 0x564d2073c7b0 2024-09-19T09:29:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.603+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d20747550 0x564d20749940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.603+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d20747550 0x564d20749940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53506/0 (socket says 172.21.15.16:53506) 2024-09-19T09:29:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.603+0000 7f3ddffff640 1 -- 172.21.15.16:0/3425603307 learned_addr learned my addr 172.21.15.16:0/3425603307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.603+0000 7f3de499c640 1 --2- 172.21.15.16:0/3425603307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d2073c7b0 0x564d2073cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.603+0000 7f3ddffff640 1 --2- 172.21.15.16:0/3425603307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073dd20 0x564d2073c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.603+0000 7f3de499c640 1 -- 172.21.15.16:0/3425603307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073dd20 msgr2=0x564d2073c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.603+0000 7f3de499c640 1 --2- 172.21.15.16:0/3425603307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073dd20 0x564d2073c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.603+0000 7f3de499c640 1 -- 172.21.15.16:0/3425603307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d20747550 msgr2=0x564d20749940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.603+0000 7f3de499c640 1 --2- 172.21.15.16:0/3425603307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d20747550 0x564d20749940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.603+0000 7f3de499c640 1 -- 172.21.15.16:0/3425603307 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d2062e3a0 con 0x564d2073c7b0 2024-09-19T09:29:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.603+0000 7f3ddffff640 1 --2- 172.21.15.16:0/3425603307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073dd20 0x564d2073c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.603+0000 7f3dddffb640 1 --2- 172.21.15.16:0/2490691913 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3dd00a3240 0x7f3dd00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.603+0000 7f3de499c640 1 --2- 172.21.15.16:0/3425603307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d2073c7b0 0x564d2073cbf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400bbf0 tx=0x7f3dd40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0f8ad9c28bc69ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.604+0000 7f3ddffff640 1 --2- 172.21.15.16:0/2490691913 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3dd00a3240 0x7f3dd00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.604+0000 7f3dddffb640 1 -- 172.21.15.16:0/2490691913 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3dd80cffd0 con 0x7f3de010bfc0 2024-09-19T09:29:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.604+0000 7f3dcaffd640 1 -- 172.21.15.16:0/3425603307 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4004030 con 0x564d2073c7b0 2024-09-19T09:29:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.604+0000 7f3dcaffd640 1 -- 172.21.15.16:0/3425603307 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dd40041d0 con 0x564d2073c7b0 2024-09-19T09:29:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.604+0000 7f3dcaffd640 1 -- 172.21.15.16:0/3425603307 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40116c0 con 0x564d2073c7b0 2024-09-19T09:29:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.605+0000 7f3de8fd9740 1 -- 172.21.15.16:0/3425603307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d2073c7b0 msgr2=0x564d2073cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.605+0000 7f3ddffff640 1 --2- 172.21.15.16:0/2490691913 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3dd00a3240 0x7f3dd00a5700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc031550 tx=0x7f3dcc009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.605+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/3425603307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d2073c7b0 0x564d2073cbf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400bbf0 tx=0x7f3dd40107a0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.605+0000 7f3de8fd9740 1 -- 172.21.15.16:0/3425603307 shutdown_connections 2024-09-19T09:29:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.605+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/3425603307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d2073c7b0 0x564d2073cbf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.605+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/3425603307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d20747550 0x564d20749940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.605+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/3425603307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073dd20 0x564d2073c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.605+0000 7f3de8fd9740 1 -- 172.21.15.16:0/3425603307 >> 172.21.15.16:0/3425603307 conn(0x564d20705310 msgr2=0x564d20738e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.605+0000 7f3de8fd9740 1 -- 172.21.15.16:0/3425603307 shutdown_connections 2024-09-19T09:29:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.605+0000 7f3de8fd9740 1 -- 172.21.15.16:0/3425603307 wait complete. 2024-09-19T09:29:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.605+0000 7f3de8fd9740 1 -- start start 2024-09-19T09:29:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.605+0000 7f3de8fd9740 1 Processor -- start 2024-09-19T09:29:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.606+0000 7f3de8fd9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 0x564d20746eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.606+0000 7f3de8fd9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073dd20 0x564d207441b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.606+0000 7f3de8fd9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d20747550 0x564d20744750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.606+0000 7f3de8fd9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564d2074a250 con 0x564d2073c7b0 2024-09-19T09:29:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.606+0000 7f3de8fd9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564d2065d060 con 0x564d2073dd20 2024-09-19T09:29:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.606+0000 7f3de8fd9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564d2074a3d0 con 0x564d20747550 2024-09-19T09:29:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.606+0000 7f3de8fd9740 10 client.0 _ll_register_callbacks cb 0x564d203b08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.606+0000 7f3de499c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 0x564d20746eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.606+0000 7f3de499c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 0x564d20746eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53520/0 (socket says 172.21.15.16:53520) 2024-09-19T09:29:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.606+0000 7f3de499c640 1 -- 172.21.15.16:0/3120860128 learned_addr learned my addr 172.21.15.16:0/3120860128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.606+0000 7f3de519d640 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d20747550 0x564d20744750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.606+0000 7f3ddffff640 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073dd20 0x564d207441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.607+0000 7f3de499c640 1 -- 172.21.15.16:0/3120860128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d20747550 msgr2=0x564d20744750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.607+0000 7f3de499c640 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d20747550 0x564d20744750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.607+0000 7f3de499c640 1 -- 172.21.15.16:0/3120860128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073dd20 msgr2=0x564d207441b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.607+0000 7f3de499c640 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073dd20 0x564d207441b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.607+0000 7f3de499c640 1 -- 172.21.15.16:0/3120860128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d204133c0 con 0x564d2073c7b0 2024-09-19T09:29:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.607+0000 7f3ddffff640 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073dd20 0x564d207441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.607+0000 7f3de499c640 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 0x564d20746eb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4000c00 tx=0x7f3dd400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61aa5696e58db76d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.607+0000 7f3dbffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.607+0000 7f3dbffff640 1 -- 172.21.15.16:0/3120860128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40035f0 con 0x564d2073c7b0 2024-09-19T09:29:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.607+0000 7f3dbffff640 1 -- 172.21.15.16:0/3120860128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3dd4003790 con 0x564d2073c7b0 2024-09-19T09:29:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.607+0000 7f3de8fd9740 1 -- 172.21.15.16:0/3120860128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dd40041d0 con 0x564d2073c7b0 2024-09-19T09:29:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.608+0000 7f3dbffff640 1 -- 172.21.15.16:0/3120860128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3dd4003990 con 0x564d2073c7b0 2024-09-19T09:29:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.608+0000 7f3de8fd9740 10 client.16071 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.608+0000 7f3de8fd9740 1 -- 172.21.15.16:0/3120860128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564d20755c40 con 0x564d2073c7b0 2024-09-19T09:29:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.608+0000 7f3dbffff640 1 -- 172.21.15.16:0/3120860128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dd40041d0 con 0x564d2073c7b0 2024-09-19T09:29:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.608+0000 7f3dbffff640 1 -- 172.21.15.16:0/3120860128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3dd4004a90 con 0x564d2073c7b0 2024-09-19T09:29:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.608+0000 7f3de8fd9740 10 client.16071 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.608+0000 7f3de8fd9740 10 client.16071 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.608+0000 7f3de8fd9740 20 client.16071 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.608+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x564d20756f90 0x564d20759380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.608+0000 7f3de8fd9740 4 client.16071 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.608+0000 7f3de8fd9740 1 -- 172.21.15.16:0/3120860128 --> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d206af860 con 0x564d20756f90 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.608+0000 7f3ddffff640 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x564d20756f90 0x564d20759380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.609+0000 7f3ddffff640 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x564d20756f90 0x564d20759380 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e144781da7458a3b server_cookie=98384a0ece2d7d05 in_seq=0 out_seq=0 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.609+0000 7f3dbffff640 10 client.16071 ms_handle_connect on v2:172.21.15.142:6832/3906366829 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3dbffff640 1 -- 172.21.15.16:0/3120860128 <== mds.4 v2:172.21.15.142:6832/3906366829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d206af860 con 0x564d20756f90 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3dbffff640 10 client.16071 handle_command_reply: tid=0 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3de8fd9740 1 client.16071 shutdown 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3de8fd9740 1 -- 172.21.15.16:0/3120860128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 msgr2=0x564d20746eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 0x564d20746eb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4000c00 tx=0x7f3dd400bfd0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3de8fd9740 1 -- 172.21.15.16:0/3120860128 shutdown_connections 2024-09-19T09:29:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x564d20756f90 0x564d20759380 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d20747550 0x564d20744750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073dd20 0x564d207441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/3120860128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 0x564d20746eb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3de8fd9740 1 -- 172.21.15.16:0/3120860128 >> 172.21.15.16:0/3120860128 conn(0x564d20705310 msgr2=0x564d2073eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3dbffff640 0 client.16071 ms_handle_reset on v2:172.21.15.142:6832/3906366829 2024-09-19T09:29:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3de8fd9740 1 -- 172.21.15.16:0/3120860128 shutdown_connections 2024-09-19T09:29:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3de8fd9740 1 -- 172.21.15.16:0/3120860128 wait complete. 2024-09-19T09:29:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.610+0000 7f3de8fd9740 20 client.16071 trim_cache size 0 max 16384 2024-09-19T09:29:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.637+0000 7f3de8fd9740 1 Processor -- start 2024-09-19T09:29:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.637+0000 7f3de8fd9740 1 -- start start 2024-09-19T09:29:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.637+0000 7f3de8fd9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073c7b0 0x564d20753520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.637+0000 7f3de8fd9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073dd20 0x564d20733b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.637+0000 7f3de8fd9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d20747550 0x564d20748940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.637+0000 7f3de8fd9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3de01af6f0 con 0x564d2073dd20 2024-09-19T09:29:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.637+0000 7f3de8fd9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3de01b1230 con 0x564d2073c7b0 2024-09-19T09:29:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.637+0000 7f3de8fd9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3de01afdc0 con 0x564d20747550 2024-09-19T09:29:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.637+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073dd20 0x564d20733b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.637+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073dd20 0x564d20733b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53532/0 (socket says 172.21.15.16:53532) 2024-09-19T09:29:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.637+0000 7f3ddffff640 1 -- 172.21.15.16:0/4051792301 learned_addr learned my addr 172.21.15.16:0/4051792301 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.637+0000 7f3de499c640 1 --2- 172.21.15.16:0/4051792301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073c7b0 0x564d20753520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.637+0000 7f3de519d640 1 --2- 172.21.15.16:0/4051792301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d20747550 0x564d20748940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.638+0000 7f3ddffff640 1 -- 172.21.15.16:0/4051792301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d20747550 msgr2=0x564d20748940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.638+0000 7f3ddffff640 1 --2- 172.21.15.16:0/4051792301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d20747550 0x564d20748940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.638+0000 7f3ddffff640 1 -- 172.21.15.16:0/4051792301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073c7b0 msgr2=0x564d20753520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.638+0000 7f3ddffff640 1 --2- 172.21.15.16:0/4051792301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073c7b0 0x564d20753520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.638+0000 7f3ddffff640 1 -- 172.21.15.16:0/4051792301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d20739b40 con 0x564d2073dd20 2024-09-19T09:29:44.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.638+0000 7f3de499c640 1 --2- 172.21.15.16:0/4051792301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073c7b0 0x564d20753520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:44.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.638+0000 7f3ddffff640 1 --2- 172.21.15.16:0/4051792301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073dd20 0x564d20733b90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc006010 tx=0x7f3dcc040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f0b3dafb9ebb03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:44.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.638+0000 7f3dca7fc640 1 -- 172.21.15.16:0/4051792301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc035040 con 0x564d2073dd20 2024-09-19T09:29:44.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.638+0000 7f3dca7fc640 1 -- 172.21.15.16:0/4051792301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dcc05b070 con 0x564d2073dd20 2024-09-19T09:29:44.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.638+0000 7f3dca7fc640 1 -- 172.21.15.16:0/4051792301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc055c20 con 0x564d2073dd20 2024-09-19T09:29:44.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.639+0000 7f3de8fd9740 1 -- 172.21.15.16:0/4051792301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073dd20 msgr2=0x564d20733b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.639+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/4051792301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073dd20 0x564d20733b90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc006010 tx=0x7f3dcc040040 comp rx=0 tx=0).stop 2024-09-19T09:29:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.639+0000 7f3de8fd9740 1 -- 172.21.15.16:0/4051792301 shutdown_connections 2024-09-19T09:29:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.639+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/4051792301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d20747550 0x564d20748940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.639+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/4051792301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073dd20 0x564d20733b90 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.639+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/4051792301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2073c7b0 0x564d20753520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.639+0000 7f3de8fd9740 1 -- 172.21.15.16:0/4051792301 >> 172.21.15.16:0/4051792301 conn(0x564d20705310 msgr2=0x564d20758fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.640+0000 7f3de8fd9740 1 -- 172.21.15.16:0/4051792301 shutdown_connections 2024-09-19T09:29:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.640+0000 7f3de8fd9740 1 -- 172.21.15.16:0/4051792301 wait complete. 2024-09-19T09:29:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.640+0000 7f3de8fd9740 1 -- start start 2024-09-19T09:29:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.640+0000 7f3de8fd9740 1 Processor -- start 2024-09-19T09:29:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.640+0000 7f3de8fd9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 0x564d20740be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de8fd9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d2073dd20 0x564d20741120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de8fd9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d20747550 0x564d2074e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de499c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 0x564d20740be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de8fd9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3de01b0b60 con 0x564d2073c7b0 2024-09-19T09:29:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de8fd9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3de01b0490 con 0x564d20747550 2024-09-19T09:29:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de8fd9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3de01ad6b0 con 0x564d2073dd20 2024-09-19T09:29:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de499c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 0x564d20740be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53536/0 (socket says 172.21.15.16:53536) 2024-09-19T09:29:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de499c640 1 -- 172.21.15.16:0/2141462512 learned_addr learned my addr 172.21.15.16:0/2141462512 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3ddffff640 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d2073dd20 0x564d20741120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de519d640 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d20747550 0x564d2074e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de499c640 1 -- 172.21.15.16:0/2141462512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d2073dd20 msgr2=0x564d20741120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de499c640 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d2073dd20 0x564d20741120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de499c640 1 -- 172.21.15.16:0/2141462512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d20747550 msgr2=0x564d2074e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de499c640 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d20747550 0x564d2074e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de499c640 1 -- 172.21.15.16:0/2141462512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d2073f300 con 0x564d2073c7b0 2024-09-19T09:29:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.641+0000 7f3de519d640 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d20747550 0x564d2074e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.642+0000 7f3de8fd9740 10 client.0 _ll_register_callbacks cb 0x7f3dd80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.642+0000 7f3de8fd9740 1 -- 172.21.15.16:0/2141462512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dcc05b070 con 0x564d2073c7b0 2024-09-19T09:29:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.642+0000 7f3de499c640 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 0x564d20740be0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4004e60 tx=0x7f3dd4011560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ca6f630f84940a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.642+0000 7f3dbffff640 10 client.16077 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.642+0000 7f3dbffff640 1 -- 172.21.15.16:0/2141462512 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd403e050 con 0x564d2073c7b0 2024-09-19T09:29:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.642+0000 7f3dbffff640 1 -- 172.21.15.16:0/2141462512 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f3dd4003b30 con 0x564d2073c7b0 2024-09-19T09:29:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.642+0000 7f3dbffff640 1 -- 172.21.15.16:0/2141462512 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3dd4004140 con 0x564d2073c7b0 2024-09-19T09:29:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.642+0000 7f3de8fd9740 10 client.16077 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.642+0000 7f3de8fd9740 1 -- 172.21.15.16:0/2141462512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=28}) -- 0x564d2074fc00 con 0x564d2073c7b0 2024-09-19T09:29:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.642+0000 7f3dbffff640 1 -- 172.21.15.16:0/2141462512 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3dd40035f0 con 0x564d2073c7b0 2024-09-19T09:29:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.642+0000 7f3de8fd9740 10 client.16077 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.642+0000 7f3de8fd9740 10 client.16077 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-19T09:29:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.642+0000 7f3de8fd9740 20 client.16077 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.643+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x564d207ca320 0x564d207cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.643+0000 7f3de8fd9740 4 client.16077 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:29:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.643+0000 7f3de8fd9740 1 -- 172.21.15.16:0/2141462512 --> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d206a9d70 con 0x564d207ca320 2024-09-19T09:29:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.643+0000 7f3ddffff640 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x564d207ca320 0x564d207cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.643+0000 7f3ddffff640 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x564d207ca320 0x564d207cc710 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b3c5b86024833157 server_cookie=f22c5dffb4f0b4e6 in_seq=0 out_seq=0 2024-09-19T09:29:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.643+0000 7f3dbffff640 10 client.16077 ms_handle_connect on v2:172.21.15.142:6832/3906366829 2024-09-19T09:29:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3dbffff640 1 -- 172.21.15.16:0/2141462512 <== mds.4 v2:172.21.15.142:6832/3906366829 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d206a9d70 con 0x564d207ca320 2024-09-19T09:29:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3dbffff640 10 client.16077 handle_command_reply: tid=0 2024-09-19T09:29:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3de8fd9740 1 client.16077 shutdown 2024-09-19T09:29:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3de8fd9740 1 -- 172.21.15.16:0/2141462512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 msgr2=0x564d20740be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 0x564d20740be0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4004e60 tx=0x7f3dd4011560 comp rx=0 tx=0).stop 2024-09-19T09:29:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3de8fd9740 1 -- 172.21.15.16:0/2141462512 shutdown_connections 2024-09-19T09:29:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.142:6832/3906366829,v1:172.21.15.142:6833/3906366829] conn(0x564d207ca320 0x564d207cc710 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564d2073dd20 0x564d20741120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d20747550 0x564d2074e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3de8fd9740 1 --2- 172.21.15.16:0/2141462512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2073c7b0 0x564d20740be0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3de8fd9740 1 -- 172.21.15.16:0/2141462512 >> 172.21.15.16:0/2141462512 conn(0x564d20705310 msgr2=0x564d2073eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3dbffff640 0 client.16077 ms_handle_reset on v2:172.21.15.142:6832/3906366829 2024-09-19T09:29:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3de8fd9740 1 -- 172.21.15.16:0/2141462512 shutdown_connections 2024-09-19T09:29:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3de8fd9740 1 -- 172.21.15.16:0/2141462512 wait complete. 2024-09-19T09:29:44.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.644+0000 7f3de8fd9740 20 client.16077 trim_cache size 0 max 16384 2024-09-19T09:29:44.651 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:29:44.651 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.645+0000 7f3de599e640 1 -- 172.21.15.16:0/2490691913 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3dd00a3240 msgr2=0x7f3dd00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.645+0000 7f3de599e640 1 --2- 172.21.15.16:0/2490691913 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3dd00a3240 0x7f3dd00a5700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc031550 tx=0x7f3dcc009560 comp rx=0 tx=0).stop 2024-09-19T09:29:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.645+0000 7f3de599e640 1 -- 172.21.15.16:0/2490691913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de010bfc0 msgr2=0x7f3de019cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.645+0000 7f3de599e640 1 --2- 172.21.15.16:0/2490691913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de010bfc0 0x7f3de019cba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009fd0 tx=0x7f3dd800ef10 comp rx=0 tx=0).stop 2024-09-19T09:29:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.645+0000 7f3de599e640 1 -- 172.21.15.16:0/2490691913 shutdown_connections 2024-09-19T09:29:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.645+0000 7f3de599e640 1 --2- 172.21.15.16:0/2490691913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de010bfc0 0x7f3de019cba0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.645+0000 7f3de599e640 1 --2- 172.21.15.16:0/2490691913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3de01034b0 0x7f3de0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.645+0000 7f3de599e640 1 --2- 172.21.15.16:0/2490691913 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3dd00a3240 0x7f3dd00a5700 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.645+0000 7f3de599e640 1 --2- 172.21.15.16:0/2490691913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3de0102b90 0x7f3de01982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.645+0000 7f3de599e640 1 -- 172.21.15.16:0/2490691913 >> 172.21.15.16:0/2490691913 conn(0x7f3de00fc610 msgr2=0x7f3de00fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.645+0000 7f3de599e640 1 -- 172.21.15.16:0/2490691913 shutdown_connections 2024-09-19T09:29:44.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:44.645+0000 7f3de599e640 1 -- 172.21.15.16:0/2490691913 wait complete. 2024-09-19T09:29:44.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:44 smithi016 ceph-mon[30979]: pgmap v402: 97 pgs: 97 active+clean; 148 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 173 MiB/s rd, 1.9 MiB/s wr, 676 op/s 2024-09-19T09:29:44.878 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:44 smithi142 ceph-mon[35032]: pgmap v402: 97 pgs: 97 active+clean; 148 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 173 MiB/s rd, 1.9 MiB/s wr, 676 op/s 2024-09-19T09:29:44.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:44 smithi097 ceph-mon[34864]: pgmap v402: 97 pgs: 97 active+clean; 148 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 173 MiB/s rd, 1.9 MiB/s wr, 676 op/s 2024-09-19T09:29:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.486+0000 7f1615508640 1 -- 172.21.15.16:0/3131408773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1610106f10 msgr2=0x7f161010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.486+0000 7f1615508640 1 --2- 172.21.15.16:0/3131408773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1610106f10 0x7f161010ea90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1608009ca0 tx=0x7f160802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:29:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.486+0000 7f1615508640 1 -- 172.21.15.16:0/3131408773 shutdown_connections 2024-09-19T09:29:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.486+0000 7f1615508640 1 --2- 172.21.15.16:0/3131408773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1610106f10 0x7f161010ea90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.486+0000 7f1615508640 1 --2- 172.21.15.16:0/3131408773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1610102790 0x7f16101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.486+0000 7f1615508640 1 --2- 172.21.15.16:0/3131408773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1610101de0 0x7f16101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.486+0000 7f1615508640 1 -- 172.21.15.16:0/3131408773 >> 172.21.15.16:0/3131408773 conn(0x7f16100fd8c0 msgr2=0x7f16100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.486+0000 7f1615508640 1 -- 172.21.15.16:0/3131408773 shutdown_connections 2024-09-19T09:29:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.486+0000 7f1615508640 1 -- 172.21.15.16:0/3131408773 wait complete. 2024-09-19T09:29:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.486+0000 7f1615508640 1 Processor -- start 2024-09-19T09:29:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.486+0000 7f1615508640 1 -- start start 2024-09-19T09:29:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1615508640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1610101de0 0x7f1610198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1615508640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1610102790 0x7f1610198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1615508640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1610106f10 0x7f161019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1615508640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1610111fc0 con 0x7f1610102790 2024-09-19T09:29:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1615508640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1610111e40 con 0x7f1610101de0 2024-09-19T09:29:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1615508640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1610112140 con 0x7f1610106f10 2024-09-19T09:29:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1607fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1610102790 0x7f1610198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1607fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1610102790 0x7f1610198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54974/0 (socket says 172.21.15.16:54974) 2024-09-19T09:29:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1607fff640 1 -- 172.21.15.16:0/3345310116 learned_addr learned my addr 172.21.15.16:0/3345310116 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f160effd640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1610101de0 0x7f1610198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f160f7fe640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1610106f10 0x7f161019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1607fff640 1 -- 172.21.15.16:0/3345310116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1610106f10 msgr2=0x7f161019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1607fff640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1610106f10 0x7f161019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1607fff640 1 -- 172.21.15.16:0/3345310116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1610101de0 msgr2=0x7f1610198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1607fff640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1610101de0 0x7f1610198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1607fff640 1 -- 172.21.15.16:0/3345310116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161019d2f0 con 0x7f1610102790 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f1607fff640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1610102790 0x7f1610198880 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f15f800d950 tx=0x7f15f800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.487+0000 7f160cff9640 1 -- 172.21.15.16:0/3345310116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8017070 con 0x7f1610102790 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.488+0000 7f160cff9640 1 -- 172.21.15.16:0/3345310116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f800de80 con 0x7f1610102790 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.488+0000 7f1615508640 1 -- 172.21.15.16:0/3345310116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161019d5e0 con 0x7f1610102790 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.488+0000 7f160cff9640 1 -- 172.21.15.16:0/3345310116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f800f620 con 0x7f1610102790 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.488+0000 7f1615508640 1 -- 172.21.15.16:0/3345310116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16101a4ec0 con 0x7f1610102790 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.489+0000 7f1615508640 1 -- 172.21.15.16:0/3345310116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16101021c0 con 0x7f1610102790 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.490+0000 7f160cff9640 1 -- 172.21.15.16:0/3345310116 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f15f8009c30 con 0x7f1610102790 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.491+0000 7f160cff9640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f15fc0a32c0 0x7f15fc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.491+0000 7f160effd640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f15fc0a32c0 0x7f15fc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.491+0000 7f160cff9640 1 -- 172.21.15.16:0/3345310116 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f15f80cf450 con 0x7f1610102790 2024-09-19T09:29:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.491+0000 7f160effd640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f15fc0a32c0 0x7f15fc0a5780 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f160000abe0 tx=0x7f160000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.495+0000 7f160cff9640 1 -- 172.21.15.16:0/3345310116 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f15f809c170 con 0x7f1610102790 2024-09-19T09:29:45.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.706+0000 7f1615508640 1 -- 172.21.15.16:0/3345310116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f161002d7c0 con 0x7f1610102790 2024-09-19T09:29:45.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.707+0000 7f160cff9640 1 -- 172.21.15.16:0/3345310116 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f15f809ba70 con 0x7f1610102790 2024-09-19T09:29:45.708 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:29:45.708 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:29:45.708 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:29:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.709+0000 7f1615508640 1 -- 172.21.15.16:0/3345310116 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f15fc0a32c0 msgr2=0x7f15fc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.709+0000 7f1615508640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f15fc0a32c0 0x7f15fc0a5780 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f160000abe0 tx=0x7f160000a2d0 comp rx=0 tx=0).stop 2024-09-19T09:29:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.709+0000 7f1615508640 1 -- 172.21.15.16:0/3345310116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1610102790 msgr2=0x7f1610198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.709+0000 7f1615508640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1610102790 0x7f1610198880 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f15f800d950 tx=0x7f15f800dd50 comp rx=0 tx=0).stop 2024-09-19T09:29:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.709+0000 7f1615508640 1 -- 172.21.15.16:0/3345310116 shutdown_connections 2024-09-19T09:29:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.709+0000 7f1615508640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f15fc0a32c0 0x7f15fc0a5780 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.709+0000 7f1615508640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1610106f10 0x7f161019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.709+0000 7f1615508640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1610102790 0x7f1610198880 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.709+0000 7f1615508640 1 --2- 172.21.15.16:0/3345310116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1610101de0 0x7f1610198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.709+0000 7f1615508640 1 -- 172.21.15.16:0/3345310116 >> 172.21.15.16:0/3345310116 conn(0x7f16100fd8c0 msgr2=0x7f16100fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:45.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.709+0000 7f1615508640 1 -- 172.21.15.16:0/3345310116 shutdown_connections 2024-09-19T09:29:45.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:45.709+0000 7f1615508640 1 -- 172.21.15.16:0/3345310116 wait complete. 2024-09-19T09:29:46.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T09:29:46.607 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:46 smithi016 ceph-mon[30979]: pgmap v403: 97 pgs: 97 active+clean; 148 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s rd, 1.5 MiB/s wr, 541 op/s 2024-09-19T09:29:46.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:46 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3345310116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:46.643 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --cancel --all' 2024-09-19T09:29:46.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --cancel --all 2024-09-19T09:29:46.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:46 smithi142 ceph-mon[35032]: pgmap v403: 97 pgs: 97 active+clean; 148 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s rd, 1.5 MiB/s wr, 541 op/s 2024-09-19T09:29:46.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:46 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3345310116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:46.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:46 smithi097 ceph-mon[34864]: pgmap v403: 97 pgs: 97 active+clean; 148 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s rd, 1.5 MiB/s wr, 541 op/s 2024-09-19T09:29:46.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:46 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3345310116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:29:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.119+0000 7fce92507640 1 -- 172.21.15.16:0/4144112461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c103720 msgr2=0x7fce8c10bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.119+0000 7fce92507640 1 --2- 172.21.15.16:0/4144112461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c103720 0x7fce8c10bc10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fce80009900 tx=0x7fce8002a480 comp rx=0 tx=0).stop 2024-09-19T09:29:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.119+0000 7fce92507640 1 -- 172.21.15.16:0/4144112461 shutdown_connections 2024-09-19T09:29:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.119+0000 7fce92507640 1 --2- 172.21.15.16:0/4144112461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce8c10c1a0 0x7fce8c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.119+0000 7fce92507640 1 --2- 172.21.15.16:0/4144112461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c103720 0x7fce8c10bc10 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.119+0000 7fce92507640 1 --2- 172.21.15.16:0/4144112461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce8c102e00 0x7fce8c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.119+0000 7fce92507640 1 -- 172.21.15.16:0/4144112461 >> 172.21.15.16:0/4144112461 conn(0x7fce8c0fc7a0 msgr2=0x7fce8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:47.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.119+0000 7fce92507640 1 -- 172.21.15.16:0/4144112461 shutdown_connections 2024-09-19T09:29:47.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.119+0000 7fce92507640 1 -- 172.21.15.16:0/4144112461 wait complete. 2024-09-19T09:29:47.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.120+0000 7fce92507640 1 Processor -- start 2024-09-19T09:29:47.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.120+0000 7fce92507640 1 -- start start 2024-09-19T09:29:47.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.120+0000 7fce92507640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce8c102e00 0x7fce8c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.120+0000 7fce92507640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce8c103720 0x7fce8c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.120+0000 7fce92507640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c10c1a0 0x7fce8c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.120+0000 7fce92507640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c111fb0 con 0x7fce8c10c1a0 2024-09-19T09:29:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.120+0000 7fce92507640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c111e30 con 0x7fce8c102e00 2024-09-19T09:29:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.120+0000 7fce92507640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c112130 con 0x7fce8c103720 2024-09-19T09:29:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce90aa4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c10c1a0 0x7fce8c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce90aa4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c10c1a0 0x7fce8c19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54990/0 (socket says 172.21.15.16:54990) 2024-09-19T09:29:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce90aa4640 1 -- 172.21.15.16:0/3758748896 learned_addr learned my addr 172.21.15.16:0/3758748896 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce8bfff640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce8c102e00 0x7fce8c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce8c103720 0x7fce8c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce90aa4640 1 -- 172.21.15.16:0/3758748896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce8c103720 msgr2=0x7fce8c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce90aa4640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce8c103720 0x7fce8c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce90aa4640 1 -- 172.21.15.16:0/3758748896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce8c102e00 msgr2=0x7fce8c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce90aa4640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce8c102e00 0x7fce8c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce90aa4640 1 -- 172.21.15.16:0/3758748896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8c19d440 con 0x7fce8c10c1a0 2024-09-19T09:29:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce90aa4640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c10c1a0 0x7fce8c19cd60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fce8400b970 tx=0x7fce8400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce897fa640 1 -- 172.21.15.16:0/3758748896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84019070 con 0x7fce8c10c1a0 2024-09-19T09:29:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce897fa640 1 -- 172.21.15.16:0/3758748896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce8400cb40 con 0x7fce8c10c1a0 2024-09-19T09:29:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce92507640 1 -- 172.21.15.16:0/3758748896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8c19d730 con 0x7fce8c10c1a0 2024-09-19T09:29:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce897fa640 1 -- 172.21.15.16:0/3758748896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84007650 con 0x7fce8c10c1a0 2024-09-19T09:29:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.121+0000 7fce92507640 1 -- 172.21.15.16:0/3758748896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8c1a5010 con 0x7fce8c10c1a0 2024-09-19T09:29:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.124+0000 7fce897fa640 1 -- 172.21.15.16:0/3758748896 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fce8401c070 con 0x7fce8c10c1a0 2024-09-19T09:29:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.124+0000 7fce948e0740 1 Processor -- start 2024-09-19T09:29:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.124+0000 7fce948e0740 1 -- start start 2024-09-19T09:29:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.124+0000 7fce948e0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.124+0000 7fce948e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.124+0000 7fce948e0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5bf370 0x55e7ab5c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.124+0000 7fce948e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e7ab51a2f0 con 0x55e7ab5b45d0 2024-09-19T09:29:47.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.124+0000 7fce948e0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e7ab4c1cf0 con 0x55e7ab5bf370 2024-09-19T09:29:47.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.124+0000 7fce948e0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e7ab5162b0 con 0x55e7ab5b5b40 2024-09-19T09:29:47.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.124+0000 7fce8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.124+0000 7fce8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55000/0 (socket says 172.21.15.16:55000) 2024-09-19T09:29:47.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.124+0000 7fce8bfff640 1 -- 172.21.15.16:0/56381711 learned_addr learned my addr 172.21.15.16:0/56381711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:47.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce897fa640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fce740a32c0 0x7fce740a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/56381711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce8bfff640 1 -- 172.21.15.16:0/56381711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5b5b40 msgr2=0x55e7ab5b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:47.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce8bfff640 1 --2- 172.21.15.16:0/56381711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce8bfff640 1 -- 172.21.15.16:0/56381711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5bf370 msgr2=0x55e7ab5c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:47.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce8bfff640 1 --2- 172.21.15.16:0/56381711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5bf370 0x55e7ab5c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce8bfff640 1 -- 172.21.15.16:0/56381711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ab4a63a0 con 0x55e7ab5b45d0 2024-09-19T09:29:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/56381711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce90aa4640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fce740a32c0 0x7fce740a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce897fa640 1 -- 172.21.15.16:0/3758748896 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fce840ceed0 con 0x7fce8c10c1a0 2024-09-19T09:29:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce8bfff640 1 --2- 172.21.15.16:0/56381711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5b4a10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00bd10 tx=0x7fce7c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cfc2c8ccbc9301 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce727fc640 1 -- 172.21.15.16:0/56381711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c004030 con 0x55e7ab5b45d0 2024-09-19T09:29:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce727fc640 1 -- 172.21.15.16:0/56381711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce7c0041d0 con 0x55e7ab5b45d0 2024-09-19T09:29:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.125+0000 7fce727fc640 1 -- 172.21.15.16:0/56381711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c0117e0 con 0x55e7ab5b45d0 2024-09-19T09:29:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.126+0000 7fce948e0740 1 -- 172.21.15.16:0/56381711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 msgr2=0x55e7ab5b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.126+0000 7fce90aa4640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fce740a32c0 0x7fce740a5780 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fce8409b4f0 tx=0x7fce8400e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.126+0000 7fce948e0740 1 --2- 172.21.15.16:0/56381711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5b4a10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00bd10 tx=0x7fce7c0107a0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.126+0000 7fce948e0740 1 -- 172.21.15.16:0/56381711 shutdown_connections 2024-09-19T09:29:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.126+0000 7fce948e0740 1 --2- 172.21.15.16:0/56381711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5bf370 0x55e7ab5c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.126+0000 7fce948e0740 1 --2- 172.21.15.16:0/56381711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5b4a10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.126+0000 7fce948e0740 1 --2- 172.21.15.16:0/56381711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.126+0000 7fce948e0740 1 -- 172.21.15.16:0/56381711 >> 172.21.15.16:0/56381711 conn(0x55e7ab57d310 msgr2=0x55e7ab5b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.126+0000 7fce948e0740 1 -- 172.21.15.16:0/56381711 shutdown_connections 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.126+0000 7fce948e0740 1 -- 172.21.15.16:0/56381711 wait complete. 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.126+0000 7fce948e0740 1 -- start start 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce948e0740 1 Processor -- start 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce948e0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5bed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce948e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5bc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce948e0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 0x55e7ab5bc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce948e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e7ab5c2070 con 0x55e7ab5b5b40 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce948e0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e7ab4d5060 con 0x55e7ab5b45d0 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce948e0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e7ab5c21f0 con 0x55e7ab5bf370 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce948e0740 10 client.0 _ll_register_callbacks cb 0x55e7ab2288c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:47.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5bc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5bc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55004/0 (socket says 172.21.15.16:55004) 2024-09-19T09:29:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce8bfff640 1 -- 172.21.15.16:0/799788016 learned_addr learned my addr 172.21.15.16:0/799788016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/799788016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5bed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce8bfff640 1 -- 172.21.15.16:0/799788016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 msgr2=0x55e7ab5bc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce8bfff640 1 --2- 172.21.15.16:0/799788016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 0x55e7ab5bc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce8bfff640 1 -- 172.21.15.16:0/799788016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b45d0 msgr2=0x55e7ab5bed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce8bfff640 1 --2- 172.21.15.16:0/799788016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5bed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.127+0000 7fce8bfff640 1 -- 172.21.15.16:0/799788016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ab28b3c0 con 0x55e7ab5b5b40 2024-09-19T09:29:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce8bfff640 1 --2- 172.21.15.16:0/799788016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5bc000 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0114a0 tx=0x7fce7c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6825ae676325137e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce6b7fe640 1 -- 172.21.15.16:0/799788016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c029060 con 0x55e7ab5b5b40 2024-09-19T09:29:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce6b7fe640 1 -- 172.21.15.16:0/799788016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fce7c0035f0 con 0x55e7ab5b5b40 2024-09-19T09:29:47.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce948e0740 1 -- 172.21.15.16:0/799788016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce7c0041d0 con 0x55e7ab5b5b40 2024-09-19T09:29:47.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce6b7fe640 1 -- 172.21.15.16:0/799788016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fce7c0037f0 con 0x55e7ab5b5b40 2024-09-19T09:29:47.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce948e0740 10 client.16095 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:47.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce948e0740 1 -- 172.21.15.16:0/799788016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e7ab5cda60 con 0x55e7ab5b5b40 2024-09-19T09:29:47.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce6b7fe640 1 -- 172.21.15.16:0/799788016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce7c0041d0 con 0x55e7ab5b5b40 2024-09-19T09:29:47.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce6b7fe640 1 -- 172.21.15.16:0/799788016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fce7c02e070 con 0x55e7ab5b5b40 2024-09-19T09:29:47.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce948e0740 10 client.16095 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:47.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce948e0740 10 client.16095 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-19T09:29:47.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce948e0740 20 client.16095 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:47.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce948e0740 1 --2- 172.21.15.16:0/799788016 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55e7ab5cee60 0x55e7ab5d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce948e0740 4 client.16095 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:47.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.128+0000 7fce948e0740 1 -- 172.21.15.16:0/799788016 --> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7ab527860 con 0x55e7ab5cee60 2024-09-19T09:29:47.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.129+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/799788016 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55e7ab5cee60 0x55e7ab5d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.129+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/799788016 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55e7ab5cee60 0x55e7ab5d1250 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d9752cfdd20bce8 server_cookie=61b15dda004f271 in_seq=0 out_seq=0 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.129+0000 7fce6b7fe640 10 client.16095 ms_handle_connect on v2:172.21.15.97:6832/108652274 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce6b7fe640 1 -- 172.21.15.16:0/799788016 <== mds.3 v2:172.21.15.97:6832/108652274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7ab527860 con 0x55e7ab5cee60 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce6b7fe640 10 client.16095 handle_command_reply: tid=0 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce948e0740 1 client.16095 shutdown 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce948e0740 1 -- 172.21.15.16:0/799788016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b5b40 msgr2=0x55e7ab5bc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce948e0740 1 --2- 172.21.15.16:0/799788016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5bc000 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0114a0 tx=0x7fce7c003f90 comp rx=0 tx=0).stop 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce948e0740 1 -- 172.21.15.16:0/799788016 shutdown_connections 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce948e0740 1 --2- 172.21.15.16:0/799788016 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55e7ab5cee60 0x55e7ab5d1250 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce948e0740 1 --2- 172.21.15.16:0/799788016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 0x55e7ab5bc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce948e0740 1 --2- 172.21.15.16:0/799788016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5bc000 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce6b7fe640 0 client.16095 ms_handle_reset on v2:172.21.15.97:6832/108652274 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce948e0740 1 --2- 172.21.15.16:0/799788016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5bed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce948e0740 1 -- 172.21.15.16:0/799788016 >> 172.21.15.16:0/799788016 conn(0x55e7ab57d310 msgr2=0x55e7ab5b6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:47.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce948e0740 1 -- 172.21.15.16:0/799788016 shutdown_connections 2024-09-19T09:29:47.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce948e0740 1 -- 172.21.15.16:0/799788016 wait complete. 2024-09-19T09:29:47.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.130+0000 7fce948e0740 20 client.16095 trim_cache size 0 max 16384 2024-09-19T09:29:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.144+0000 7fce948e0740 1 Processor -- start 2024-09-19T09:29:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.144+0000 7fce948e0740 1 -- start start 2024-09-19T09:29:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.144+0000 7fce948e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5bdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.144+0000 7fce948e0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.144+0000 7fce948e0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 0x55e7ab5b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.144+0000 7fce948e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c1ad670 con 0x55e7ab5b45d0 2024-09-19T09:29:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.144+0000 7fce948e0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c1af1b0 con 0x55e7ab5b5b40 2024-09-19T09:29:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.144+0000 7fce948e0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c1add40 con 0x55e7ab5bf370 2024-09-19T09:29:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.145+0000 7fce8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5bdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.145+0000 7fce8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5bdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55012/0 (socket says 172.21.15.16:55012) 2024-09-19T09:29:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.145+0000 7fce8b7fe640 1 -- 172.21.15.16:0/538138837 learned_addr learned my addr 172.21.15.16:0/538138837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.145+0000 7fce8bfff640 1 --2- 172.21.15.16:0/538138837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.145+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/538138837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 0x55e7ab5b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.145+0000 7fce8b7fe640 1 -- 172.21.15.16:0/538138837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 msgr2=0x55e7ab5b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.145+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/538138837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 0x55e7ab5b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.145+0000 7fce8b7fe640 1 -- 172.21.15.16:0/538138837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b5b40 msgr2=0x55e7ab5b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.145+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/538138837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.145+0000 7fce8b7fe640 1 -- 172.21.15.16:0/538138837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ab5b1960 con 0x55e7ab5b45d0 2024-09-19T09:29:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.145+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/538138837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5bdf70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fce80005ec0 tx=0x7fce8002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb67f6dcff9e4be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.145+0000 7fce71ffb640 1 -- 172.21.15.16:0/538138837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80035040 con 0x55e7ab5b45d0 2024-09-19T09:29:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.146+0000 7fce71ffb640 1 -- 172.21.15.16:0/538138837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce80005500 con 0x55e7ab5b45d0 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.146+0000 7fce71ffb640 1 -- 172.21.15.16:0/538138837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80004e40 con 0x55e7ab5b45d0 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.146+0000 7fce948e0740 1 -- 172.21.15.16:0/538138837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 msgr2=0x55e7ab5bdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.146+0000 7fce948e0740 1 --2- 172.21.15.16:0/538138837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5bdf70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fce80005ec0 tx=0x7fce8002a480 comp rx=0 tx=0).stop 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.146+0000 7fce948e0740 1 -- 172.21.15.16:0/538138837 shutdown_connections 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.146+0000 7fce948e0740 1 --2- 172.21.15.16:0/538138837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 0x55e7ab5b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.146+0000 7fce948e0740 1 --2- 172.21.15.16:0/538138837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.146+0000 7fce948e0740 1 --2- 172.21.15.16:0/538138837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5bdf70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.146+0000 7fce948e0740 1 -- 172.21.15.16:0/538138837 >> 172.21.15.16:0/538138837 conn(0x55e7ab57d310 msgr2=0x55e7ab5bd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.146+0000 7fce948e0740 1 -- 172.21.15.16:0/538138837 shutdown_connections 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.146+0000 7fce948e0740 1 -- 172.21.15.16:0/538138837 wait complete. 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.147+0000 7fce948e0740 1 -- start start 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.147+0000 7fce948e0740 1 Processor -- start 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce948e0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce948e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce948e0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 0x55e7ab5b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce948e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c1aeae0 con 0x55e7ab5b5b40 2024-09-19T09:29:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce948e0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c1ae410 con 0x55e7ab5b45d0 2024-09-19T09:29:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce948e0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c1ab630 con 0x55e7ab5bf370 2024-09-19T09:29:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce948e0740 10 client.0 _ll_register_callbacks cb 0x7fce7c00bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce8b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55026/0 (socket says 172.21.15.16:55026) 2024-09-19T09:29:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce8bfff640 1 -- 172.21.15.16:0/3353167194 learned_addr learned my addr 172.21.15.16:0/3353167194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/3353167194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 0x55e7ab5b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce8b7fe640 1 -- 172.21.15.16:0/3353167194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 msgr2=0x55e7ab5b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/3353167194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 0x55e7ab5b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce8b7fe640 1 -- 172.21.15.16:0/3353167194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b5b40 msgr2=0x55e7ab5b8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/3353167194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce8b7fe640 1 -- 172.21.15.16:0/3353167194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ab5b9bf0 con 0x55e7ab5b45d0 2024-09-19T09:29:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce8b7fe640 1 --2- 172.21.15.16:0/3353167194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5c7c50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce80033000 tx=0x7fce80049040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f79132aedcec03aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.148+0000 7fce6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-19T09:29:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.149+0000 7fce6b7fe640 1 -- 172.21.15.16:0/3353167194 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8005a060 con 0x55e7ab5b45d0 2024-09-19T09:29:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.149+0000 7fce6b7fe640 1 -- 172.21.15.16:0/3353167194 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fce80004750 con 0x55e7ab5b45d0 2024-09-19T09:29:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.149+0000 7fce948e0740 1 -- 172.21.15.16:0/3353167194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce80005500 con 0x55e7ab5b45d0 2024-09-19T09:29:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.149+0000 7fce6b7fe640 1 -- 172.21.15.16:0/3353167194 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fce80035040 con 0x55e7ab5b45d0 2024-09-19T09:29:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.149+0000 7fce948e0740 10 client.25661 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.149+0000 7fce948e0740 1 -- 172.21.15.16:0/3353167194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e7ab6398f0 con 0x55e7ab5b45d0 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.149+0000 7fce6b7fe640 1 -- 172.21.15.16:0/3353167194 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce80005500 con 0x55e7ab5b45d0 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.150+0000 7fce6b7fe640 1 -- 172.21.15.16:0/3353167194 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fce80031620 con 0x55e7ab5b45d0 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.150+0000 7fce948e0740 10 client.25661 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.150+0000 7fce948e0740 10 client.25661 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.150+0000 7fce948e0740 20 client.25661 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.150+0000 7fce948e0740 1 --2- 172.21.15.16:0/3353167194 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55e7ab639f60 0x55e7ab63c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.150+0000 7fce948e0740 4 client.25661 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.150+0000 7fce948e0740 1 -- 172.21.15.16:0/3353167194 --> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7ab521d70 con 0x55e7ab639f60 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.150+0000 7fce8bfff640 1 --2- 172.21.15.16:0/3353167194 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55e7ab639f60 0x55e7ab63c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.150+0000 7fce6b7fe640 10 client.25661 ms_handle_connect on v2:172.21.15.97:6832/108652274 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.150+0000 7fce8bfff640 1 --2- 172.21.15.16:0/3353167194 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55e7ab639f60 0x55e7ab63c350 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5e3f4ae821a207fb server_cookie=5f5c48bfc9ebc3c in_seq=0 out_seq=0 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.150+0000 7fce6b7fe640 1 -- 172.21.15.16:0/3353167194 <== mds.3 v2:172.21.15.97:6832/108652274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7ab521d70 con 0x55e7ab639f60 2024-09-19T09:29:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.150+0000 7fce6b7fe640 10 client.25661 handle_command_reply: tid=0 2024-09-19T09:29:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.150+0000 7fce948e0740 1 client.25661 shutdown 2024-09-19T09:29:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce948e0740 1 -- 172.21.15.16:0/3353167194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b45d0 msgr2=0x55e7ab5c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce948e0740 1 --2- 172.21.15.16:0/3353167194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5c7c50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce80033000 tx=0x7fce80049040 comp rx=0 tx=0).stop 2024-09-19T09:29:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce948e0740 1 -- 172.21.15.16:0/3353167194 shutdown_connections 2024-09-19T09:29:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce948e0740 1 --2- 172.21.15.16:0/3353167194 >> [v2:172.21.15.97:6832/108652274,v1:172.21.15.97:6833/108652274] conn(0x55e7ab639f60 0x55e7ab63c350 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce948e0740 1 --2- 172.21.15.16:0/3353167194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7ab5bf370 0x55e7ab5b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce948e0740 1 --2- 172.21.15.16:0/3353167194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e7ab5b5b40 0x55e7ab5b8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce6b7fe640 0 client.25661 ms_handle_reset on v2:172.21.15.97:6832/108652274 2024-09-19T09:29:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce948e0740 1 --2- 172.21.15.16:0/3353167194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ab5b45d0 0x55e7ab5c7c50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce948e0740 1 -- 172.21.15.16:0/3353167194 >> 172.21.15.16:0/3353167194 conn(0x55e7ab57d310 msgr2=0x55e7ab5be080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce948e0740 1 -- 172.21.15.16:0/3353167194 shutdown_connections 2024-09-19T09:29:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce948e0740 1 -- 172.21.15.16:0/3353167194 wait complete. 2024-09-19T09:29:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce948e0740 20 client.25661 trim_cache size 0 max 16384 2024-09-19T09:29:47.163 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:29:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce92507640 1 -- 172.21.15.16:0/3758748896 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fce740a32c0 msgr2=0x7fce740a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce92507640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fce740a32c0 0x7fce740a5780 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fce8409b4f0 tx=0x7fce8400e040 comp rx=0 tx=0).stop 2024-09-19T09:29:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce92507640 1 -- 172.21.15.16:0/3758748896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c10c1a0 msgr2=0x7fce8c19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.151+0000 7fce92507640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c10c1a0 0x7fce8c19cd60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fce8400b970 tx=0x7fce8400be30 comp rx=0 tx=0).stop 2024-09-19T09:29:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.152+0000 7fce92507640 1 -- 172.21.15.16:0/3758748896 shutdown_connections 2024-09-19T09:29:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.152+0000 7fce92507640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fce740a32c0 0x7fce740a5780 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.152+0000 7fce92507640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce8c103720 0x7fce8c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.152+0000 7fce92507640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c10c1a0 0x7fce8c19cd60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.152+0000 7fce92507640 1 --2- 172.21.15.16:0/3758748896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce8c102e00 0x7fce8c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.152+0000 7fce92507640 1 -- 172.21.15.16:0/3758748896 >> 172.21.15.16:0/3758748896 conn(0x7fce8c0fc7a0 msgr2=0x7fce8c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.152+0000 7fce92507640 1 -- 172.21.15.16:0/3758748896 shutdown_connections 2024-09-19T09:29:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:47.152+0000 7fce92507640 1 -- 172.21.15.16:0/3758748896 wait complete. 2024-09-19T09:29:47.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T09:29:47.812 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.287+0000 7fe471fca640 1 -- 172.21.15.16:0/252051825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe46c102dd0 msgr2=0x7fe46c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.288+0000 7fe471fca640 1 --2- 172.21.15.16:0/252051825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe46c102dd0 0x7fe46c1031b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009cd0 tx=0x7fe45c02a8d0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.288+0000 7fe471fca640 1 -- 172.21.15.16:0/252051825 shutdown_connections 2024-09-19T09:29:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.288+0000 7fe471fca640 1 --2- 172.21.15.16:0/252051825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46c103780 0x7fe46c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.288+0000 7fe471fca640 1 --2- 172.21.15.16:0/252051825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe46c108300 0x7fe46c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.288+0000 7fe471fca640 1 --2- 172.21.15.16:0/252051825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe46c102dd0 0x7fe46c1031b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.288+0000 7fe471fca640 1 -- 172.21.15.16:0/252051825 >> 172.21.15.16:0/252051825 conn(0x7fe46c0fc7d0 msgr2=0x7fe46c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.288+0000 7fe471fca640 1 -- 172.21.15.16:0/252051825 shutdown_connections 2024-09-19T09:29:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.288+0000 7fe471fca640 1 -- 172.21.15.16:0/252051825 wait complete. 2024-09-19T09:29:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.288+0000 7fe471fca640 1 Processor -- start 2024-09-19T09:29:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.288+0000 7fe471fca640 1 -- start start 2024-09-19T09:29:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe471fca640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46c102dd0 0x7fe46c1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe471fca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe46c103780 0x7fe46c1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe471fca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe46c108300 0x7fe46c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe471fca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1120b0 con 0x7fe46c108300 2024-09-19T09:29:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe471fca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c111f30 con 0x7fe46c103780 2024-09-19T09:29:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe471fca640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c112230 con 0x7fe46c102dd0 2024-09-19T09:29:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe46bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe46c108300 0x7fe46c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe46bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe46c108300 0x7fe46c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55052/0 (socket says 172.21.15.16:55052) 2024-09-19T09:29:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe46bfff640 1 -- 172.21.15.16:0/1992350475 learned_addr learned my addr 172.21.15.16:0/1992350475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe46b7fe640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46c102dd0 0x7fe46c1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe46affd640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe46c103780 0x7fe46c1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe46bfff640 1 -- 172.21.15.16:0/1992350475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46c102dd0 msgr2=0x7fe46c1a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe46bfff640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46c102dd0 0x7fe46c1a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe46bfff640 1 -- 172.21.15.16:0/1992350475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe46c103780 msgr2=0x7fe46c1a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe46bfff640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe46c103780 0x7fe46c1a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe46bfff640 1 -- 172.21.15.16:0/1992350475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46c19ad40 con 0x7fe46c108300 2024-09-19T09:29:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.289+0000 7fe46bfff640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe46c108300 0x7fe46c19a610 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe46400d750 tx=0x7fe46400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.290+0000 7fe468ff9640 1 -- 172.21.15.16:0/1992350475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464017070 con 0x7fe46c108300 2024-09-19T09:29:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.290+0000 7fe468ff9640 1 -- 172.21.15.16:0/1992350475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe464005500 con 0x7fe46c108300 2024-09-19T09:29:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.290+0000 7fe471fca640 1 -- 172.21.15.16:0/1992350475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46c19b030 con 0x7fe46c108300 2024-09-19T09:29:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.290+0000 7fe468ff9640 1 -- 172.21.15.16:0/1992350475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464004d90 con 0x7fe46c108300 2024-09-19T09:29:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.290+0000 7fe471fca640 1 -- 172.21.15.16:0/1992350475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46c1a7320 con 0x7fe46c108300 2024-09-19T09:29:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.293+0000 7fe468ff9640 1 -- 172.21.15.16:0/1992350475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe464004f30 con 0x7fe46c108300 2024-09-19T09:29:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.293+0000 7fe4743a3740 1 Processor -- start 2024-09-19T09:29:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.293+0000 7fe4743a3740 1 -- start start 2024-09-19T09:29:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.293+0000 7fe4743a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 0x5625059fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.293+0000 7fe4743a3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fd320 0x562505a076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.293+0000 7fe4743a3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562505a07be0 0x562505a09fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.293+0000 7fe4743a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562505961200 con 0x5625059fca00 2024-09-19T09:29:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.293+0000 7fe4743a3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562505907dc0 con 0x5625059fd320 2024-09-19T09:29:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.293+0000 7fe4743a3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56250595d1c0 con 0x562505a07be0 2024-09-19T09:29:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.293+0000 7fe468ff9640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe4540a3240 0x7fe4540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.293+0000 7fe46affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 0x5625059fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.293+0000 7fe46affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 0x5625059fcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55056/0 (socket says 172.21.15.16:55056) 2024-09-19T09:29:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.293+0000 7fe46affd640 1 -- 172.21.15.16:0/856053981 learned_addr learned my addr 172.21.15.16:0/856053981 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe46bfff640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe4540a3240 0x7fe4540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe46b7fe640 1 --2- 172.21.15.16:0/856053981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fd320 0x562505a076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe468ff9640 1 -- 172.21.15.16:0/1992350475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe4640cf1d0 con 0x7fe46c108300 2024-09-19T09:29:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe46affd640 1 --2- 172.21.15.16:0/856053981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562505a07be0 0x562505a09fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe46affd640 1 -- 172.21.15.16:0/856053981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562505a07be0 msgr2=0x562505a09fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe46affd640 1 --2- 172.21.15.16:0/856053981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562505a07be0 0x562505a09fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe46affd640 1 -- 172.21.15.16:0/856053981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fd320 msgr2=0x562505a076a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe46affd640 1 --2- 172.21.15.16:0/856053981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fd320 0x562505a076a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe46affd640 1 -- 172.21.15.16:0/856053981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625056d2620 con 0x5625059fca00 2024-09-19T09:29:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe46affd640 1 --2- 172.21.15.16:0/856053981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 0x5625059fcde0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe460004650 tx=0x7fe4600041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69209c6a93796a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe46bfff640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe4540a3240 0x7fe4540a5700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe464097d00 tx=0x7fe464052040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe451ffb640 1 -- 172.21.15.16:0/856053981 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460005020 con 0x5625059fca00 2024-09-19T09:29:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe451ffb640 1 -- 172.21.15.16:0/856053981 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4600056a0 con 0x5625059fca00 2024-09-19T09:29:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.294+0000 7fe451ffb640 1 -- 172.21.15.16:0/856053981 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460010660 con 0x5625059fca00 2024-09-19T09:29:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.295+0000 7fe4743a3740 1 -- 172.21.15.16:0/856053981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 msgr2=0x5625059fcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.295+0000 7fe4743a3740 1 --2- 172.21.15.16:0/856053981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 0x5625059fcde0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe460004650 tx=0x7fe4600041d0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.295+0000 7fe4743a3740 1 -- 172.21.15.16:0/856053981 shutdown_connections 2024-09-19T09:29:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.295+0000 7fe4743a3740 1 --2- 172.21.15.16:0/856053981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562505a07be0 0x562505a09fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.295+0000 7fe4743a3740 1 --2- 172.21.15.16:0/856053981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fd320 0x562505a076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.295+0000 7fe4743a3740 1 --2- 172.21.15.16:0/856053981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 0x5625059fcde0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.295+0000 7fe4743a3740 1 -- 172.21.15.16:0/856053981 >> 172.21.15.16:0/856053981 conn(0x5625059f8f90 msgr2=0x5625059f9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.295+0000 7fe4743a3740 1 -- 172.21.15.16:0/856053981 shutdown_connections 2024-09-19T09:29:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.295+0000 7fe4743a3740 1 -- 172.21.15.16:0/856053981 wait complete. 2024-09-19T09:29:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.295+0000 7fe4743a3740 1 -- start start 2024-09-19T09:29:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe4743a3740 1 Processor -- start 2024-09-19T09:29:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe4743a3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fca00 0x562505a07460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe4743a3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625059fd320 0x562505a04710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe4743a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562505a07be0 0x562505a04c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe4743a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562505a0a7b0 con 0x562505a07be0 2024-09-19T09:29:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe4743a3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5625058fbc50 con 0x5625059fca00 2024-09-19T09:29:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe4743a3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562505a0a930 con 0x5625059fd320 2024-09-19T09:29:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe4743a3740 10 client.0 _ll_register_callbacks cb 0x56250590dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe46affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562505a07be0 0x562505a04c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe46affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562505a07be0 0x562505a04c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55060/0 (socket says 172.21.15.16:55060) 2024-09-19T09:29:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe46affd640 1 -- 172.21.15.16:0/3451189861 learned_addr learned my addr 172.21.15.16:0/3451189861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe46affd640 1 --2- 172.21.15.16:0/3451189861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fca00 0x562505a07460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe46b7fe640 1 --2- 172.21.15.16:0/3451189861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625059fd320 0x562505a04710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe46affd640 1 -- 172.21.15.16:0/3451189861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625059fd320 msgr2=0x562505a04710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe46affd640 1 --2- 172.21.15.16:0/3451189861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625059fd320 0x562505a04710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe46affd640 1 -- 172.21.15.16:0/3451189861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fca00 msgr2=0x562505a07460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe46affd640 1 --2- 172.21.15.16:0/3451189861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fca00 0x562505a07460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.296+0000 7fe46affd640 1 -- 172.21.15.16:0/3451189861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562505a05430 con 0x562505a07be0 2024-09-19T09:29:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.297+0000 7fe46affd640 1 --2- 172.21.15.16:0/3451189861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562505a07be0 0x562505a04c50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe460004e40 tx=0x7fe460004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beeca382c8161de6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.297+0000 7fe44affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.297+0000 7fe44affd640 1 -- 172.21.15.16:0/3451189861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460005210 con 0x562505a07be0 2024-09-19T09:29:48.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.297+0000 7fe44affd640 1 -- 172.21.15.16:0/3451189861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe4600053b0 con 0x562505a07be0 2024-09-19T09:29:48.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.297+0000 7fe4743a3740 1 -- 172.21.15.16:0/3451189861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4600056a0 con 0x562505a07be0 2024-09-19T09:29:48.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.297+0000 7fe44affd640 1 -- 172.21.15.16:0/3451189861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe460010460 con 0x562505a07be0 2024-09-19T09:29:48.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.297+0000 7fe4743a3740 10 client.16113 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:48.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.297+0000 7fe4743a3740 1 -- 172.21.15.16:0/3451189861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562505a16280 con 0x562505a07be0 2024-09-19T09:29:48.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.297+0000 7fe44affd640 1 -- 172.21.15.16:0/3451189861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4600056a0 con 0x562505a07be0 2024-09-19T09:29:48.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.297+0000 7fe44affd640 1 -- 172.21.15.16:0/3451189861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe46001f410 con 0x562505a07be0 2024-09-19T09:29:48.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.297+0000 7fe4743a3740 10 client.16113 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.297+0000 7fe4743a3740 10 client.16113 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:29:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.298+0000 7fe4743a3740 20 client.16113 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.298+0000 7fe4743a3740 1 --2- 172.21.15.16:0/3451189861 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562505a175d0 0x562505a199c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.298+0000 7fe4743a3740 4 client.16113 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.298+0000 7fe4743a3740 1 -- 172.21.15.16:0/3451189861 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56250596e770 con 0x562505a175d0 2024-09-19T09:29:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.298+0000 7fe46b7fe640 1 --2- 172.21.15.16:0/3451189861 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562505a175d0 0x562505a199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.298+0000 7fe44affd640 10 client.16113 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.298+0000 7fe46b7fe640 1 --2- 172.21.15.16:0/3451189861 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562505a175d0 0x562505a199c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e923f0aac8a41c9 server_cookie=5cfc908a1efa0d84 in_seq=0 out_seq=0 2024-09-19T09:29:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe44affd640 1 -- 172.21.15.16:0/3451189861 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56250596e770 con 0x562505a175d0 2024-09-19T09:29:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe44affd640 10 client.16113 handle_command_reply: tid=0 2024-09-19T09:29:48.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe4743a3740 1 client.16113 shutdown 2024-09-19T09:29:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe4743a3740 1 -- 172.21.15.16:0/3451189861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562505a07be0 msgr2=0x562505a04c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe4743a3740 1 --2- 172.21.15.16:0/3451189861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562505a07be0 0x562505a04c50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe460004e40 tx=0x7fe460004d40 comp rx=0 tx=0).stop 2024-09-19T09:29:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe4743a3740 1 -- 172.21.15.16:0/3451189861 shutdown_connections 2024-09-19T09:29:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe4743a3740 1 --2- 172.21.15.16:0/3451189861 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562505a175d0 0x562505a199c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe4743a3740 1 --2- 172.21.15.16:0/3451189861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625059fd320 0x562505a04710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe4743a3740 1 --2- 172.21.15.16:0/3451189861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562505a07be0 0x562505a04c50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe44affd640 0 client.16113 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe4743a3740 1 --2- 172.21.15.16:0/3451189861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fca00 0x562505a07460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe4743a3740 1 -- 172.21.15.16:0/3451189861 >> 172.21.15.16:0/3451189861 conn(0x5625059f8f90 msgr2=0x5625059f81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe4743a3740 1 -- 172.21.15.16:0/3451189861 shutdown_connections 2024-09-19T09:29:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe4743a3740 1 -- 172.21.15.16:0/3451189861 wait complete. 2024-09-19T09:29:48.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.299+0000 7fe4743a3740 20 client.16113 trim_cache size 0 max 16384 2024-09-19T09:29:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.316+0000 7fe4743a3740 1 Processor -- start 2024-09-19T09:29:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.316+0000 7fe4743a3740 1 -- start start 2024-09-19T09:29:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe4743a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 0x562505a13a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe4743a3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fd320 0x5625059ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe4743a3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562505a07be0 0x562505a1f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe4743a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1af8a0 con 0x5625059fca00 2024-09-19T09:29:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe4743a3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1b13e0 con 0x5625059fd320 2024-09-19T09:29:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe4743a3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1aff70 con 0x562505a07be0 2024-09-19T09:29:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe46affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 0x562505a13a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe46affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 0x562505a13a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55074/0 (socket says 172.21.15.16:55074) 2024-09-19T09:29:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe46affd640 1 -- 172.21.15.16:0/1310654308 learned_addr learned my addr 172.21.15.16:0/1310654308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe46b7fe640 1 --2- 172.21.15.16:0/1310654308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fd320 0x5625059ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe46affd640 1 -- 172.21.15.16:0/1310654308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562505a07be0 msgr2=0x562505a1f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe46affd640 1 --2- 172.21.15.16:0/1310654308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562505a07be0 0x562505a1f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe46affd640 1 -- 172.21.15.16:0/1310654308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fd320 msgr2=0x5625059ffc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe46affd640 1 --2- 172.21.15.16:0/1310654308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fd320 0x5625059ffc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe46affd640 1 -- 172.21.15.16:0/1310654308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625059f8dc0 con 0x5625059fca00 2024-09-19T09:29:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe46affd640 1 --2- 172.21.15.16:0/1310654308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 0x562505a13a10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe46000b9d0 tx=0x7fe460003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef18b4a3c197a926 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe4517fa640 1 -- 172.21.15.16:0/1310654308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460004940 con 0x5625059fca00 2024-09-19T09:29:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe4517fa640 1 -- 172.21.15.16:0/1310654308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe460025020 con 0x5625059fca00 2024-09-19T09:29:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.317+0000 7fe4517fa640 1 -- 172.21.15.16:0/1310654308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460005210 con 0x5625059fca00 2024-09-19T09:29:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.318+0000 7fe4743a3740 1 -- 172.21.15.16:0/1310654308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 msgr2=0x562505a13a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:48.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.318+0000 7fe4743a3740 1 --2- 172.21.15.16:0/1310654308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 0x562505a13a10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe46000b9d0 tx=0x7fe460003d00 comp rx=0 tx=0).stop 2024-09-19T09:29:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.318+0000 7fe4743a3740 1 -- 172.21.15.16:0/1310654308 shutdown_connections 2024-09-19T09:29:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.318+0000 7fe4743a3740 1 --2- 172.21.15.16:0/1310654308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562505a07be0 0x562505a1f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.318+0000 7fe4743a3740 1 --2- 172.21.15.16:0/1310654308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625059fd320 0x5625059ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.318+0000 7fe4743a3740 1 --2- 172.21.15.16:0/1310654308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fca00 0x562505a13a10 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.318+0000 7fe4743a3740 1 -- 172.21.15.16:0/1310654308 >> 172.21.15.16:0/1310654308 conn(0x5625059f8f90 msgr2=0x5625059ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.318+0000 7fe4743a3740 1 -- 172.21.15.16:0/1310654308 shutdown_connections 2024-09-19T09:29:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.318+0000 7fe4743a3740 1 -- 172.21.15.16:0/1310654308 wait complete. 2024-09-19T09:29:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.318+0000 7fe4743a3740 1 -- start start 2024-09-19T09:29:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.319+0000 7fe4743a3740 1 Processor -- start 2024-09-19T09:29:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.319+0000 7fe4743a3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625059fca00 0x562505a0b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.319+0000 7fe4743a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fd320 0x562505a0b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.319+0000 7fe4743a3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562505a07be0 0x562505a7a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.319+0000 7fe4743a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1b0d10 con 0x5625059fd320 2024-09-19T09:29:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.319+0000 7fe4743a3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1b0640 con 0x562505a07be0 2024-09-19T09:29:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.319+0000 7fe4743a3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1ad860 con 0x5625059fca00 2024-09-19T09:29:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.319+0000 7fe4743a3740 10 client.0 _ll_register_callbacks cb 0x7fe460003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.319+0000 7fe46b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fd320 0x562505a0b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.319+0000 7fe46b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fd320 0x562505a0b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55088/0 (socket says 172.21.15.16:55088) 2024-09-19T09:29:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.319+0000 7fe46b7fe640 1 -- 172.21.15.16:0/368159597 learned_addr learned my addr 172.21.15.16:0/368159597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.319+0000 7fe46affd640 1 --2- 172.21.15.16:0/368159597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562505a07be0 0x562505a7a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.319+0000 7fe46affd640 1 --2- 172.21.15.16:0/368159597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625059fca00 0x562505a0b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.320+0000 7fe46b7fe640 1 -- 172.21.15.16:0/368159597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625059fca00 msgr2=0x562505a0b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.320+0000 7fe46b7fe640 1 --2- 172.21.15.16:0/368159597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625059fca00 0x562505a0b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.320+0000 7fe46b7fe640 1 -- 172.21.15.16:0/368159597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562505a07be0 msgr2=0x562505a7a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.320+0000 7fe46b7fe640 1 --2- 172.21.15.16:0/368159597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562505a07be0 0x562505a7a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.320+0000 7fe46b7fe640 1 -- 172.21.15.16:0/368159597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562505a7a700 con 0x5625059fd320 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.320+0000 7fe46b7fe640 1 --2- 172.21.15.16:0/368159597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fd320 0x562505a0b800 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe45c02adf0 tx=0x7fe45c031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b2501386866b26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.320+0000 7fe44affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.320+0000 7fe44affd640 1 -- 172.21.15.16:0/368159597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c056060 con 0x5625059fd320 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.320+0000 7fe44affd640 1 -- 172.21.15.16:0/368159597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe45c031620 con 0x5625059fd320 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.320+0000 7fe4743a3740 1 -- 172.21.15.16:0/368159597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe460025020 con 0x5625059fd320 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.320+0000 7fe44affd640 1 -- 172.21.15.16:0/368159597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe45c031820 con 0x5625059fd320 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.320+0000 7fe4743a3740 10 client.16119 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.320+0000 7fe4743a3740 1 -- 172.21.15.16:0/368159597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562505a81ad0 con 0x5625059fd320 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.321+0000 7fe44affd640 1 -- 172.21.15.16:0/368159597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe460025020 con 0x5625059fd320 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.321+0000 7fe44affd640 1 -- 172.21.15.16:0/368159597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe45c05b070 con 0x5625059fd320 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.321+0000 7fe4743a3740 10 client.16119 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.321+0000 7fe4743a3740 10 client.16119 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.321+0000 7fe4743a3740 20 client.16119 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.321+0000 7fe4743a3740 1 --2- 172.21.15.16:0/368159597 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562505a821d0 0x562505a845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.321+0000 7fe4743a3740 4 client.16119 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.321+0000 7fe4743a3740 1 -- 172.21.15.16:0/368159597 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x562505968c80 con 0x562505a821d0 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.321+0000 7fe46affd640 1 --2- 172.21.15.16:0/368159597 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562505a821d0 0x562505a845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.321+0000 7fe46affd640 1 --2- 172.21.15.16:0/368159597 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562505a821d0 0x562505a845f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=817829c8ea1ffdcd server_cookie=cde748235ec86875 in_seq=0 out_seq=0 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.321+0000 7fe44affd640 10 client.16119 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe44affd640 1 -- 172.21.15.16:0/368159597 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x562505968c80 con 0x562505a821d0 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe44affd640 10 client.16119 handle_command_reply: tid=0 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe4743a3740 1 client.16119 shutdown 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe4743a3740 1 -- 172.21.15.16:0/368159597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fd320 msgr2=0x562505a0b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe4743a3740 1 --2- 172.21.15.16:0/368159597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fd320 0x562505a0b800 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe45c02adf0 tx=0x7fe45c031cc0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe4743a3740 1 -- 172.21.15.16:0/368159597 shutdown_connections 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe4743a3740 1 --2- 172.21.15.16:0/368159597 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x562505a821d0 0x562505a845f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe4743a3740 1 --2- 172.21.15.16:0/368159597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562505a07be0 0x562505a7a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe4743a3740 1 --2- 172.21.15.16:0/368159597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625059fd320 0x562505a0b800 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe4743a3740 1 --2- 172.21.15.16:0/368159597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5625059fca00 0x562505a0b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe4743a3740 1 -- 172.21.15.16:0/368159597 >> 172.21.15.16:0/368159597 conn(0x5625059f8f90 msgr2=0x5625059ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe44affd640 0 client.16119 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe4743a3740 1 -- 172.21.15.16:0/368159597 shutdown_connections 2024-09-19T09:29:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe4743a3740 1 -- 172.21.15.16:0/368159597 wait complete. 2024-09-19T09:29:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.322+0000 7fe4743a3740 20 client.16119 trim_cache size 0 max 16384 2024-09-19T09:29:48.335 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:29:48.335 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-19T09:29:48.335 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24500, 2024-09-19T09:29:48.335 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 12, 2024-09-19T09:29:48.336 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": {} 2024-09-19T09:29:48.336 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:29:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.323+0000 7fe471fca640 1 -- 172.21.15.16:0/1992350475 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe4540a3240 msgr2=0x7fe4540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.323+0000 7fe471fca640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe4540a3240 0x7fe4540a5700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe464097d00 tx=0x7fe464052040 comp rx=0 tx=0).stop 2024-09-19T09:29:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.323+0000 7fe471fca640 1 -- 172.21.15.16:0/1992350475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe46c108300 msgr2=0x7fe46c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.323+0000 7fe471fca640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe46c108300 0x7fe46c19a610 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe46400d750 tx=0x7fe46400dc10 comp rx=0 tx=0).stop 2024-09-19T09:29:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.323+0000 7fe471fca640 1 -- 172.21.15.16:0/1992350475 shutdown_connections 2024-09-19T09:29:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.323+0000 7fe471fca640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe46c108300 0x7fe46c19a610 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.323+0000 7fe471fca640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe46c103780 0x7fe46c1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.323+0000 7fe471fca640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe4540a3240 0x7fe4540a5700 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.323+0000 7fe471fca640 1 --2- 172.21.15.16:0/1992350475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46c102dd0 0x7fe46c1a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.323+0000 7fe471fca640 1 -- 172.21.15.16:0/1992350475 >> 172.21.15.16:0/1992350475 conn(0x7fe46c0fc7d0 msgr2=0x7fe46c100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.323+0000 7fe471fca640 1 -- 172.21.15.16:0/1992350475 shutdown_connections 2024-09-19T09:29:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:48.323+0000 7fe471fca640 1 -- 172.21.15.16:0/1992350475 wait complete. 2024-09-19T09:29:48.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:48 smithi016 ceph-mon[30979]: pgmap v404: 97 pgs: 97 active+clean; 69 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 162 MiB/s rd, 1.7 MiB/s wr, 613 op/s 2024-09-19T09:29:48.879 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:48 smithi142 ceph-mon[35032]: pgmap v404: 97 pgs: 97 active+clean; 69 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 162 MiB/s rd, 1.7 MiB/s wr, 613 op/s 2024-09-19T09:29:48.886 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:48 smithi097 ceph-mon[34864]: pgmap v404: 97 pgs: 97 active+clean; 69 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 162 MiB/s rd, 1.7 MiB/s wr, 613 op/s 2024-09-19T09:29:49.109 INFO:tasks.quiescer:done joining Quiescers 2024-09-19T09:29:49.110 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-19T09:29:49.120 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T09:29:49.160 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T09:29:49.170 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T09:29:49.192 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.653+0000 7fed1b81d640 1 -- 172.21.15.16:0/1363468279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed14103780 msgr2=0x7fed14107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.653+0000 7fed1b81d640 1 --2- 172.21.15.16:0/1363468279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed14103780 0x7fed14107bc0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fed08009930 tx=0x7fed0802a420 comp rx=0 tx=0).stop 2024-09-19T09:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.653+0000 7fed1b81d640 1 -- 172.21.15.16:0/1363468279 shutdown_connections 2024-09-19T09:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.653+0000 7fed1b81d640 1 --2- 172.21.15.16:0/1363468279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed14108300 0x7fed1410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.653+0000 7fed1b81d640 1 --2- 172.21.15.16:0/1363468279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed14103780 0x7fed14107bc0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.653+0000 7fed1b81d640 1 --2- 172.21.15.16:0/1363468279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed14102dd0 0x7fed141031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.653+0000 7fed1b81d640 1 -- 172.21.15.16:0/1363468279 >> 172.21.15.16:0/1363468279 conn(0x7fed140fc7d0 msgr2=0x7fed140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.653+0000 7fed1b81d640 1 -- 172.21.15.16:0/1363468279 shutdown_connections 2024-09-19T09:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.653+0000 7fed1b81d640 1 -- 172.21.15.16:0/1363468279 wait complete. 2024-09-19T09:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.653+0000 7fed1b81d640 1 Processor -- start 2024-09-19T09:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.653+0000 7fed1b81d640 1 -- start start 2024-09-19T09:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed1b81d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed14102dd0 0x7fed141a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed1b81d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed14103780 0x7fed141a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed1b81d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed14108300 0x7fed1419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed1b81d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed14112070 con 0x7fed14108300 2024-09-19T09:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed1b81d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fed14111ef0 con 0x7fed14103780 2024-09-19T09:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed1b81d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed141121f0 con 0x7fed14102dd0 2024-09-19T09:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed19dba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed14108300 0x7fed1419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed19dba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed14108300 0x7fed1419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55114/0 (socket says 172.21.15.16:55114) 2024-09-19T09:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed19dba640 1 -- 172.21.15.16:0/4239739189 learned_addr learned my addr 172.21.15.16:0/4239739189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed195b9640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed14102dd0 0x7fed141a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed18db8640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed14103780 0x7fed141a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed19dba640 1 -- 172.21.15.16:0/4239739189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed14102dd0 msgr2=0x7fed141a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed19dba640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed14102dd0 0x7fed141a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed19dba640 1 -- 172.21.15.16:0/4239739189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed14103780 msgr2=0x7fed141a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed19dba640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed14103780 0x7fed141a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed19dba640 1 -- 172.21.15.16:0/4239739189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1419aeb0 con 0x7fed14108300 2024-09-19T09:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed19dba640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed14108300 0x7fed1419a640 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00d750 tx=0x7fed0c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.654+0000 7fed127fc640 1 -- 172.21.15.16:0/4239739189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c017070 con 0x7fed14108300 2024-09-19T09:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.655+0000 7fed127fc640 1 -- 172.21.15.16:0/4239739189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed0c005500 con 0x7fed14108300 2024-09-19T09:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.655+0000 7fed1b81d640 1 -- 172.21.15.16:0/4239739189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1419b1a0 con 0x7fed14108300 2024-09-19T09:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.655+0000 7fed1b81d640 1 -- 172.21.15.16:0/4239739189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed141a74f0 con 0x7fed14108300 2024-09-19T09:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.655+0000 7fed127fc640 1 -- 172.21.15.16:0/4239739189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c004d90 con 0x7fed14108300 2024-09-19T09:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.658+0000 7fed1dbf6740 1 Processor -- start 2024-09-19T09:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.658+0000 7fed127fc640 1 -- 172.21.15.16:0/4239739189 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fed0c005020 con 0x7fed14108300 2024-09-19T09:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.658+0000 7fed1dbf6740 1 -- start start 2024-09-19T09:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.658+0000 7fed1dbf6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a584b40 0x55d34a583090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.658+0000 7fed1dbf6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a5835d0 0x55d34a583a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.658+0000 7fed1dbf6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 0x55d34a590760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.658+0000 7fed1dbf6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d34a4e92f0 con 0x55d34a58e370 2024-09-19T09:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.658+0000 7fed1dbf6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d34a490cf0 con 0x55d34a5835d0 2024-09-19T09:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.658+0000 7fed1dbf6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d34a4e52b0 con 0x55d34a584b40 2024-09-19T09:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.658+0000 7fed18db8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 0x55d34a590760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.658+0000 7fed18db8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 0x55d34a590760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55120/0 (socket says 172.21.15.16:55120) 2024-09-19T09:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.658+0000 7fed18db8640 1 -- 172.21.15.16:0/3752978651 learned_addr learned my addr 172.21.15.16:0/3752978651 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fed18db8640 1 --2- 172.21.15.16:0/3752978651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a584b40 0x55d34a583090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fed195b9640 1 --2- 172.21.15.16:0/3752978651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a5835d0 0x55d34a583a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fed127fc640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fecfc0a3240 0x7fecfc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fed18db8640 1 -- 172.21.15.16:0/3752978651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a584b40 msgr2=0x55d34a583090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fed18db8640 1 --2- 172.21.15.16:0/3752978651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a584b40 0x55d34a583090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fed18db8640 1 -- 172.21.15.16:0/3752978651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a5835d0 msgr2=0x55d34a583a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fed18db8640 1 --2- 172.21.15.16:0/3752978651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a5835d0 0x55d34a583a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fed18db8640 1 -- 172.21.15.16:0/3752978651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34a4753a0 con 0x55d34a58e370 2024-09-19T09:29:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fed19dba640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fecfc0a3240 0x7fecfc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fed127fc640 1 -- 172.21.15.16:0/4239739189 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fed0c0cf1b0 con 0x7fed14108300 2024-09-19T09:29:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fed18db8640 1 --2- 172.21.15.16:0/3752978651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 0x55d34a590760 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fed140697d0 tx=0x7fed08005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ef8d7484a444e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fecfb7fe640 1 -- 172.21.15.16:0/3752978651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08035070 con 0x55d34a58e370 2024-09-19T09:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fecfb7fe640 1 -- 172.21.15.16:0/3752978651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed080035f0 con 0x55d34a58e370 2024-09-19T09:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.659+0000 7fecfb7fe640 1 -- 172.21.15.16:0/3752978651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0803d6b0 con 0x55d34a58e370 2024-09-19T09:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.660+0000 7fed1dbf6740 1 -- 172.21.15.16:0/3752978651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 msgr2=0x55d34a590760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.660+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/3752978651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 0x55d34a590760 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fed140697d0 tx=0x7fed08005780 comp rx=0 tx=0).stop 2024-09-19T09:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.660+0000 7fed19dba640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fecfc0a3240 0x7fecfc0a5700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fed0c098250 tx=0x7fed0c0662c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.660+0000 7fed1dbf6740 1 -- 172.21.15.16:0/3752978651 shutdown_connections 2024-09-19T09:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.660+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/3752978651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 0x55d34a590760 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.660+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/3752978651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a5835d0 0x55d34a583a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.660+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/3752978651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a584b40 0x55d34a583090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.660+0000 7fed1dbf6740 1 -- 172.21.15.16:0/3752978651 >> 172.21.15.16:0/3752978651 conn(0x55d34a54c310 msgr2=0x55d34a57fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.660+0000 7fed1dbf6740 1 -- 172.21.15.16:0/3752978651 shutdown_connections 2024-09-19T09:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.660+0000 7fed1dbf6740 1 -- 172.21.15.16:0/3752978651 wait complete. 2024-09-19T09:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed1dbf6740 1 -- start start 2024-09-19T09:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed1dbf6740 1 Processor -- start 2024-09-19T09:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed1dbf6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a5835d0 0x55d34a58dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed1dbf6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a584b40 0x55d34a58afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed1dbf6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a58e370 0x55d34a58b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed1dbf6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d34a591010 con 0x55d34a5835d0 2024-09-19T09:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed1dbf6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d34a4a4060 con 0x55d34a584b40 2024-09-19T09:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed1dbf6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d34a591190 con 0x55d34a58e370 2024-09-19T09:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed1dbf6740 10 client.0 _ll_register_callbacks cb 0x55d34a1f78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed18db8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a5835d0 0x55d34a58dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed18db8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a5835d0 0x55d34a58dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55128/0 (socket says 172.21.15.16:55128) 2024-09-19T09:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed18db8640 1 -- 172.21.15.16:0/1217435809 learned_addr learned my addr 172.21.15.16:0/1217435809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed195b9640 1 --2- 172.21.15.16:0/1217435809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a584b40 0x55d34a58afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed18db8640 1 -- 172.21.15.16:0/1217435809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a58e370 msgr2=0x55d34a58b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed18db8640 1 --2- 172.21.15.16:0/1217435809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a58e370 0x55d34a58b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.661+0000 7fed18db8640 1 -- 172.21.15.16:0/1217435809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a584b40 msgr2=0x55d34a58afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.662+0000 7fed18db8640 1 --2- 172.21.15.16:0/1217435809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a584b40 0x55d34a58afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.662+0000 7fed18db8640 1 -- 172.21.15.16:0/1217435809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34a25a3c0 con 0x55d34a5835d0 2024-09-19T09:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.662+0000 7fed195b9640 1 --2- 172.21.15.16:0/1217435809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a584b40 0x55d34a58afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.662+0000 7fed18db8640 1 --2- 172.21.15.16:0/1217435809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a5835d0 0x55d34a58dd20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fed08003f70 tx=0x7fed08031ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6768117025030cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.662+0000 7fecf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.662+0000 7fecf8ff9640 1 -- 172.21.15.16:0/1217435809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0804f020 con 0x55d34a5835d0 2024-09-19T09:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.662+0000 7fecf8ff9640 1 -- 172.21.15.16:0/1217435809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fed08004410 con 0x55d34a5835d0 2024-09-19T09:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.662+0000 7fed1dbf6740 1 -- 172.21.15.16:0/1217435809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed080035f0 con 0x55d34a5835d0 2024-09-19T09:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.662+0000 7fecf8ff9640 1 -- 172.21.15.16:0/1217435809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed08035070 con 0x55d34a5835d0 2024-09-19T09:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.662+0000 7fed1dbf6740 10 client.16131 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.662+0000 7fed1dbf6740 1 -- 172.21.15.16:0/1217435809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d34a59cac0 con 0x55d34a5835d0 2024-09-19T09:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.662+0000 7fecf8ff9640 1 -- 172.21.15.16:0/1217435809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed080035f0 con 0x55d34a5835d0 2024-09-19T09:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.662+0000 7fecf8ff9640 1 -- 172.21.15.16:0/1217435809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fed0803d980 con 0x55d34a5835d0 2024-09-19T09:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.663+0000 7fed1dbf6740 10 client.16131 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.663+0000 7fed1dbf6740 10 client.16131 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T09:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.663+0000 7fed1dbf6740 20 client.16131 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.663+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/1217435809 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55d34a59de10 0x55d34a5a0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.663+0000 7fed1dbf6740 4 client.16131 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.663+0000 7fed1dbf6740 1 -- 172.21.15.16:0/1217435809 --> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d34a4f6860 con 0x55d34a59de10 2024-09-19T09:29:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.663+0000 7fed195b9640 1 --2- 172.21.15.16:0/1217435809 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55d34a59de10 0x55d34a5a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.663+0000 7fed195b9640 1 --2- 172.21.15.16:0/1217435809 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55d34a59de10 0x55d34a5a0200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68bf04500670b50 server_cookie=9a8a2f81624369ff in_seq=0 out_seq=0 2024-09-19T09:29:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.663+0000 7fecf8ff9640 10 client.16131 ms_handle_connect on v2:172.21.15.97:6836/2018284957 2024-09-19T09:29:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fecf8ff9640 1 -- 172.21.15.16:0/1217435809 <== mds.2 v2:172.21.15.97:6836/2018284957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d34a4f6860 con 0x55d34a59de10 2024-09-19T09:29:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fecf8ff9640 10 client.16131 handle_command_reply: tid=0 2024-09-19T09:29:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fed1dbf6740 1 client.16131 shutdown 2024-09-19T09:29:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fed1dbf6740 1 -- 172.21.15.16:0/1217435809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a5835d0 msgr2=0x55d34a58dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/1217435809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a5835d0 0x55d34a58dd20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fed08003f70 tx=0x7fed08031ab0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fed1dbf6740 1 -- 172.21.15.16:0/1217435809 shutdown_connections 2024-09-19T09:29:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/1217435809 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55d34a59de10 0x55d34a5a0200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/1217435809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a58e370 0x55d34a58b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/1217435809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a584b40 0x55d34a58afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fecf8ff9640 0 client.16131 ms_handle_reset on v2:172.21.15.97:6836/2018284957 2024-09-19T09:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/1217435809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a5835d0 0x55d34a58dd20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fed1dbf6740 1 -- 172.21.15.16:0/1217435809 >> 172.21.15.16:0/1217435809 conn(0x55d34a54c310 msgr2=0x55d34a585960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fed1dbf6740 1 -- 172.21.15.16:0/1217435809 shutdown_connections 2024-09-19T09:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.664+0000 7fed1dbf6740 1 -- 172.21.15.16:0/1217435809 wait complete. 2024-09-19T09:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.665+0000 7fed1dbf6740 20 client.16131 trim_cache size 0 max 16384 2024-09-19T09:29:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed1dbf6740 1 Processor -- start 2024-09-19T09:29:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed1dbf6740 1 -- start start 2024-09-19T09:29:49.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed1dbf6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a5835d0 0x55d34a586120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed1dbf6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a584b40 0x55d34a586660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed1dbf6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 0x55d34a5a3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed1dbf6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed141af7b0 con 0x55d34a58e370 2024-09-19T09:29:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed1dbf6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fed141b12f0 con 0x55d34a584b40 2024-09-19T09:29:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed1dbf6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed141afe80 con 0x55d34a5835d0 2024-09-19T09:29:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed18db8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 0x55d34a5a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed18db8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 0x55d34a5a3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55130/0 (socket says 172.21.15.16:55130) 2024-09-19T09:29:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed18db8640 1 -- 172.21.15.16:0/2528842398 learned_addr learned my addr 172.21.15.16:0/2528842398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed195b9640 1 --2- 172.21.15.16:0/2528842398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a584b40 0x55d34a586660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed18db8640 1 --2- 172.21.15.16:0/2528842398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a5835d0 0x55d34a586120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed18db8640 1 -- 172.21.15.16:0/2528842398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a5835d0 msgr2=0x55d34a586120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed18db8640 1 --2- 172.21.15.16:0/2528842398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a5835d0 0x55d34a586120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed18db8640 1 -- 172.21.15.16:0/2528842398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a584b40 msgr2=0x55d34a586660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed18db8640 1 --2- 172.21.15.16:0/2528842398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a584b40 0x55d34a586660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed18db8640 1 -- 172.21.15.16:0/2528842398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34a58b8e0 con 0x55d34a58e370 2024-09-19T09:29:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.678+0000 7fed195b9640 1 --2- 172.21.15.16:0/2528842398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a584b40 0x55d34a586660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.679+0000 7fed18db8640 1 --2- 172.21.15.16:0/2528842398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 0x55d34a5a3220 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fed0803ddd0 tx=0x7fed08005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7dca1586c5c78a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.679+0000 7fecfaffd640 1 -- 172.21.15.16:0/2528842398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0804f070 con 0x55d34a58e370 2024-09-19T09:29:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.679+0000 7fecfaffd640 1 -- 172.21.15.16:0/2528842398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed0804acd0 con 0x55d34a58e370 2024-09-19T09:29:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.679+0000 7fecfaffd640 1 -- 172.21.15.16:0/2528842398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0803d740 con 0x55d34a58e370 2024-09-19T09:29:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.680+0000 7fed1dbf6740 1 -- 172.21.15.16:0/2528842398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 msgr2=0x55d34a5a3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.680+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/2528842398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 0x55d34a5a3220 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fed0803ddd0 tx=0x7fed08005780 comp rx=0 tx=0).stop 2024-09-19T09:29:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.680+0000 7fed1dbf6740 1 -- 172.21.15.16:0/2528842398 shutdown_connections 2024-09-19T09:29:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.680+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/2528842398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a58e370 0x55d34a5a3220 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.680+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/2528842398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a584b40 0x55d34a586660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.680+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/2528842398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a5835d0 0x55d34a586120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.680+0000 7fed1dbf6740 1 -- 172.21.15.16:0/2528842398 >> 172.21.15.16:0/2528842398 conn(0x55d34a54c310 msgr2=0x55d34a58d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.680+0000 7fed1dbf6740 1 -- 172.21.15.16:0/2528842398 shutdown_connections 2024-09-19T09:29:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.680+0000 7fed1dbf6740 1 -- 172.21.15.16:0/2528842398 wait complete. 2024-09-19T09:29:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.680+0000 7fed1dbf6740 1 -- start start 2024-09-19T09:29:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.680+0000 7fed1dbf6740 1 Processor -- start 2024-09-19T09:29:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed1dbf6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a5835d0 0x55d34a5a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed1dbf6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a584b40 0x55d34a5a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed1dbf6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a58e370 0x55d34a5a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed1dbf6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed141b0c20 con 0x55d34a584b40 2024-09-19T09:29:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed1dbf6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fed141b0550 con 0x55d34a5835d0 2024-09-19T09:29:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed1dbf6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed141ad770 con 0x55d34a58e370 2024-09-19T09:29:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed1dbf6740 10 client.0 _ll_register_callbacks cb 0x7fed04009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed195b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a584b40 0x55d34a5a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed18db8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a5835d0 0x55d34a5a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed195b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a584b40 0x55d34a5a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55134/0 (socket says 172.21.15.16:55134) 2024-09-19T09:29:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed195b9640 1 -- 172.21.15.16:0/3669941172 learned_addr learned my addr 172.21.15.16:0/3669941172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed18db8640 1 --2- 172.21.15.16:0/3669941172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a58e370 0x55d34a5a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed195b9640 1 -- 172.21.15.16:0/3669941172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a58e370 msgr2=0x55d34a5a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed195b9640 1 --2- 172.21.15.16:0/3669941172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a58e370 0x55d34a5a0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed195b9640 1 -- 172.21.15.16:0/3669941172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a5835d0 msgr2=0x55d34a5a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed195b9640 1 --2- 172.21.15.16:0/3669941172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a5835d0 0x55d34a5a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed195b9640 1 -- 172.21.15.16:0/3669941172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34a5a1470 con 0x55d34a584b40 2024-09-19T09:29:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed18db8640 1 --2- 172.21.15.16:0/3669941172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a58e370 0x55d34a5a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fed195b9640 1 --2- 172.21.15.16:0/3669941172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a584b40 0x55d34a5a09f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x55d34a58a860 tx=0x7fed0401e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbba71252c08ae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fecf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.681+0000 7fecf8ff9640 1 -- 172.21.15.16:0/3669941172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04004d20 con 0x55d34a584b40 2024-09-19T09:29:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fecf8ff9640 1 -- 172.21.15.16:0/3669941172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fed040145f0 con 0x55d34a584b40 2024-09-19T09:29:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fed1dbf6740 1 -- 172.21.15.16:0/3669941172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed0804acd0 con 0x55d34a584b40 2024-09-19T09:29:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fecf8ff9640 1 -- 172.21.15.16:0/3669941172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed040147f0 con 0x55d34a584b40 2024-09-19T09:29:49.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fed1dbf6740 10 client.16137 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fed1dbf6740 1 -- 172.21.15.16:0/3669941172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d34a608490 con 0x55d34a584b40 2024-09-19T09:29:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fecf8ff9640 1 -- 172.21.15.16:0/3669941172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed0804acd0 con 0x55d34a584b40 2024-09-19T09:29:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fecf8ff9640 1 -- 172.21.15.16:0/3669941172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fed0402f050 con 0x55d34a584b40 2024-09-19T09:29:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fed1dbf6740 10 client.16137 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fed1dbf6740 10 client.16137 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T09:29:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fed1dbf6740 20 client.16137 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/3669941172 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55d34a608bc0 0x55d34a60afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fed1dbf6740 4 client.16137 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:29:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fed1dbf6740 1 -- 172.21.15.16:0/3669941172 --> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d34a4f0d70 con 0x55d34a608bc0 2024-09-19T09:29:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.682+0000 7fed18db8640 1 --2- 172.21.15.16:0/3669941172 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55d34a608bc0 0x55d34a60afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fed18db8640 1 --2- 172.21.15.16:0/3669941172 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55d34a608bc0 0x55d34a60afe0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=709332b45d80c165 server_cookie=e3e8099a1e00506d in_seq=0 out_seq=0 2024-09-19T09:29:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fecf8ff9640 10 client.16137 ms_handle_connect on v2:172.21.15.97:6836/2018284957 2024-09-19T09:29:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fecf8ff9640 1 -- 172.21.15.16:0/3669941172 <== mds.2 v2:172.21.15.97:6836/2018284957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d34a4f0d70 con 0x55d34a608bc0 2024-09-19T09:29:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fecf8ff9640 10 client.16137 handle_command_reply: tid=0 2024-09-19T09:29:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fed1dbf6740 1 client.16137 shutdown 2024-09-19T09:29:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fed1dbf6740 1 -- 172.21.15.16:0/3669941172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a584b40 msgr2=0x55d34a5a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/3669941172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a584b40 0x55d34a5a09f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x55d34a58a860 tx=0x7fed0401e880 comp rx=0 tx=0).stop 2024-09-19T09:29:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fed1dbf6740 1 -- 172.21.15.16:0/3669941172 shutdown_connections 2024-09-19T09:29:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/3669941172 >> [v2:172.21.15.97:6836/2018284957,v1:172.21.15.97:6837/2018284957] conn(0x55d34a608bc0 0x55d34a60afe0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/3669941172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34a58e370 0x55d34a5a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/3669941172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d34a584b40 0x55d34a5a09f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fed1dbf6740 1 --2- 172.21.15.16:0/3669941172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d34a5835d0 0x55d34a5a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fed1dbf6740 1 -- 172.21.15.16:0/3669941172 >> 172.21.15.16:0/3669941172 conn(0x55d34a54c310 msgr2=0x55d34a589770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fecf8ff9640 0 client.16137 ms_handle_reset on v2:172.21.15.97:6836/2018284957 2024-09-19T09:29:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fed1dbf6740 1 -- 172.21.15.16:0/3669941172 shutdown_connections 2024-09-19T09:29:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fed1dbf6740 1 -- 172.21.15.16:0/3669941172 wait complete. 2024-09-19T09:29:49.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.683+0000 7fed1dbf6740 20 client.16137 trim_cache size 0 max 16384 2024-09-19T09:29:49.695 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:29:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.684+0000 7fed1b81d640 1 -- 172.21.15.16:0/4239739189 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fecfc0a3240 msgr2=0x7fecfc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.684+0000 7fed1b81d640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fecfc0a3240 0x7fecfc0a5700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fed0c098250 tx=0x7fed0c0662c0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.684+0000 7fed1b81d640 1 -- 172.21.15.16:0/4239739189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed14108300 msgr2=0x7fed1419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.684+0000 7fed1b81d640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed14108300 0x7fed1419a640 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00d750 tx=0x7fed0c00dc10 comp rx=0 tx=0).stop 2024-09-19T09:29:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.684+0000 7fed1b81d640 1 -- 172.21.15.16:0/4239739189 shutdown_connections 2024-09-19T09:29:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.684+0000 7fed1b81d640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed14108300 0x7fed1419a640 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.684+0000 7fed1b81d640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed14103780 0x7fed141a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.684+0000 7fed1b81d640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fecfc0a3240 0x7fecfc0a5700 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.684+0000 7fed1b81d640 1 --2- 172.21.15.16:0/4239739189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed14102dd0 0x7fed141a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.684+0000 7fed1b81d640 1 -- 172.21.15.16:0/4239739189 >> 172.21.15.16:0/4239739189 conn(0x7fed140fc7d0 msgr2=0x7fed14100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.684+0000 7fed1b81d640 1 -- 172.21.15.16:0/4239739189 shutdown_connections 2024-09-19T09:29:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:49.684+0000 7fed1b81d640 1 -- 172.21.15.16:0/4239739189 wait complete. 2024-09-19T09:29:50.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T09:29:50.693 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:51.119 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:51 smithi016 ceph-mon[30979]: pgmap v405: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 113 MiB/s rd, 1.2 MiB/s wr, 452 op/s 2024-09-19T09:29:51.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.204+0000 7fe994af8640 1 -- 172.21.15.16:0/2202020844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990101de0 msgr2=0x7fe9901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:51.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.204+0000 7fe994af8640 1 --2- 172.21.15.16:0/2202020844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990101de0 0x7fe9901021c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe980009900 tx=0x7fe98002a410 comp rx=0 tx=0).stop 2024-09-19T09:29:51.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.204+0000 7fe994af8640 1 -- 172.21.15.16:0/2202020844 shutdown_connections 2024-09-19T09:29:51.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.204+0000 7fe994af8640 1 --2- 172.21.15.16:0/2202020844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe990106f10 0x7fe99010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.204+0000 7fe994af8640 1 --2- 172.21.15.16:0/2202020844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe990102790 0x7fe9901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.204+0000 7fe994af8640 1 --2- 172.21.15.16:0/2202020844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990101de0 0x7fe9901021c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.204+0000 7fe994af8640 1 -- 172.21.15.16:0/2202020844 >> 172.21.15.16:0/2202020844 conn(0x7fe9900fd8c0 msgr2=0x7fe9900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.204+0000 7fe994af8640 1 -- 172.21.15.16:0/2202020844 shutdown_connections 2024-09-19T09:29:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.204+0000 7fe994af8640 1 -- 172.21.15.16:0/2202020844 wait complete. 2024-09-19T09:29:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe994af8640 1 Processor -- start 2024-09-19T09:29:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe994af8640 1 -- start start 2024-09-19T09:29:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe994af8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe990101de0 0x7fe990198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe994af8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe990102790 0x7fe9901989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe994af8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990106f10 0x7fe99019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe994af8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe990111f90 con 0x7fe990106f10 2024-09-19T09:29:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe994af8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe990111e10 con 0x7fe990101de0 2024-09-19T09:29:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe994af8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe990112110 con 0x7fe990102790 2024-09-19T09:29:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe98ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990106f10 0x7fe99019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe98ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990106f10 0x7fe99019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55162/0 (socket says 172.21.15.16:55162) 2024-09-19T09:29:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe98ed9d640 1 -- 172.21.15.16:0/693963468 learned_addr learned my addr 172.21.15.16:0/693963468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe98e59c640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe990101de0 0x7fe990198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe987fff640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe990102790 0x7fe9901989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe98ed9d640 1 -- 172.21.15.16:0/693963468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe990102790 msgr2=0x7fe9901989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe98ed9d640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe990102790 0x7fe9901989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe98ed9d640 1 -- 172.21.15.16:0/693963468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe990101de0 msgr2=0x7fe990198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe98ed9d640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe990101de0 0x7fe990198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.205+0000 7fe98ed9d640 1 -- 172.21.15.16:0/693963468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99019d440 con 0x7fe990106f10 2024-09-19T09:29:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.206+0000 7fe98ed9d640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990106f10 0x7fe99019cd60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe98800b970 tx=0x7fe98800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.206+0000 7fe9877fe640 1 -- 172.21.15.16:0/693963468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988019070 con 0x7fe990106f10 2024-09-19T09:29:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.206+0000 7fe994af8640 1 -- 172.21.15.16:0/693963468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99019d730 con 0x7fe990106f10 2024-09-19T09:29:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.206+0000 7fe9877fe640 1 -- 172.21.15.16:0/693963468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98800cb40 con 0x7fe990106f10 2024-09-19T09:29:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.206+0000 7fe9877fe640 1 -- 172.21.15.16:0/693963468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988007650 con 0x7fe990106f10 2024-09-19T09:29:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.206+0000 7fe994af8640 1 -- 172.21.15.16:0/693963468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9901a5010 con 0x7fe990106f10 2024-09-19T09:29:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.208+0000 7fe9877fe640 1 -- 172.21.15.16:0/693963468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe98801c070 con 0x7fe990106f10 2024-09-19T09:29:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.208+0000 7fe996ed1740 1 Processor -- start 2024-09-19T09:29:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe996ed1740 1 -- start start 2024-09-19T09:29:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe996ed1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a914090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe996ed1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a9145d0 0x56319a914a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe996ed1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a91f370 0x56319a921760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe996ed1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56319a87a2f0 con 0x56319a915b40 2024-09-19T09:29:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe996ed1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56319a821cf0 con 0x56319a9145d0 2024-09-19T09:29:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe996ed1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56319a8762b0 con 0x56319a91f370 2024-09-19T09:29:51.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe9877fe640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe97c0a3240 0x7fe97c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe987fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a914090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe987fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a914090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55170/0 (socket says 172.21.15.16:55170) 2024-09-19T09:29:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe987fff640 1 -- 172.21.15.16:0/79854776 learned_addr learned my addr 172.21.15.16:0/79854776 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe98e59c640 1 --2- 172.21.15.16:0/79854776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a9145d0 0x56319a914a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe98ed9d640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe97c0a3240 0x7fe97c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe9877fe640 1 -- 172.21.15.16:0/693963468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe9880cef60 con 0x7fe990106f10 2024-09-19T09:29:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.209+0000 7fe987fff640 1 --2- 172.21.15.16:0/79854776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a91f370 0x56319a921760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.210+0000 7fe98e59c640 1 -- 172.21.15.16:0/79854776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a91f370 msgr2=0x56319a921760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.210+0000 7fe98e59c640 1 --2- 172.21.15.16:0/79854776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a91f370 0x56319a921760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.210+0000 7fe98e59c640 1 -- 172.21.15.16:0/79854776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 msgr2=0x56319a914090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.210+0000 7fe98e59c640 1 --2- 172.21.15.16:0/79854776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a914090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.210+0000 7fe98e59c640 1 -- 172.21.15.16:0/79854776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56319a8063a0 con 0x56319a9145d0 2024-09-19T09:29:51.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.210+0000 7fe987fff640 1 --2- 172.21.15.16:0/79854776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a914090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:29:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.210+0000 7fe98ed9d640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe97c0a3240 0x7fe97c0a5700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe988000f80 tx=0x7fe98801c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.210+0000 7fe98e59c640 1 --2- 172.21.15.16:0/79854776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a9145d0 0x56319a914a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe980009230 tx=0x7fe980031630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a31116d561a24d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.210+0000 7fe984ff9640 1 -- 172.21.15.16:0/79854776 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98003f070 con 0x56319a9145d0 2024-09-19T09:29:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.210+0000 7fe984ff9640 1 -- 172.21.15.16:0/79854776 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe980031a20 con 0x56319a9145d0 2024-09-19T09:29:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.210+0000 7fe984ff9640 1 -- 172.21.15.16:0/79854776 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980035070 con 0x56319a9145d0 2024-09-19T09:29:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.211+0000 7fe996ed1740 1 -- 172.21.15.16:0/79854776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a9145d0 msgr2=0x56319a914a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.211+0000 7fe996ed1740 1 --2- 172.21.15.16:0/79854776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a9145d0 0x56319a914a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe980009230 tx=0x7fe980031630 comp rx=0 tx=0).stop 2024-09-19T09:29:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.211+0000 7fe996ed1740 1 -- 172.21.15.16:0/79854776 shutdown_connections 2024-09-19T09:29:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.211+0000 7fe996ed1740 1 --2- 172.21.15.16:0/79854776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a91f370 0x56319a921760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.211+0000 7fe996ed1740 1 --2- 172.21.15.16:0/79854776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a9145d0 0x56319a914a10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.211+0000 7fe996ed1740 1 --2- 172.21.15.16:0/79854776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a914090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.211+0000 7fe996ed1740 1 -- 172.21.15.16:0/79854776 >> 172.21.15.16:0/79854776 conn(0x56319a8dd310 msgr2=0x56319a910c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.211+0000 7fe996ed1740 1 -- 172.21.15.16:0/79854776 shutdown_connections 2024-09-19T09:29:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.211+0000 7fe996ed1740 1 -- 172.21.15.16:0/79854776 wait complete. 2024-09-19T09:29:51.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.211+0000 7fe996ed1740 1 -- start start 2024-09-19T09:29:51.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe996ed1740 1 Processor -- start 2024-09-19T09:29:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe996ed1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a9145d0 0x56319a91ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe996ed1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a91bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe996ed1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a91f370 0x56319a91c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe996ed1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56319a922010 con 0x56319a915b40 2024-09-19T09:29:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe996ed1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56319a835060 con 0x56319a9145d0 2024-09-19T09:29:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe996ed1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56319a922190 con 0x56319a91f370 2024-09-19T09:29:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe996ed1740 10 client.0 _ll_register_callbacks cb 0x56319a5888c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe98e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a91bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe98e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a91bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55186/0 (socket says 172.21.15.16:55186) 2024-09-19T09:29:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe98e59c640 1 -- 172.21.15.16:0/2560313733 learned_addr learned my addr 172.21.15.16:0/2560313733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe987fff640 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a91f370 0x56319a91c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe987fff640 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a9145d0 0x56319a91ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe98e59c640 1 -- 172.21.15.16:0/2560313733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a91f370 msgr2=0x56319a91c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe98e59c640 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a91f370 0x56319a91c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe98e59c640 1 -- 172.21.15.16:0/2560313733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a9145d0 msgr2=0x56319a91ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe98e59c640 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a9145d0 0x56319a91ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe98e59c640 1 -- 172.21.15.16:0/2560313733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56319a5eb3c0 con 0x56319a915b40 2024-09-19T09:29:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe987fff640 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a91f370 0x56319a91c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe98e59c640 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a91bfa0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe98002a930 tx=0x7fe9800052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b991c07d39c9d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.212+0000 7fe975ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe975ffb640 1 -- 172.21.15.16:0/2560313733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98004b020 con 0x56319a915b40 2024-09-19T09:29:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe975ffb640 1 -- 172.21.15.16:0/2560313733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe9800049b0 con 0x56319a915b40 2024-09-19T09:29:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe996ed1740 1 -- 172.21.15.16:0/2560313733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe980031a20 con 0x56319a915b40 2024-09-19T09:29:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe975ffb640 1 -- 172.21.15.16:0/2560313733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe980035070 con 0x56319a915b40 2024-09-19T09:29:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe996ed1740 10 client.16149 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe996ed1740 1 -- 172.21.15.16:0/2560313733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56319a92da40 con 0x56319a915b40 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe975ffb640 1 -- 172.21.15.16:0/2560313733 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe980031a20 con 0x56319a915b40 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe975ffb640 1 -- 172.21.15.16:0/2560313733 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe98003b430 con 0x56319a915b40 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe996ed1740 10 client.16149 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe996ed1740 10 client.16149 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe996ed1740 20 client.16149 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe996ed1740 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56319a92ed90 0x56319a931180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe996ed1740 4 client.16149 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.213+0000 7fe996ed1740 1 -- 172.21.15.16:0/2560313733 --> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56319a887860 con 0x56319a92ed90 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.214+0000 7fe987fff640 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56319a92ed90 0x56319a931180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.214+0000 7fe987fff640 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56319a92ed90 0x56319a931180 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4bef402761296d0 server_cookie=6f4a2de820f0b776 in_seq=0 out_seq=0 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.214+0000 7fe975ffb640 10 client.16149 ms_handle_connect on v2:172.21.15.142:6838/3822134597 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe975ffb640 1 -- 172.21.15.16:0/2560313733 <== mds.1 v2:172.21.15.142:6838/3822134597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56319a887860 con 0x56319a92ed90 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe975ffb640 10 client.16149 handle_command_reply: tid=0 2024-09-19T09:29:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe996ed1740 1 client.16149 shutdown 2024-09-19T09:29:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe996ed1740 1 -- 172.21.15.16:0/2560313733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 msgr2=0x56319a91bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe996ed1740 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a91bfa0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe98002a930 tx=0x7fe9800052d0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe996ed1740 1 -- 172.21.15.16:0/2560313733 shutdown_connections 2024-09-19T09:29:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe996ed1740 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56319a92ed90 0x56319a931180 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe996ed1740 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a91f370 0x56319a91c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe996ed1740 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a91bfa0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe996ed1740 1 --2- 172.21.15.16:0/2560313733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a9145d0 0x56319a91ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe996ed1740 1 -- 172.21.15.16:0/2560313733 >> 172.21.15.16:0/2560313733 conn(0x56319a8dd310 msgr2=0x56319a916960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe975ffb640 0 client.16149 ms_handle_reset on v2:172.21.15.142:6838/3822134597 2024-09-19T09:29:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe996ed1740 1 -- 172.21.15.16:0/2560313733 shutdown_connections 2024-09-19T09:29:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe996ed1740 1 -- 172.21.15.16:0/2560313733 wait complete. 2024-09-19T09:29:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.215+0000 7fe996ed1740 20 client.16149 trim_cache size 0 max 16384 2024-09-19T09:29:51.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.228+0000 7fe996ed1740 1 Processor -- start 2024-09-19T09:29:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.228+0000 7fe996ed1740 1 -- start start 2024-09-19T09:29:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.228+0000 7fe996ed1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a9145d0 0x56319a91df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.228+0000 7fe996ed1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a933dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.228+0000 7fe996ed1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a91f370 0x56319a934300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.228+0000 7fe996ed1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901ad600 con 0x56319a915b40 2024-09-19T09:29:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.228+0000 7fe996ed1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901af140 con 0x56319a91f370 2024-09-19T09:29:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.228+0000 7fe996ed1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901adcd0 con 0x56319a9145d0 2024-09-19T09:29:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe98e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a933dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe98e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a933dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55200/0 (socket says 172.21.15.16:55200) 2024-09-19T09:29:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe98e59c640 1 -- 172.21.15.16:0/1262248061 learned_addr learned my addr 172.21.15.16:0/1262248061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:51.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe987fff640 1 --2- 172.21.15.16:0/1262248061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a9145d0 0x56319a91df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe987fff640 1 --2- 172.21.15.16:0/1262248061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a91f370 0x56319a934300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe98e59c640 1 -- 172.21.15.16:0/1262248061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a9145d0 msgr2=0x56319a91df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe98e59c640 1 --2- 172.21.15.16:0/1262248061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a9145d0 0x56319a91df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe98e59c640 1 -- 172.21.15.16:0/1262248061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a91f370 msgr2=0x56319a934300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe98e59c640 1 --2- 172.21.15.16:0/1262248061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a91f370 0x56319a934300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe98e59c640 1 -- 172.21.15.16:0/1262248061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56319a910510 con 0x56319a915b40 2024-09-19T09:29:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe98e59c640 1 --2- 172.21.15.16:0/1262248061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a933dc0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe980009230 tx=0x7fe980031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d105d07266c9ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe977fff640 1 -- 172.21.15.16:0/1262248061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98004b040 con 0x56319a915b40 2024-09-19T09:29:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe977fff640 1 -- 172.21.15.16:0/1262248061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98003b430 con 0x56319a915b40 2024-09-19T09:29:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.229+0000 7fe977fff640 1 -- 172.21.15.16:0/1262248061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98003f040 con 0x56319a915b40 2024-09-19T09:29:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.230+0000 7fe996ed1740 1 -- 172.21.15.16:0/1262248061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 msgr2=0x56319a933dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.230+0000 7fe996ed1740 1 --2- 172.21.15.16:0/1262248061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a933dc0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe980009230 tx=0x7fe980031630 comp rx=0 tx=0).stop 2024-09-19T09:29:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.230+0000 7fe996ed1740 1 -- 172.21.15.16:0/1262248061 shutdown_connections 2024-09-19T09:29:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.230+0000 7fe996ed1740 1 --2- 172.21.15.16:0/1262248061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a91f370 0x56319a934300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.230+0000 7fe996ed1740 1 --2- 172.21.15.16:0/1262248061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a915b40 0x56319a933dc0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.230+0000 7fe996ed1740 1 --2- 172.21.15.16:0/1262248061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a9145d0 0x56319a91df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.230+0000 7fe996ed1740 1 -- 172.21.15.16:0/1262248061 >> 172.21.15.16:0/1262248061 conn(0x56319a8dd310 msgr2=0x56319a91d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.230+0000 7fe996ed1740 1 -- 172.21.15.16:0/1262248061 shutdown_connections 2024-09-19T09:29:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.230+0000 7fe996ed1740 1 -- 172.21.15.16:0/1262248061 wait complete. 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.230+0000 7fe996ed1740 1 -- start start 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.230+0000 7fe996ed1740 1 Processor -- start 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe996ed1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a9145d0 0x56319a934280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe996ed1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a915b40 0x56319a931620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe996ed1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a91f370 0x56319a931b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe996ed1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901aea70 con 0x56319a91f370 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe996ed1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901ae3a0 con 0x56319a915b40 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe996ed1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901ab5c0 con 0x56319a9145d0 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe996ed1740 10 client.0 _ll_register_callbacks cb 0x7fe97800c440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe987fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a91f370 0x56319a931b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe987fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a91f370 0x56319a931b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55202/0 (socket says 172.21.15.16:55202) 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe987fff640 1 -- 172.21.15.16:0/1144015444 learned_addr learned my addr 172.21.15.16:0/1144015444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe98e59c640 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a915b40 0x56319a931620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe987fff640 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a9145d0 0x56319a934280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe987fff640 1 -- 172.21.15.16:0/1144015444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a9145d0 msgr2=0x56319a934280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe987fff640 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a9145d0 0x56319a934280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe987fff640 1 -- 172.21.15.16:0/1144015444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a915b40 msgr2=0x56319a931620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe987fff640 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a915b40 0x56319a931620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe987fff640 1 -- 172.21.15.16:0/1144015444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56319a9320d0 con 0x56319a91f370 2024-09-19T09:29:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe98e59c640 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a915b40 0x56319a931620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe987fff640 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a91f370 0x56319a931b60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x56319a915070 tx=0x7fe9800056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea4c18ccaadda17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.231+0000 7fe975ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe975ffb640 1 -- 172.21.15.16:0/1144015444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97802e030 con 0x56319a91f370 2024-09-19T09:29:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe975ffb640 1 -- 172.21.15.16:0/1144015444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fe97801b040 con 0x56319a91f370 2024-09-19T09:29:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe996ed1740 1 -- 172.21.15.16:0/1144015444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe98003b430 con 0x56319a91f370 2024-09-19T09:29:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe975ffb640 1 -- 172.21.15.16:0/1144015444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe978025040 con 0x56319a91f370 2024-09-19T09:29:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe996ed1740 10 client.16155 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe996ed1740 1 -- 172.21.15.16:0/1144015444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56319a999490 con 0x56319a91f370 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe975ffb640 1 -- 172.21.15.16:0/1144015444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98003b430 con 0x56319a91f370 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe975ffb640 1 -- 172.21.15.16:0/1144015444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe978003680 con 0x56319a91f370 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe996ed1740 10 client.16155 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe996ed1740 10 client.16155 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe996ed1740 20 client.16155 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe996ed1740 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56319a999be0 0x56319a99c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe996ed1740 4 client.16155 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.232+0000 7fe996ed1740 1 -- 172.21.15.16:0/1144015444 --> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56319a881d70 con 0x56319a999be0 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe98e59c640 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56319a999be0 0x56319a99c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe98e59c640 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56319a999be0 0x56319a99c000 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63b88f5d44b7b66a server_cookie=c3734e868086df5a in_seq=0 out_seq=0 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe975ffb640 10 client.16155 ms_handle_connect on v2:172.21.15.142:6838/3822134597 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe975ffb640 1 -- 172.21.15.16:0/1144015444 <== mds.1 v2:172.21.15.142:6838/3822134597 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56319a881d70 con 0x56319a999be0 2024-09-19T09:29:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe975ffb640 10 client.16155 handle_command_reply: tid=0 2024-09-19T09:29:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe996ed1740 1 client.16155 shutdown 2024-09-19T09:29:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe996ed1740 1 -- 172.21.15.16:0/1144015444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a91f370 msgr2=0x56319a931b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe996ed1740 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a91f370 0x56319a931b60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x56319a915070 tx=0x7fe9800056b0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe996ed1740 1 -- 172.21.15.16:0/1144015444 shutdown_connections 2024-09-19T09:29:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe996ed1740 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56319a91f370 0x56319a931b60 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe996ed1740 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56319a915b40 0x56319a931620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe996ed1740 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.142:6838/3822134597,v1:172.21.15.142:6839/3822134597] conn(0x56319a999be0 0x56319a99c000 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe996ed1740 1 --2- 172.21.15.16:0/1144015444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56319a9145d0 0x56319a934280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.233+0000 7fe996ed1740 1 -- 172.21.15.16:0/1144015444 >> 172.21.15.16:0/1144015444 conn(0x56319a8dd310 msgr2=0x56319a91e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe975ffb640 0 client.16155 ms_handle_reset on v2:172.21.15.142:6838/3822134597 2024-09-19T09:29:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe996ed1740 1 -- 172.21.15.16:0/1144015444 shutdown_connections 2024-09-19T09:29:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe996ed1740 1 -- 172.21.15.16:0/1144015444 wait complete. 2024-09-19T09:29:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe996ed1740 20 client.16155 trim_cache size 0 max 16384 2024-09-19T09:29:51.241 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:29:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe994af8640 1 -- 172.21.15.16:0/693963468 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe97c0a3240 msgr2=0x7fe97c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe994af8640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe97c0a3240 0x7fe97c0a5700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe988000f80 tx=0x7fe98801c000 comp rx=0 tx=0).stop 2024-09-19T09:29:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe994af8640 1 -- 172.21.15.16:0/693963468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990106f10 msgr2=0x7fe99019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe994af8640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990106f10 0x7fe99019cd60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe98800b970 tx=0x7fe98800be30 comp rx=0 tx=0).stop 2024-09-19T09:29:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe994af8640 1 -- 172.21.15.16:0/693963468 shutdown_connections 2024-09-19T09:29:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe994af8640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fe97c0a3240 0x7fe97c0a5700 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe994af8640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe990102790 0x7fe9901989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe994af8640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990106f10 0x7fe99019cd60 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe994af8640 1 --2- 172.21.15.16:0/693963468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe990101de0 0x7fe990198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe994af8640 1 -- 172.21.15.16:0/693963468 >> 172.21.15.16:0/693963468 conn(0x7fe9900fd8c0 msgr2=0x7fe9900fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe994af8640 1 -- 172.21.15.16:0/693963468 shutdown_connections 2024-09-19T09:29:51.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:51.234+0000 7fe994af8640 1 -- 172.21.15.16:0/693963468 wait complete. 2024-09-19T09:29:51.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:51 smithi142 ceph-mon[35032]: pgmap v405: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 113 MiB/s rd, 1.2 MiB/s wr, 452 op/s 2024-09-19T09:29:51.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:51 smithi097 ceph-mon[34864]: pgmap v405: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 113 MiB/s rd, 1.2 MiB/s wr, 452 op/s 2024-09-19T09:29:51.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T09:29:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:52.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.653+0000 7f7b09c0a640 1 -- 172.21.15.16:0/3790172741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b04106a80 msgr2=0x7f7b04106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.653+0000 7f7b09c0a640 1 --2- 172.21.15.16:0/3790172741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b04106a80 0x7f7b04106e60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009930 tx=0x7f7af402a420 comp rx=0 tx=0).stop 2024-09-19T09:29:52.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.653+0000 7f7b09c0a640 1 -- 172.21.15.16:0/3790172741 shutdown_connections 2024-09-19T09:29:52.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.653+0000 7f7b09c0a640 1 --2- 172.21.15.16:0/3790172741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b04101420 0x7f7b0410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.653+0000 7f7b09c0a640 1 --2- 172.21.15.16:0/3790172741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b04100a80 0x7f7b04100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.653+0000 7f7b09c0a640 1 --2- 172.21.15.16:0/3790172741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b04106a80 0x7f7b04106e60 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.653+0000 7f7b09c0a640 1 -- 172.21.15.16:0/3790172741 >> 172.21.15.16:0/3790172741 conn(0x7f7b040fc7a0 msgr2=0x7f7b040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.653+0000 7f7b09c0a640 1 -- 172.21.15.16:0/3790172741 shutdown_connections 2024-09-19T09:29:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.653+0000 7f7b09c0a640 1 -- 172.21.15.16:0/3790172741 wait complete. 2024-09-19T09:29:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b09c0a640 1 Processor -- start 2024-09-19T09:29:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b09c0a640 1 -- start start 2024-09-19T09:29:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b09c0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b04100a80 0x7f7b041a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b09c0a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b04101420 0x7f7b041a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b09c0a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b04106a80 0x7f7b0419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b09c0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04110f70 con 0x7f7b04100a80 2024-09-19T09:29:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b09c0a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04110df0 con 0x7f7b04101420 2024-09-19T09:29:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b09c0a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041110f0 con 0x7f7b04106a80 2024-09-19T09:29:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b04100a80 0x7f7b041a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b04100a80 0x7f7b041a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55210/0 (socket says 172.21.15.16:55210) 2024-09-19T09:29:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b037fe640 1 -- 172.21.15.16:0/2016868216 learned_addr learned my addr 172.21.15.16:0/2016868216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b03fff640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b04106a80 0x7f7b0419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7afaffd640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b04101420 0x7f7b041a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b037fe640 1 -- 172.21.15.16:0/2016868216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b04106a80 msgr2=0x7f7b0419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b037fe640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b04106a80 0x7f7b0419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b037fe640 1 -- 172.21.15.16:0/2016868216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b04101420 msgr2=0x7f7b041a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.654+0000 7f7b037fe640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b04101420 0x7f7b041a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.655+0000 7f7b037fe640 1 -- 172.21.15.16:0/2016868216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0419ad00 con 0x7f7b04100a80 2024-09-19T09:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.655+0000 7f7afaffd640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b04101420 0x7f7b041a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.655+0000 7f7b037fe640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b04100a80 0x7f7b041a0410 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7af402a940 tx=0x7f7af4003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.655+0000 7f7b017fa640 1 -- 172.21.15.16:0/2016868216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af403f070 con 0x7f7b04100a80 2024-09-19T09:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.655+0000 7f7b017fa640 1 -- 172.21.15.16:0/2016868216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af403b460 con 0x7f7b04100a80 2024-09-19T09:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.655+0000 7f7b09c0a640 1 -- 172.21.15.16:0/2016868216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0419af90 con 0x7f7b04100a80 2024-09-19T09:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.655+0000 7f7b017fa640 1 -- 172.21.15.16:0/2016868216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4035030 con 0x7f7b04100a80 2024-09-19T09:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.655+0000 7f7b09c0a640 1 -- 172.21.15.16:0/2016868216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b041a72e0 con 0x7f7b04100a80 2024-09-19T09:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.657+0000 7f7b0bfe3740 1 Processor -- start 2024-09-19T09:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.657+0000 7f7b017fa640 1 -- 172.21.15.16:0/2016868216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7af4042070 con 0x7f7b04100a80 2024-09-19T09:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.657+0000 7f7b0bfe3740 1 -- start start 2024-09-19T09:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7b0bfe3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d181b40 0x56241d180090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7b0bfe3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d1805d0 0x56241d180a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7b0bfe3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d18d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7b0bfe3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56241d0e62f0 con 0x56241d18b370 2024-09-19T09:29:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7b0bfe3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56241d08dcf0 con 0x56241d1805d0 2024-09-19T09:29:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7b0bfe3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56241d0e22b0 con 0x56241d181b40 2024-09-19T09:29:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7afaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d18d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7afaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d18d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55222/0 (socket says 172.21.15.16:55222) 2024-09-19T09:29:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7afaffd640 1 -- 172.21.15.16:0/136473896 learned_addr learned my addr 172.21.15.16:0/136473896 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7b037fe640 1 --2- 172.21.15.16:0/136473896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d1805d0 0x56241d180a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7b03fff640 1 --2- 172.21.15.16:0/136473896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d181b40 0x56241d180090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7afaffd640 1 -- 172.21.15.16:0/136473896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d181b40 msgr2=0x56241d180090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7afaffd640 1 --2- 172.21.15.16:0/136473896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d181b40 0x56241d180090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7afaffd640 1 -- 172.21.15.16:0/136473896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d1805d0 msgr2=0x56241d180a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:52.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7afaffd640 1 --2- 172.21.15.16:0/136473896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d1805d0 0x56241d180a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7afaffd640 1 -- 172.21.15.16:0/136473896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241d0723a0 con 0x56241d18b370 2024-09-19T09:29:52.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7b037fe640 1 --2- 172.21.15.16:0/136473896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d1805d0 0x56241d180a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:52.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.658+0000 7f7b03fff640 1 --2- 172.21.15.16:0/136473896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d181b40 0x56241d180090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:52.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.659+0000 7f7b017fa640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7af00a3240 0x7f7af00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.659+0000 7f7afaffd640 1 --2- 172.21.15.16:0/136473896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d18d760 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00a9a0 tx=0x7f7aec00ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c45dd5f4cb18e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:52.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.659+0000 7f7b03fff640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7af00a3240 0x7f7af00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.659+0000 7f7af9ffb640 1 -- 172.21.15.16:0/136473896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec015070 con 0x56241d18b370 2024-09-19T09:29:52.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.659+0000 7f7b017fa640 1 -- 172.21.15.16:0/2016868216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7af40f02c0 con 0x7f7b04100a80 2024-09-19T09:29:52.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.659+0000 7f7af9ffb640 1 -- 172.21.15.16:0/136473896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aec00cc00 con 0x56241d18b370 2024-09-19T09:29:52.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.659+0000 7f7af9ffb640 1 -- 172.21.15.16:0/136473896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec018600 con 0x56241d18b370 2024-09-19T09:29:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.660+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/136473896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 msgr2=0x56241d18d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.660+0000 7f7b03fff640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7af00a3240 0x7f7af00a5700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00af30 tx=0x7f7afc00a550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.660+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/136473896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d18d760 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00a9a0 tx=0x7f7aec00ae60 comp rx=0 tx=0).stop 2024-09-19T09:29:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.660+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/136473896 shutdown_connections 2024-09-19T09:29:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.660+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/136473896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d18d760 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.660+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/136473896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d1805d0 0x56241d180a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.660+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/136473896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d181b40 0x56241d180090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.660+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/136473896 >> 172.21.15.16:0/136473896 conn(0x56241d149310 msgr2=0x56241d17cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.660+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/136473896 shutdown_connections 2024-09-19T09:29:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.660+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/136473896 wait complete. 2024-09-19T09:29:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.660+0000 7f7b0bfe3740 1 -- start start 2024-09-19T09:29:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.660+0000 7f7b0bfe3740 1 Processor -- start 2024-09-19T09:29:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b0bfe3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d1805d0 0x56241d18ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b0bfe3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d181b40 0x56241d187f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b0bfe3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d188460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b0bfe3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56241d18e070 con 0x56241d18b370 2024-09-19T09:29:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b0bfe3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56241d0a1060 con 0x56241d1805d0 2024-09-19T09:29:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d188460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d188460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55234/0 (socket says 172.21.15.16:55234) 2024-09-19T09:29:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b03fff640 1 -- 172.21.15.16:0/720313848 learned_addr learned my addr 172.21.15.16:0/720313848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b0bfe3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56241d18e1f0 con 0x56241d181b40 2024-09-19T09:29:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b037fe640 1 --2- 172.21.15.16:0/720313848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d1805d0 0x56241d18ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b0bfe3740 10 client.0 _ll_register_callbacks cb 0x56241cdf48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7afaffd640 1 --2- 172.21.15.16:0/720313848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d181b40 0x56241d187f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b03fff640 1 -- 172.21.15.16:0/720313848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d181b40 msgr2=0x56241d187f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b03fff640 1 --2- 172.21.15.16:0/720313848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d181b40 0x56241d187f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b03fff640 1 -- 172.21.15.16:0/720313848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d1805d0 msgr2=0x56241d18ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b03fff640 1 --2- 172.21.15.16:0/720313848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d1805d0 0x56241d18ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.661+0000 7f7b03fff640 1 -- 172.21.15.16:0/720313848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56241ce573c0 con 0x56241d18b370 2024-09-19T09:29:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7b03fff640 1 --2- 172.21.15.16:0/720313848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d188460 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7afc019e20 tx=0x7f7afc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371ad10b26ba0ba6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7ae2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7ae2ffd640 1 -- 172.21.15.16:0/720313848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc027060 con 0x56241d18b370 2024-09-19T09:29:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7ae2ffd640 1 -- 172.21.15.16:0/720313848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7afc0190d0 con 0x56241d18b370 2024-09-19T09:29:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/720313848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aec00cc00 con 0x56241d18b370 2024-09-19T09:29:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7ae2ffd640 1 -- 172.21.15.16:0/720313848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7afc017420 con 0x56241d18b370 2024-09-19T09:29:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7b0bfe3740 10 client.16167 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/720313848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56241d199a20 con 0x56241d18b370 2024-09-19T09:29:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7ae2ffd640 1 -- 172.21.15.16:0/720313848 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aec00cc00 con 0x56241d18b370 2024-09-19T09:29:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7ae2ffd640 1 -- 172.21.15.16:0/720313848 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7afc02d610 con 0x56241d18b370 2024-09-19T09:29:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7b0bfe3740 10 client.16167 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7b0bfe3740 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7b0bfe3740 20 client.16167 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/720313848 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56241d19ae30 0x56241d19d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7b0bfe3740 4 client.16167 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T09:29:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.662+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/720313848 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56241d0f3860 con 0x56241d19ae30 2024-09-19T09:29:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.663+0000 7f7b037fe640 1 --2- 172.21.15.16:0/720313848 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56241d19ae30 0x56241d19d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.663+0000 7f7b037fe640 1 --2- 172.21.15.16:0/720313848 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56241d19ae30 0x56241d19d280 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af27a7b56c4b1bd2 server_cookie=ce0fcd22a68d5754 in_seq=0 out_seq=0 2024-09-19T09:29:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.663+0000 7f7ae2ffd640 10 client.16167 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.664+0000 7f7ae2ffd640 1 -- 172.21.15.16:0/720313848 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56241d0f3860 con 0x56241d19ae30 2024-09-19T09:29:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.664+0000 7f7ae2ffd640 10 client.16167 handle_command_reply: tid=0 2024-09-19T09:29:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.664+0000 7f7b0bfe3740 1 client.16167 shutdown 2024-09-19T09:29:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.664+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/720313848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 msgr2=0x56241d188460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.664+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/720313848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d188460 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7afc019e20 tx=0x7f7afc004a40 comp rx=0 tx=0).stop 2024-09-19T09:29:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.664+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/720313848 shutdown_connections 2024-09-19T09:29:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.664+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/720313848 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56241d19ae30 0x56241d19d280 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.664+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/720313848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d181b40 0x56241d187f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.664+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/720313848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d188460 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.664+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/720313848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d1805d0 0x56241d18ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.664+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/720313848 >> 172.21.15.16:0/720313848 conn(0x56241d149310 msgr2=0x56241d182960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.664+0000 7f7ae2ffd640 0 client.16167 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.665+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/720313848 shutdown_connections 2024-09-19T09:29:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.665+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/720313848 wait complete. 2024-09-19T09:29:52.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.665+0000 7f7b0bfe3740 20 client.16167 trim_cache size 0 max 16384 2024-09-19T09:29:52.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.680+0000 7f7b0bfe3740 1 Processor -- start 2024-09-19T09:29:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b0bfe3740 1 -- start start 2024-09-19T09:29:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b0bfe3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d1805d0 0x56241d197340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b0bfe3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d181b40 0x56241d17c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b0bfe3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d18b370 0x56241d17cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b0bfe3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041af8d0 con 0x56241d1805d0 2024-09-19T09:29:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b0bfe3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b1410 con 0x56241d181b40 2024-09-19T09:29:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b0bfe3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041affa0 con 0x56241d18b370 2024-09-19T09:29:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d1805d0 0x56241d197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d1805d0 0x56241d197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55236/0 (socket says 172.21.15.16:55236) 2024-09-19T09:29:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b037fe640 1 -- 172.21.15.16:0/2956472959 learned_addr learned my addr 172.21.15.16:0/2956472959 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7afaffd640 1 --2- 172.21.15.16:0/2956472959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d181b40 0x56241d17c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b037fe640 1 -- 172.21.15.16:0/2956472959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d18b370 msgr2=0x56241d17cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b037fe640 1 --2- 172.21.15.16:0/2956472959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d18b370 0x56241d17cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b037fe640 1 -- 172.21.15.16:0/2956472959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d181b40 msgr2=0x56241d17c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b037fe640 1 --2- 172.21.15.16:0/2956472959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d181b40 0x56241d17c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b037fe640 1 -- 172.21.15.16:0/2956472959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241d17d180 con 0x56241d1805d0 2024-09-19T09:29:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7b037fe640 1 --2- 172.21.15.16:0/2956472959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d1805d0 0x56241d197340 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7af4005ec0 tx=0x7f7af403b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a820dd876ae8965 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.681+0000 7f7af97fa640 1 -- 172.21.15.16:0/2956472959 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4049040 con 0x56241d1805d0 2024-09-19T09:29:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.682+0000 7f7af97fa640 1 -- 172.21.15.16:0/2956472959 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af40bc8a0 con 0x56241d1805d0 2024-09-19T09:29:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.682+0000 7f7af97fa640 1 -- 172.21.15.16:0/2956472959 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af40b8f10 con 0x56241d1805d0 2024-09-19T09:29:52.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.682+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/2956472959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d1805d0 msgr2=0x56241d197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.682+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/2956472959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d1805d0 0x56241d197340 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7af4005ec0 tx=0x7f7af403b910 comp rx=0 tx=0).stop 2024-09-19T09:29:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.682+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/2956472959 shutdown_connections 2024-09-19T09:29:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.682+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/2956472959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d18b370 0x56241d17cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.682+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/2956472959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d181b40 0x56241d17c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.682+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/2956472959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d1805d0 0x56241d197340 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.682+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/2956472959 >> 172.21.15.16:0/2956472959 conn(0x56241d149310 msgr2=0x56241d187ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.682+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/2956472959 shutdown_connections 2024-09-19T09:29:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.682+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/2956472959 wait complete. 2024-09-19T09:29:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7b0bfe3740 1 -- start start 2024-09-19T09:29:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7b0bfe3740 1 Processor -- start 2024-09-19T09:29:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7b0bfe3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d1805d0 0x56241d193690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7b0bfe3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d181b40 0x56241d193bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7b0bfe3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d184730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7b0bfe3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b0d40 con 0x56241d18b370 2024-09-19T09:29:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7b0bfe3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b0670 con 0x56241d181b40 2024-09-19T09:29:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7b0bfe3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041ad890 con 0x56241d1805d0 2024-09-19T09:29:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7b0bfe3740 10 client.0 _ll_register_callbacks cb 0x7f7afc00ab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T09:29:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7b03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d184730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7b03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d184730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55248/0 (socket says 172.21.15.16:55248) 2024-09-19T09:29:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7b03fff640 1 -- 172.21.15.16:0/456267972 learned_addr learned my addr 172.21.15.16:0/456267972 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7afaffd640 1 --2- 172.21.15.16:0/456267972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d181b40 0x56241d193bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.683+0000 7f7b037fe640 1 --2- 172.21.15.16:0/456267972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d1805d0 0x56241d193690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7b03fff640 1 -- 172.21.15.16:0/456267972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d1805d0 msgr2=0x56241d193690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7b03fff640 1 --2- 172.21.15.16:0/456267972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d1805d0 0x56241d193690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7b03fff640 1 -- 172.21.15.16:0/456267972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d181b40 msgr2=0x56241d193bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T09:29:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7b03fff640 1 --2- 172.21.15.16:0/456267972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d181b40 0x56241d193bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7b03fff640 1 -- 172.21.15.16:0/456267972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56241d184f60 con 0x56241d18b370 2024-09-19T09:29:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7b03fff640 1 --2- 172.21.15.16:0/456267972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d184730 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009c10 tx=0x7f7afc016fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b353c7eab587d011 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7ae2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-19T09:29:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7ae2ffd640 1 -- 172.21.15.16:0/456267972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc02dc00 con 0x56241d18b370 2024-09-19T09:29:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7ae2ffd640 1 -- 172.21.15.16:0/456267972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f7afc02d450 con 0x56241d18b370 2024-09-19T09:29:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/456267972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7af40bc8a0 con 0x56241d18b370 2024-09-19T09:29:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7ae2ffd640 1 -- 172.21.15.16:0/456267972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7afc03d050 con 0x56241d18b370 2024-09-19T09:29:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7b0bfe3740 10 client.16173 fetch_fsmap learned FSMap version 28 2024-09-19T09:29:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/456267972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56241d205660 con 0x56241d18b370 2024-09-19T09:29:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7ae2ffd640 1 -- 172.21.15.16:0/456267972 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af40bc8a0 con 0x56241d18b370 2024-09-19T09:29:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.684+0000 7f7ae2ffd640 1 -- 172.21.15.16:0/456267972 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7afc017c60 con 0x56241d18b370 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.685+0000 7f7b0bfe3740 10 client.16173 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.685+0000 7f7b0bfe3740 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.685+0000 7f7b0bfe3740 20 client.16173 populate_metadata read hostname 'smithi016' 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.685+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/456267972 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56241d205d50 0x56241d208170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.685+0000 7f7b0bfe3740 4 client.16173 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.685+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/456267972 --> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56241d0edd70 con 0x56241d205d50 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.685+0000 7f7b037fe640 1 --2- 172.21.15.16:0/456267972 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56241d205d50 0x56241d208170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.685+0000 7f7ae2ffd640 10 client.16173 ms_handle_connect on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.685+0000 7f7b037fe640 1 --2- 172.21.15.16:0/456267972 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56241d205d50 0x56241d208170 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbcdf5d3ae57591d server_cookie=9c2b7b2f649d55f7 in_seq=0 out_seq=0 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.685+0000 7f7ae2ffd640 1 -- 172.21.15.16:0/456267972 <== mds.0 v2:172.21.15.97:6838/3225392797 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56241d0edd70 con 0x56241d205d50 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.685+0000 7f7ae2ffd640 10 client.16173 handle_command_reply: tid=0 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.685+0000 7f7b0bfe3740 1 client.16173 shutdown 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/456267972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 msgr2=0x56241d184730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/456267972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d184730 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009c10 tx=0x7f7afc016fd0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/456267972 shutdown_connections 2024-09-19T09:29:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/456267972 >> [v2:172.21.15.97:6838/3225392797,v1:172.21.15.97:6839/3225392797] conn(0x56241d205d50 0x56241d208170 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/456267972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56241d18b370 0x56241d184730 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/456267972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56241d181b40 0x56241d193bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b0bfe3740 1 --2- 172.21.15.16:0/456267972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56241d1805d0 0x56241d193690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7ae2ffd640 0 client.16173 ms_handle_reset on v2:172.21.15.97:6838/3225392797 2024-09-19T09:29:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/456267972 >> 172.21.15.16:0/456267972 conn(0x56241d149310 msgr2=0x56241d187ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/456267972 shutdown_connections 2024-09-19T09:29:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b0bfe3740 1 -- 172.21.15.16:0/456267972 wait complete. 2024-09-19T09:29:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b0bfe3740 20 client.16173 trim_cache size 0 max 16384 2024-09-19T09:29:52.693 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-19T09:29:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b09c0a640 1 -- 172.21.15.16:0/2016868216 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7af00a3240 msgr2=0x7f7af00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b09c0a640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7af00a3240 0x7f7af00a5700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00af30 tx=0x7f7afc00a550 comp rx=0 tx=0).stop 2024-09-19T09:29:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b09c0a640 1 -- 172.21.15.16:0/2016868216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b04100a80 msgr2=0x7f7b041a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.686+0000 7f7b09c0a640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b04100a80 0x7f7b041a0410 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7af402a940 tx=0x7f7af4003c00 comp rx=0 tx=0).stop 2024-09-19T09:29:52.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.687+0000 7f7b09c0a640 1 -- 172.21.15.16:0/2016868216 shutdown_connections 2024-09-19T09:29:52.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.687+0000 7f7b09c0a640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f7af00a3240 0x7f7af00a5700 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.687+0000 7f7b09c0a640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b04106a80 0x7f7b0419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.687+0000 7f7b09c0a640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b04101420 0x7f7b041a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.687+0000 7f7b09c0a640 1 --2- 172.21.15.16:0/2016868216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b04100a80 0x7f7b041a0410 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:52.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.687+0000 7f7b09c0a640 1 -- 172.21.15.16:0/2016868216 >> 172.21.15.16:0/2016868216 conn(0x7f7b040fc7a0 msgr2=0x7f7b0406a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:52.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.687+0000 7f7b09c0a640 1 -- 172.21.15.16:0/2016868216 shutdown_connections 2024-09-19T09:29:52.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:52.687+0000 7f7b09c0a640 1 -- 172.21.15.16:0/2016868216 wait complete. 2024-09-19T09:29:53.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:53 smithi016 ceph-mon[30979]: pgmap v406: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s rd, 838 KiB/s wr, 346 op/s 2024-09-19T09:29:53.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:53 smithi142 ceph-mon[35032]: pgmap v406: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s rd, 838 KiB/s wr, 346 op/s 2024-09-19T09:29:53.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:53 smithi097 ceph-mon[34864]: pgmap v406: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s rd, 838 KiB/s wr, 346 op/s 2024-09-19T09:29:53.448 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-19T09:29:53.449 INFO:tasks.fwd_scrub:done joining 2024-09-19T09:29:53.449 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T09:29:53.461 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T09:29:53.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.567+0000 7ff949bf8640 1 -- 172.21.15.16:0/2832181928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff944102610 msgr2=0x7ff944104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.567+0000 7ff949bf8640 1 --2- 172.21.15.16:0/2832181928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff944102610 0x7ff944104a00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff934009900 tx=0x7ff93402a410 comp rx=0 tx=0).stop 2024-09-19T09:29:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.567+0000 7ff949bf8640 1 -- 172.21.15.16:0/2832181928 shutdown_connections 2024-09-19T09:29:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.567+0000 7ff949bf8640 1 --2- 172.21.15.16:0/2832181928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff944107870 0x7ff944109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.567+0000 7ff949bf8640 1 --2- 172.21.15.16:0/2832181928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff944104f40 0x7ff944107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.567+0000 7ff949bf8640 1 --2- 172.21.15.16:0/2832181928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff944102610 0x7ff944104a00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.567+0000 7ff949bf8640 1 -- 172.21.15.16:0/2832181928 >> 172.21.15.16:0/2832181928 conn(0x7ff9440fc1b0 msgr2=0x7ff9440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.567+0000 7ff949bf8640 1 -- 172.21.15.16:0/2832181928 shutdown_connections 2024-09-19T09:29:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.567+0000 7ff949bf8640 1 -- 172.21.15.16:0/2832181928 wait complete. 2024-09-19T09:29:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.567+0000 7ff949bf8640 1 Processor -- start 2024-09-19T09:29:53.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.567+0000 7ff949bf8640 1 -- start start 2024-09-19T09:29:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff949bf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff944102610 0x7ff944101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff949bf8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff944104f40 0x7ff9441005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff949bf8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff944107870 0x7ff944100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff949bf8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff94410d800 con 0x7ff944102610 2024-09-19T09:29:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff949bf8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff94410d680 con 0x7ff944107870 2024-09-19T09:29:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff949bf8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff94410d980 con 0x7ff944104f40 2024-09-19T09:29:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff9437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff944102610 0x7ff944101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff9437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff944102610 0x7ff944101f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55264/0 (socket says 172.21.15.16:55264) 2024-09-19T09:29:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff9437fe640 1 -- 172.21.15.16:0/3624433718 learned_addr learned my addr 172.21.15.16:0/3624433718 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff943fff640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff944107870 0x7ff944100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff942ffd640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff944104f40 0x7ff9441005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff9437fe640 1 -- 172.21.15.16:0/3624433718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff944104f40 msgr2=0x7ff9441005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff9437fe640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff944104f40 0x7ff9441005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff9437fe640 1 -- 172.21.15.16:0/3624433718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff944107870 msgr2=0x7ff944100b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff9437fe640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff944107870 0x7ff944100b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff9437fe640 1 -- 172.21.15.16:0/3624433718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff944101390 con 0x7ff944102610 2024-09-19T09:29:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff942ffd640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff944104f40 0x7ff9441005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff9437fe640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff944102610 0x7ff944101f80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff934009230 tx=0x7ff934004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.568+0000 7ff940ff9640 1 -- 172.21.15.16:0/3624433718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93403f030 con 0x7ff944102610 2024-09-19T09:29:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.569+0000 7ff940ff9640 1 -- 172.21.15.16:0/3624433718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff934003ce0 con 0x7ff944102610 2024-09-19T09:29:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.569+0000 7ff949bf8640 1 -- 172.21.15.16:0/3624433718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9441a1f70 con 0x7ff944102610 2024-09-19T09:29:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.569+0000 7ff940ff9640 1 -- 172.21.15.16:0/3624433718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934035070 con 0x7ff944102610 2024-09-19T09:29:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.569+0000 7ff949bf8640 1 -- 172.21.15.16:0/3624433718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9441a2380 con 0x7ff944102610 2024-09-19T09:29:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.570+0000 7ff949bf8640 1 -- 172.21.15.16:0/3624433718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9440fdca0 con 0x7ff944102610 2024-09-19T09:29:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.571+0000 7ff940ff9640 1 -- 172.21.15.16:0/3624433718 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff934042030 con 0x7ff944102610 2024-09-19T09:29:53.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.571+0000 7ff940ff9640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff92c0a3250 0x7ff92c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:53.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.571+0000 7ff940ff9640 1 -- 172.21.15.16:0/3624433718 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7ff9340f0350 con 0x7ff944102610 2024-09-19T09:29:53.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.571+0000 7ff942ffd640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff92c0a3250 0x7ff92c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:53.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.572+0000 7ff942ffd640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff92c0a3250 0x7ff92c0a5710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff944101cf0 tx=0x7ff938005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:53.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.576+0000 7ff940ff9640 1 -- 172.21.15.16:0/3624433718 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff9340bd070 con 0x7ff944102610 2024-09-19T09:29:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.752+0000 7ff949bf8640 1 -- 172.21.15.16:0/3624433718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff944062f70 con 0x7ff944102610 2024-09-19T09:29:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.753+0000 7ff940ff9640 1 -- 172.21.15.16:0/3624433718 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22665 (secure 0 0 0) 0x7ff9340bc970 con 0x7ff944102610 2024-09-19T09:29:53.755 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:29:53.755 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","created":"2024-09-19T09:17:48.242238+0000","modified":"2024-09-19T09:25:42.982671+0000","last_up_change":"2024-09-19T09:24:40.190289+0000","last_in_change":"2024-09-19T09:24:17.190380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:20:57.748686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:25:25.694451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:25:28.262997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbf3b28b-2282-4057-9e65-19ceb78ff19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6803","nonce":1578957623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6805","nonce":1578957623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6809","nonce":1578957623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6807","nonce":1578957623}]},"public_addr":"172.21.15.16:6803/1578957623","cluster_addr":"172.21.15.16:6805/1578957623","heartbeat_back_addr":"172.21.15.16:6809/1578957623","heartbeat_front_addr":"172.21.15.16:6807/1578957623","state":["exists","up"]},{"osd":1,"uuid":"c2517aeb-1d79-41ad-a180-369eecb9732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6801","nonce":3804152349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6803","nonce":3804152349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6807","nonce":3804152349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6805","nonce":3804152349}]},"public_addr":"172.21.15.97:6801/3804152349","cluster_addr":"172.21.15.97:6803/3804152349","heartbeat_back_addr":"172.21.15.97:6807/3804152349","heartbeat_front_addr":"172.21.15.97:6805/3804152349","state":["exists","up"]},{"osd":2,"uuid":"3c367518-5085-471a-98cb-ec5bf5facdf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6801","nonce":2692613951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6803","nonce":2692613951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6807","nonce":2692613951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6805","nonce":2692613951}]},"public_addr":"172.21.15.142:6801/2692613951","cluster_addr":"172.21.15.142:6803/2692613951","heartbeat_back_addr":"172.21.15.142:6807/2692613951","heartbeat_front_addr":"172.21.15.142:6805/2692613951","state":["exists","up"]},{"osd":3,"uuid":"e7118925-f0fa-46b3-8552-3e3bb76a6489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6811","nonce":3169290004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6813","nonce":3169290004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6817","nonce":3169290004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6815","nonce":3169290004}]},"public_addr":"172.21.15.16:6811/3169290004","cluster_addr":"172.21.15.16:6813/3169290004","heartbeat_back_addr":"172.21.15.16:6817/3169290004","heartbeat_front_addr":"172.21.15.16:6815/3169290004","state":["exists","up"]},{"osd":4,"uuid":"dc1964f7-af9c-4853-843b-9818f87c7768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6809","nonce":3491353825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6811","nonce":3491353825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6815","nonce":3491353825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6813","nonce":3491353825}]},"public_addr":"172.21.15.97:6809/3491353825","cluster_addr":"172.21.15.97:6811/3491353825","heartbeat_back_addr":"172.21.15.97:6815/3491353825","heartbeat_front_addr":"172.21.15.97:6813/3491353825","state":["exists","up"]},{"osd":5,"uuid":"4d7fbafc-d9ba-4036-a89d-aa295b8a23a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6809","nonce":577099986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6811","nonce":577099986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6815","nonce":577099986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6813","nonce":577099986}]},"public_addr":"172.21.15.142:6809/577099986","cluster_addr":"172.21.15.142:6811/577099986","heartbeat_back_addr":"172.21.15.142:6815/577099986","heartbeat_front_addr":"172.21.15.142:6813/577099986","state":["exists","up"]},{"osd":6,"uuid":"e3f346fc-c027-44f8-bd37-bccf6ec09e3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6819","nonce":4053539536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6821","nonce":4053539536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6825","nonce":4053539536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6823","nonce":4053539536}]},"public_addr":"172.21.15.16:6819/4053539536","cluster_addr":"172.21.15.16:6821/4053539536","heartbeat_back_addr":"172.21.15.16:6825/4053539536","heartbeat_front_addr":"172.21.15.16:6823/4053539536","state":["exists","up"]},{"osd":7,"uuid":"d390b547-8949-4d4b-8481-c49d6ae7e424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6817","nonce":1157122670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6819","nonce":1157122670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6823","nonce":1157122670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6821","nonce":1157122670}]},"public_addr":"172.21.15.97:6817/1157122670","cluster_addr":"172.21.15.97:6819/1157122670","heartbeat_back_addr":"172.21.15.97:6823/1157122670","heartbeat_front_addr":"172.21.15.97:6821/1157122670","state":["exists","up"]},{"osd":8,"uuid":"78a476c4-5deb-4f9e-a266-db4493850732","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6817","nonce":643795004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6819","nonce":643795004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6823","nonce":643795004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6821","nonce":643795004}]},"public_addr":"172.21.15.142:6817/643795004","cluster_addr":"172.21.15.142:6819/643795004","heartbeat_back_addr":"172.21.15.142:6823/643795004","heartbeat_front_addr":"172.21.15.142:6821/643795004","state":["exists","up"]},{"osd":9,"uuid":"0415d187-4a77-42f7-b8bb-2d0b52e9cb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6827","nonce":2651132360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6829","nonce":2651132360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2651132360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6831","nonce":2651132360}]},"public_addr":"172.21.15.16:6827/2651132360","cluster_addr":"172.21.15.16:6829/2651132360","heartbeat_back_addr":"172.21.15.16:6833/2651132360","heartbeat_front_addr":"172.21.15.16:6831/2651132360","state":["exists","up"]},{"osd":10,"uuid":"6fa78f96-cdfe-42a0-8d27-0705a0c7df8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6825","nonce":1848371486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6827","nonce":1848371486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6831","nonce":1848371486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6829","nonce":1848371486}]},"public_addr":"172.21.15.97:6825/1848371486","cluster_addr":"172.21.15.97:6827/1848371486","heartbeat_back_addr":"172.21.15.97:6831/1848371486","heartbeat_front_addr":"172.21.15.97:6829/1848371486","state":["exists","up"]},{"osd":11,"uuid":"d27f20e3-84ce-470c-9889-cd86906b158d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6825","nonce":155083878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6827","nonce":155083878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6831","nonce":155083878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6829","nonce":155083878}]},"public_addr":"172.21.15.142:6825/155083878","cluster_addr":"172.21.15.142:6827/155083878","heartbeat_back_addr":"172.21.15.142:6831/155083878","heartbeat_front_addr":"172.21.15.142:6829/155083878","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:05.088764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:29.928529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:55.236930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:20.379471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:44.984151+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:10.410074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:35.233553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:59.928700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:24.503069+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:49.853073+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:14.006841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:38.546467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3710267992":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3740275561":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/2755421444":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/357390846":"2024-09-20T09:18:17.070034+0000","172.21.15.16:6800/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3104087402":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3546947161":"2024-09-20T09:18:17.070034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:29:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.755+0000 7ff949bf8640 1 -- 172.21.15.16:0/3624433718 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff92c0a3250 msgr2=0x7ff92c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.755+0000 7ff949bf8640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff92c0a3250 0x7ff92c0a5710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff944101cf0 tx=0x7ff938005d30 comp rx=0 tx=0).stop 2024-09-19T09:29:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.755+0000 7ff949bf8640 1 -- 172.21.15.16:0/3624433718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff944102610 msgr2=0x7ff944101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.755+0000 7ff949bf8640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff944102610 0x7ff944101f80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff934009230 tx=0x7ff934004360 comp rx=0 tx=0).stop 2024-09-19T09:29:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.755+0000 7ff949bf8640 1 -- 172.21.15.16:0/3624433718 shutdown_connections 2024-09-19T09:29:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.755+0000 7ff949bf8640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7ff92c0a3250 0x7ff92c0a5710 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.755+0000 7ff949bf8640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff944104f40 0x7ff9441005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.755+0000 7ff949bf8640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff944107870 0x7ff944100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.755+0000 7ff949bf8640 1 --2- 172.21.15.16:0/3624433718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff944102610 0x7ff944101f80 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.755+0000 7ff949bf8640 1 -- 172.21.15.16:0/3624433718 >> 172.21.15.16:0/3624433718 conn(0x7ff9440fc1b0 msgr2=0x7ff944103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.755+0000 7ff949bf8640 1 -- 172.21.15.16:0/3624433718 shutdown_connections 2024-09-19T09:29:53.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.756+0000 7ff949bf8640 1 -- 172.21.15.16:0/3624433718 wait complete. 2024-09-19T09:29:53.768 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T09:20:57.748686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T09:25:25.694451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T09:25:28.262997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T09:29:53.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T09:29:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.874+0000 7f41c58ac640 1 -- 172.21.15.16:0/759810371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c0107880 msgr2=0x7f41c0109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.874+0000 7f41c58ac640 1 --2- 172.21.15.16:0/759810371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c0107880 0x7f41c0109c70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009ca0 tx=0x7f41b802a8c0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.874+0000 7f41c58ac640 1 -- 172.21.15.16:0/759810371 shutdown_connections 2024-09-19T09:29:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.874+0000 7f41c58ac640 1 --2- 172.21.15.16:0/759810371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0104f50 0x7f41c0107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.874+0000 7f41c58ac640 1 --2- 172.21.15.16:0/759810371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c0107880 0x7f41c0109c70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.874+0000 7f41c58ac640 1 --2- 172.21.15.16:0/759810371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c0102620 0x7f41c0104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.874+0000 7f41c58ac640 1 -- 172.21.15.16:0/759810371 >> 172.21.15.16:0/759810371 conn(0x7f41c00fc140 msgr2=0x7f41c00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.874+0000 7f41c58ac640 1 -- 172.21.15.16:0/759810371 shutdown_connections 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.874+0000 7f41c58ac640 1 -- 172.21.15.16:0/759810371 wait complete. 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.874+0000 7f41c58ac640 1 Processor -- start 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.874+0000 7f41c58ac640 1 -- start start 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41c58ac640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0102620 0x7f41c0197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41c58ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c0104f50 0x7f41c01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41c58ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c0107880 0x7f41c019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41c58ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41c010d7a0 con 0x7f41c0104f50 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41c58ac640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41c010d620 con 0x7f41c0107880 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41c58ac640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41c010d920 con 0x7f41c0102620 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41be7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c0104f50 0x7f41c01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41be7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c0104f50 0x7f41c01983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55284/0 (socket says 172.21.15.16:55284) 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41be7fc640 1 -- 172.21.15.16:0/1402201163 learned_addr learned my addr 172.21.15.16:0/1402201163 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41bf7fe640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c0107880 0x7f41c019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41beffd640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0102620 0x7f41c0197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41be7fc640 1 -- 172.21.15.16:0/1402201163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0102620 msgr2=0x7f41c0197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41be7fc640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0102620 0x7f41c0197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41be7fc640 1 -- 172.21.15.16:0/1402201163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c0107880 msgr2=0x7f41c019f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41be7fc640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c0107880 0x7f41c019f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41be7fc640 1 -- 172.21.15.16:0/1402201163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c019faa0 con 0x7f41c0104f50 2024-09-19T09:29:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41bf7fe640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c0107880 0x7f41c019f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41beffd640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0102620 0x7f41c0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:29:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41be7fc640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c0104f50 0x7f41c01983c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f41b4005fd0 tx=0x7f41b400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41c48aa640 1 -- 172.21.15.16:0/1402201163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4017070 con 0x7f41c0104f50 2024-09-19T09:29:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41c58ac640 1 -- 172.21.15.16:0/1402201163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c019fd90 con 0x7f41c0104f50 2024-09-19T09:29:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41c48aa640 1 -- 172.21.15.16:0/1402201163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b4009c30 con 0x7f41c0104f50 2024-09-19T09:29:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41c48aa640 1 -- 172.21.15.16:0/1402201163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4010460 con 0x7f41c0104f50 2024-09-19T09:29:53.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.875+0000 7f41c58ac640 1 -- 172.21.15.16:0/1402201163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c01a0250 con 0x7f41c0104f50 2024-09-19T09:29:53.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.876+0000 7f41c58ac640 1 -- 172.21.15.16:0/1402201163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c00fdc30 con 0x7f41c0104f50 2024-09-19T09:29:53.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.878+0000 7f41c48aa640 1 -- 172.21.15.16:0/1402201163 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f41b4003e70 con 0x7f41c0104f50 2024-09-19T09:29:53.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.878+0000 7f41c48aa640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41a80a3250 0x7f41a80a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:53.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.878+0000 7f41beffd640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41a80a3250 0x7f41a80a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:53.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.878+0000 7f41c48aa640 1 -- 172.21.15.16:0/1402201163 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f41b40cfd80 con 0x7f41c0104f50 2024-09-19T09:29:53.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.879+0000 7f41beffd640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41a80a3250 0x7f41a80a5710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009b50 tx=0x7f41b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:53.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:53.881+0000 7f41c48aa640 1 -- 172.21.15.16:0/1402201163 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f41b4020030 con 0x7f41c0104f50 2024-09-19T09:29:54.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.062+0000 7f41c58ac640 1 -- 172.21.15.16:0/1402201163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f41c002da30 con 0x7f41c0104f50 2024-09-19T09:29:54.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.062+0000 7f41c48aa640 1 -- 172.21.15.16:0/1402201163 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f41b409caa0 con 0x7f41c0104f50 2024-09-19T09:29:54.064 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-19T09:29:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.064+0000 7f41c58ac640 1 -- 172.21.15.16:0/1402201163 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41a80a3250 msgr2=0x7f41a80a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.064+0000 7f41c58ac640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41a80a3250 0x7f41a80a5710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009b50 tx=0x7f41b0005f90 comp rx=0 tx=0).stop 2024-09-19T09:29:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.064+0000 7f41c58ac640 1 -- 172.21.15.16:0/1402201163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c0104f50 msgr2=0x7f41c01983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.064+0000 7f41c58ac640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c0104f50 0x7f41c01983c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f41b4005fd0 tx=0x7f41b400dde0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.065+0000 7f41c58ac640 1 -- 172.21.15.16:0/1402201163 shutdown_connections 2024-09-19T09:29:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.065+0000 7f41c58ac640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c0107880 0x7f41c019f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.065+0000 7f41c58ac640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c0104f50 0x7f41c01983c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.065+0000 7f41c58ac640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f41a80a3250 0x7f41a80a5710 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.065+0000 7f41c58ac640 1 --2- 172.21.15.16:0/1402201163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41c0102620 0x7f41c0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.065+0000 7f41c58ac640 1 -- 172.21.15.16:0/1402201163 >> 172.21.15.16:0/1402201163 conn(0x7f41c00fc140 msgr2=0x7f41c0103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.065+0000 7f41c58ac640 1 -- 172.21.15.16:0/1402201163 shutdown_connections 2024-09-19T09:29:54.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.065+0000 7f41c58ac640 1 -- 172.21.15.16:0/1402201163 wait complete. 2024-09-19T09:29:54.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:54 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3624433718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:29:54.078 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T09:29:54.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.183+0000 7f47ed48d640 1 -- 172.21.15.16:0/1537808582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8102650 msgr2=0x7f47e8104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.183+0000 7f47ed48d640 1 --2- 172.21.15.16:0/1537808582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8102650 0x7f47e8104a40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009900 tx=0x7f47d802a3f0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.183+0000 7f47ed48d640 1 -- 172.21.15.16:0/1537808582 shutdown_connections 2024-09-19T09:29:54.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.183+0000 7f47ed48d640 1 --2- 172.21.15.16:0/1537808582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47e81078b0 0x7f47e8109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.183+0000 7f47ed48d640 1 --2- 172.21.15.16:0/1537808582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e8104f80 0x7f47e8107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.183+0000 7f47ed48d640 1 --2- 172.21.15.16:0/1537808582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8102650 0x7f47e8104a40 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.183+0000 7f47ed48d640 1 -- 172.21.15.16:0/1537808582 >> 172.21.15.16:0/1537808582 conn(0x7f47e80fc1b0 msgr2=0x7f47e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:54.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.183+0000 7f47ed48d640 1 -- 172.21.15.16:0/1537808582 shutdown_connections 2024-09-19T09:29:54.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.183+0000 7f47ed48d640 1 -- 172.21.15.16:0/1537808582 wait complete. 2024-09-19T09:29:54.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.184+0000 7f47ed48d640 1 Processor -- start 2024-09-19T09:29:54.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.184+0000 7f47ed48d640 1 -- start start 2024-09-19T09:29:54.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.184+0000 7f47ed48d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8102650 0x7f47e8197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:54.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.184+0000 7f47ed48d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47e8104f80 0x7f47e81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:54.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.184+0000 7f47ed48d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e81078b0 0x7f47e819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:54.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.184+0000 7f47ed48d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47e810d7d0 con 0x7f47e8102650 2024-09-19T09:29:54.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.184+0000 7f47ed48d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f47e810d650 con 0x7f47e81078b0 2024-09-19T09:29:54.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.184+0000 7f47ed48d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47e810d950 con 0x7f47e8104f80 2024-09-19T09:29:54.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.184+0000 7f47e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8102650 0x7f47e8197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:54.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.184+0000 7f47e6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8102650 0x7f47e8197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55302/0 (socket says 172.21.15.16:55302) 2024-09-19T09:29:54.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.184+0000 7f47e6ffd640 1 -- 172.21.15.16:0/1213728845 learned_addr learned my addr 172.21.15.16:0/1213728845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:54.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.184+0000 7f47dffff640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47e8104f80 0x7f47e81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.185+0000 7f47e77fe640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e81078b0 0x7f47e819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.185+0000 7f47e6ffd640 1 -- 172.21.15.16:0/1213728845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47e8104f80 msgr2=0x7f47e81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.185+0000 7f47e6ffd640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47e8104f80 0x7f47e81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.185+0000 7f47e6ffd640 1 -- 172.21.15.16:0/1213728845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e81078b0 msgr2=0x7f47e819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.185+0000 7f47e6ffd640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e81078b0 0x7f47e819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.185+0000 7f47e6ffd640 1 -- 172.21.15.16:0/1213728845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e819fad0 con 0x7f47e8102650 2024-09-19T09:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.185+0000 7f47dffff640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47e8104f80 0x7f47e81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.185+0000 7f47e6ffd640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8102650 0x7f47e8197eb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f47d8005940 tx=0x7f47d8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.185+0000 7f47e4ff9640 1 -- 172.21.15.16:0/1213728845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d803f030 con 0x7f47e8102650 2024-09-19T09:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.185+0000 7f47e4ff9640 1 -- 172.21.15.16:0/1213728845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d8003ce0 con 0x7f47e8102650 2024-09-19T09:29:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.185+0000 7f47e4ff9640 1 -- 172.21.15.16:0/1213728845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8035070 con 0x7f47e8102650 2024-09-19T09:29:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.185+0000 7f47ed48d640 1 -- 172.21.15.16:0/1213728845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e819fd60 con 0x7f47e8102650 2024-09-19T09:29:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.185+0000 7f47ed48d640 1 -- 172.21.15.16:0/1213728845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e81a01d0 con 0x7f47e8102650 2024-09-19T09:29:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.186+0000 7f47ed48d640 1 -- 172.21.15.16:0/1213728845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e80fdca0 con 0x7f47e8102650 2024-09-19T09:29:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.187+0000 7f47e4ff9640 1 -- 172.21.15.16:0/1213728845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f47d8042030 con 0x7f47e8102650 2024-09-19T09:29:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.188+0000 7f47e4ff9640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f47d40a32d0 0x7f47d40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.188+0000 7f47dffff640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f47d40a32d0 0x7f47d40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.188+0000 7f47e4ff9640 1 -- 172.21.15.16:0/1213728845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f47d80f05a0 con 0x7f47e8102650 2024-09-19T09:29:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.189+0000 7f47dffff640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f47d40a32d0 0x7f47d40a5790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f47e81993c0 tx=0x7f47d000a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:54.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.192+0000 7f47e4ff9640 1 -- 172.21.15.16:0/1213728845 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f47d80bd280 con 0x7f47e8102650 2024-09-19T09:29:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.369+0000 7f47ed48d640 1 -- 172.21.15.16:0/1213728845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f47e802da50 con 0x7f47e8102650 2024-09-19T09:29:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.369+0000 7f47e4ff9640 1 -- 172.21.15.16:0/1213728845 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7f47d80bcb80 con 0x7f47e8102650 2024-09-19T09:29:54.371 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-19T09:29:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.371+0000 7f47ed48d640 1 -- 172.21.15.16:0/1213728845 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f47d40a32d0 msgr2=0x7f47d40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.371+0000 7f47ed48d640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f47d40a32d0 0x7f47d40a5790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f47e81993c0 tx=0x7f47d000a5c0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.372+0000 7f47ed48d640 1 -- 172.21.15.16:0/1213728845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8102650 msgr2=0x7f47e8197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.372+0000 7f47ed48d640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8102650 0x7f47e8197eb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f47d8005940 tx=0x7f47d8004360 comp rx=0 tx=0).stop 2024-09-19T09:29:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.372+0000 7f47ed48d640 1 -- 172.21.15.16:0/1213728845 shutdown_connections 2024-09-19T09:29:54.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.372+0000 7f47ed48d640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f47d40a32d0 0x7f47d40a5790 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.372+0000 7f47ed48d640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47e8104f80 0x7f47e81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.372+0000 7f47ed48d640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47e81078b0 0x7f47e819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.372+0000 7f47ed48d640 1 --2- 172.21.15.16:0/1213728845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8102650 0x7f47e8197eb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.372+0000 7f47ed48d640 1 -- 172.21.15.16:0/1213728845 >> 172.21.15.16:0/1213728845 conn(0x7f47e80fc1b0 msgr2=0x7f47e8103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.372+0000 7f47ed48d640 1 -- 172.21.15.16:0/1213728845 shutdown_connections 2024-09-19T09:29:54.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.372+0000 7f47ed48d640 1 -- 172.21.15.16:0/1213728845 wait complete. 2024-09-19T09:29:54.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:54 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3624433718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:29:54.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T09:29:54.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:54 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3624433718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:29:54.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.491+0000 7fcc5306e640 1 -- 172.21.15.16:0/3459439101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c1025e0 msgr2=0x7fcc4c108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.491+0000 7fcc5306e640 1 --2- 172.21.15.16:0/3459439101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c1025e0 0x7fcc4c108e70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009ca0 tx=0x7fcc4402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.491+0000 7fcc5306e640 1 -- 172.21.15.16:0/3459439101 shutdown_connections 2024-09-19T09:29:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.491+0000 7fcc5306e640 1 --2- 172.21.15.16:0/3459439101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c101c20 0x7fcc4c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.491+0000 7fcc5306e640 1 --2- 172.21.15.16:0/3459439101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c1025e0 0x7fcc4c108e70 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.491+0000 7fcc5306e640 1 --2- 172.21.15.16:0/3459439101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4c100a30 0x7fcc4c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.491+0000 7fcc5306e640 1 -- 172.21.15.16:0/3459439101 >> 172.21.15.16:0/3459439101 conn(0x7fcc4c0fc1b0 msgr2=0x7fcc4c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.491+0000 7fcc5306e640 1 -- 172.21.15.16:0/3459439101 shutdown_connections 2024-09-19T09:29:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.491+0000 7fcc5306e640 1 -- 172.21.15.16:0/3459439101 wait complete. 2024-09-19T09:29:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.492+0000 7fcc5306e640 1 Processor -- start 2024-09-19T09:29:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.492+0000 7fcc5306e640 1 -- start start 2024-09-19T09:29:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.492+0000 7fcc5306e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4c100a30 0x7fcc4c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.492+0000 7fcc5306e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c101c20 0x7fcc4c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.492+0000 7fcc5306e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c1025e0 0x7fcc4c19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.492+0000 7fcc5306e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c10c690 con 0x7fcc4c1025e0 2024-09-19T09:29:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.492+0000 7fcc5306e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c10c510 con 0x7fcc4c100a30 2024-09-19T09:29:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.492+0000 7fcc5306e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c10c810 con 0x7fcc4c101c20 2024-09-19T09:29:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.492+0000 7fcc5160b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c1025e0 0x7fcc4c19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.492+0000 7fcc5160b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c1025e0 0x7fcc4c19f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55322/0 (socket says 172.21.15.16:55322) 2024-09-19T09:29:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.492+0000 7fcc5160b640 1 -- 172.21.15.16:0/2557913436 learned_addr learned my addr 172.21.15.16:0/2557913436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.492+0000 7fcc4bfff640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c101c20 0x7fcc4c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.492+0000 7fcc50e0a640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4c100a30 0x7fcc4c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.493+0000 7fcc5160b640 1 -- 172.21.15.16:0/2557913436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c101c20 msgr2=0x7fcc4c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.493+0000 7fcc5160b640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c101c20 0x7fcc4c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.493+0000 7fcc5160b640 1 -- 172.21.15.16:0/2557913436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4c100a30 msgr2=0x7fcc4c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.493+0000 7fcc5160b640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4c100a30 0x7fcc4c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.493+0000 7fcc5160b640 1 -- 172.21.15.16:0/2557913436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c19fac0 con 0x7fcc4c1025e0 2024-09-19T09:29:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.493+0000 7fcc5160b640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c1025e0 0x7fcc4c19f450 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcc44031ee0 tx=0x7fcc44031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.493+0000 7fcc49ffb640 1 -- 172.21.15.16:0/2557913436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44003c30 con 0x7fcc4c1025e0 2024-09-19T09:29:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.493+0000 7fcc49ffb640 1 -- 172.21.15.16:0/2557913436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc44003dd0 con 0x7fcc4c1025e0 2024-09-19T09:29:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.493+0000 7fcc5306e640 1 -- 172.21.15.16:0/2557913436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4c19fd50 con 0x7fcc4c1025e0 2024-09-19T09:29:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.493+0000 7fcc49ffb640 1 -- 172.21.15.16:0/2557913436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4402fc10 con 0x7fcc4c1025e0 2024-09-19T09:29:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.493+0000 7fcc5306e640 1 -- 172.21.15.16:0/2557913436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4c1a0260 con 0x7fcc4c1025e0 2024-09-19T09:29:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.494+0000 7fcc5306e640 1 -- 172.21.15.16:0/2557913436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc4c1a04c0 con 0x7fcc4c1025e0 2024-09-19T09:29:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.495+0000 7fcc49ffb640 1 -- 172.21.15.16:0/2557913436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fcc440046c0 con 0x7fcc4c1025e0 2024-09-19T09:29:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.496+0000 7fcc49ffb640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fcc380a32d0 0x7fcc380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.497+0000 7fcc50e0a640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fcc380a32d0 0x7fcc380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.497+0000 7fcc49ffb640 1 -- 172.21.15.16:0/2557913436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fcc440f0b30 con 0x7fcc4c1025e0 2024-09-19T09:29:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.497+0000 7fcc50e0a640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fcc380a32d0 0x7fcc380a5790 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c005fd0 tx=0x7fcc3c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:54.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.501+0000 7fcc49ffb640 1 -- 172.21.15.16:0/2557913436 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcc440bd810 con 0x7fcc4c1025e0 2024-09-19T09:29:54.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.677+0000 7fcc5306e640 1 -- 172.21.15.16:0/2557913436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcc4c1a0820 con 0x7fcc4c1025e0 2024-09-19T09:29:54.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.677+0000 7fcc49ffb640 1 -- 172.21.15.16:0/2557913436 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7fcc440bd110 con 0x7fcc4c1025e0 2024-09-19T09:29:54.678 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-19T09:29:54.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.679+0000 7fcc5306e640 1 -- 172.21.15.16:0/2557913436 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fcc380a32d0 msgr2=0x7fcc380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.679+0000 7fcc5306e640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fcc380a32d0 0x7fcc380a5790 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c005fd0 tx=0x7fcc3c00b410 comp rx=0 tx=0).stop 2024-09-19T09:29:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.679+0000 7fcc5306e640 1 -- 172.21.15.16:0/2557913436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c1025e0 msgr2=0x7fcc4c19f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.679+0000 7fcc5306e640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c1025e0 0x7fcc4c19f450 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcc44031ee0 tx=0x7fcc44031f10 comp rx=0 tx=0).stop 2024-09-19T09:29:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.679+0000 7fcc5306e640 1 -- 172.21.15.16:0/2557913436 shutdown_connections 2024-09-19T09:29:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.679+0000 7fcc5306e640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fcc380a32d0 0x7fcc380a5790 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.679+0000 7fcc5306e640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c101c20 0x7fcc4c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.679+0000 7fcc5306e640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c1025e0 0x7fcc4c19f450 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.679+0000 7fcc5306e640 1 --2- 172.21.15.16:0/2557913436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc4c100a30 0x7fcc4c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.679+0000 7fcc5306e640 1 -- 172.21.15.16:0/2557913436 >> 172.21.15.16:0/2557913436 conn(0x7fcc4c0fc1b0 msgr2=0x7fcc4c06a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.679+0000 7fcc5306e640 1 -- 172.21.15.16:0/2557913436 shutdown_connections 2024-09-19T09:29:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.679+0000 7fcc5306e640 1 -- 172.21.15.16:0/2557913436 wait complete. 2024-09-19T09:29:54.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T09:29:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.795+0000 7f422caba640 1 -- 172.21.15.16:0/1871561955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4228101c00 msgr2=0x7f4228102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.795+0000 7f422caba640 1 --2- 172.21.15.16:0/1871561955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4228101c00 0x7f4228102080 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4210009900 tx=0x7f421002a410 comp rx=0 tx=0).stop 2024-09-19T09:29:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.796+0000 7f422caba640 1 -- 172.21.15.16:0/1871561955 shutdown_connections 2024-09-19T09:29:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.796+0000 7f422caba640 1 --2- 172.21.15.16:0/1871561955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42281025c0 0x7f4228108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.796+0000 7f422caba640 1 --2- 172.21.15.16:0/1871561955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4228101c00 0x7f4228102080 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.796+0000 7f422caba640 1 --2- 172.21.15.16:0/1871561955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4228100a10 0x7f4228100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.796+0000 7f422caba640 1 -- 172.21.15.16:0/1871561955 >> 172.21.15.16:0/1871561955 conn(0x7f42280fc170 msgr2=0x7f42280fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.796+0000 7f422caba640 1 -- 172.21.15.16:0/1871561955 shutdown_connections 2024-09-19T09:29:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.796+0000 7f422caba640 1 -- 172.21.15.16:0/1871561955 wait complete. 2024-09-19T09:29:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.796+0000 7f422caba640 1 Processor -- start 2024-09-19T09:29:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.796+0000 7f422caba640 1 -- start start 2024-09-19T09:29:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422caba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4228100a10 0x7f42281980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422caba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4228101c00 0x7f42281985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422caba640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42281025c0 0x7f422819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422caba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f422810c7e0 con 0x7f4228100a10 2024-09-19T09:29:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422caba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f422810c660 con 0x7f4228101c00 2024-09-19T09:29:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422caba640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f422810c960 con 0x7f42281025c0 2024-09-19T09:29:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4228100a10 0x7f42281980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4228100a10 0x7f42281980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55340/0 (socket says 172.21.15.16:55340) 2024-09-19T09:29:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422659c640 1 -- 172.21.15.16:0/414189664 learned_addr learned my addr 172.21.15.16:0/414189664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:29:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f4225d9b640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4228101c00 0x7f42281985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f4226d9d640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42281025c0 0x7f422819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422659c640 1 -- 172.21.15.16:0/414189664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42281025c0 msgr2=0x7f422819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422659c640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42281025c0 0x7f422819f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422659c640 1 -- 172.21.15.16:0/414189664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4228101c00 msgr2=0x7f42281985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422659c640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4228101c00 0x7f42281985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422659c640 1 -- 172.21.15.16:0/414189664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422819fcc0 con 0x7f4228100a10 2024-09-19T09:29:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422659c640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4228100a10 0x7f42281980a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f421800b450 tx=0x7f421800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f421f7fe640 1 -- 172.21.15.16:0/414189664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218015020 con 0x7f4228100a10 2024-09-19T09:29:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f421f7fe640 1 -- 172.21.15.16:0/414189664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4218011d70 con 0x7f4228100a10 2024-09-19T09:29:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f422caba640 1 -- 172.21.15.16:0/414189664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422819ffb0 con 0x7f4228100a10 2024-09-19T09:29:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.797+0000 7f421f7fe640 1 -- 172.21.15.16:0/414189664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421800f340 con 0x7f4228100a10 2024-09-19T09:29:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.798+0000 7f422caba640 1 -- 172.21.15.16:0/414189664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42281a04a0 con 0x7f4228100a10 2024-09-19T09:29:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.798+0000 7f422caba640 1 -- 172.21.15.16:0/414189664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42281a0730 con 0x7f4228100a10 2024-09-19T09:29:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.800+0000 7f421f7fe640 1 -- 172.21.15.16:0/414189664 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4218018020 con 0x7f4228100a10 2024-09-19T09:29:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.800+0000 7f421f7fe640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f42140a32d0 0x7f42140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:29:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.800+0000 7f4225d9b640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f42140a32d0 0x7f42140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:29:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.800+0000 7f421f7fe640 1 -- 172.21.15.16:0/414189664 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f42180cf2f0 con 0x7f4228100a10 2024-09-19T09:29:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.801+0000 7f4225d9b640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f42140a32d0 0x7f42140a5790 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4210005ab0 tx=0x7f42100315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:29:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:54.804+0000 7f421f7fe640 1 -- 172.21.15.16:0/414189664 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f421801f020 con 0x7f4228100a10 2024-09-19T09:29:55.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.030+0000 7f422caba640 1 -- 172.21.15.16:0/414189664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f42281a0900 con 0x7f4228100a10 2024-09-19T09:29:55.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.038+0000 7f421f7fe640 1 -- 172.21.15.16:0/414189664 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+742340 (secure 0 0 0) 0x7f421800d040 con 0x7f4228100a10 2024-09-19T09:29:55.049 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:29:55.049 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:29:55.049 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 18, 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: "active_gid": 14150, 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: "active_name": "x", 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addrs": { 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6800", 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4287557979 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6801", 2024-09-19T09:29:55.050 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4287557979 2024-09-19T09:29:55.051 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.051 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T09:29:55.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.051 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addr": "172.21.15.16:6801/4287557979", 2024-09-19T09:29:55.051 INFO:teuthology.orchestra.run.smithi016.stdout: "active_change": "2024-09-19T09:18:41.244183+0000", 2024-09-19T09:29:55.051 INFO:teuthology.orchestra.run.smithi016.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T09:29:55.051 INFO:teuthology.orchestra.run.smithi016.stdout: "available": true, 2024-09-19T09:29:55.051 INFO:teuthology.orchestra.run.smithi016.stdout: "standbys": [ 2024-09-19T09:29:55.051 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.051 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24109, 2024-09-19T09:29:55.051 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "z", 2024-09-19T09:29:55.051 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-09-19T09:29:55.051 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-19T09:29:55.052 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-19T09:29:55.052 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.052 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.052 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.052 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:55.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:55.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.052 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.052 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:55.052 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.053 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.053 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T09:29:55.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.053 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.053 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.053 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.053 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.053 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.053 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.053 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.053 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.054 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.054 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.054 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.054 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.054 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.054 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.054 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.054 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.054 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.054 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.054 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.055 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.055 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.055 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.055 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.055 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.055 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.055 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.056 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.056 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.056 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.056 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.056 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.056 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.056 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.056 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.056 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.056 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.056 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-19T09:29:55.058 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.059 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.059 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.059 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.059 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.059 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.059 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-19T09:29:55.059 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.059 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.059 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.059 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-19T09:29:55.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-19T09:29:55.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.060 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.060 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.060 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-19T09:29:55.060 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.060 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-19T09:29:55.060 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.060 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.060 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.060 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-19T09:29:55.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-19T09:29:55.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.061 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.061 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.061 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-19T09:29:55.061 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.061 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.061 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.062 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-19T09:29:55.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-19T09:29:55.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.062 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.062 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.062 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.062 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.062 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.062 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-19T09:29:55.062 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.063 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-19T09:29:55.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-19T09:29:55.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.063 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.063 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.063 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-19T09:29:55.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.063 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.063 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-19T09:29:55.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.064 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.064 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-19T09:29:55.064 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-19T09:29:55.064 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.064 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.064 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.064 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.064 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.064 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.064 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-19T09:29:55.064 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.065 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.065 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.065 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-19T09:29:55.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-19T09:29:55.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.066 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.066 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.066 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.066 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T09:29:55.066 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.066 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.066 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-19T09:29:55.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-19T09:29:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-19T09:29:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.068 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.068 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-19T09:29:55.068 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.068 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.068 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.068 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-19T09:29:55.068 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-19T09:29:55.068 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.068 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.069 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.069 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.069 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T09:29:55.069 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.069 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.069 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.069 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-19T09:29:55.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-19T09:29:55.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.069 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.069 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.070 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.071 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:55.071 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-19T09:29:55.071 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-19T09:29:55.071 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.071 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T09:29:55.071 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:29:55.071 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.071 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.071 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-19T09:29:55.071 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-19T09:29:55.071 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:55.071 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-19T09:29:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-19T09:29:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-19T09:29:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T09:29:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-19T09:29:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-19T09:29:55.072 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.073 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T09:29:55.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.073 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.073 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.073 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T09:29:55.073 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T09:29:55.073 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.073 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.073 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.074 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-19T09:29:55.075 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-19T09:29:55.075 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.075 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.075 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.075 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-19T09:29:55.075 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T09:29:55.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:29:55.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.075 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.078 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.078 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.078 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.078 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.078 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.078 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.078 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.078 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.078 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.078 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.079 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.079 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.079 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.079 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.079 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.081 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.082 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.082 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.082 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-19T09:29:55.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-19T09:29:55.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:55.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.082 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:55.082 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.082 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.083 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T09:29:55.083 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.083 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.083 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.083 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-19T09:29:55.083 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-19T09:29:55.083 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.083 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-19T09:29:55.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.084 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.084 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-19T09:29:55.084 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-19T09:29:55.084 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-19T09:29:55.084 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-19T09:29:55.084 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-19T09:29:55.084 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.084 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-19T09:29:55.084 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.084 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.085 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.086 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.087 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-19T09:29:55.087 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-19T09:29:55.087 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.087 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:55.087 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-19T09:29:55.087 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.087 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.087 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T09:29:55.087 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T09:29:55.087 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.088 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.088 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-19T09:29:55.088 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-19T09:29:55.088 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:55.088 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.088 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.088 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.088 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T09:29:55.089 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.089 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.089 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-19T09:29:55.089 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.089 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.089 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.089 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-19T09:29:55.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-19T09:29:55.090 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:55.090 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.090 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.090 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-19T09:29:55.090 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.090 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.090 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.090 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T09:29:55.090 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.090 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.090 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.091 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-19T09:29:55.091 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-19T09:29:55.091 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.091 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.091 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-19T09:29:55.091 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.091 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.091 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T09:29:55.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.091 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.092 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-19T09:29:55.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-19T09:29:55.092 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.092 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.092 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-19T09:29:55.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.092 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.092 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T09:29:55.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.093 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-19T09:29:55.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-19T09:29:55.093 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.093 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.094 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.094 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.094 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.094 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.094 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T09:29:55.094 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T09:29:55.094 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.094 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.094 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-19T09:29:55.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-19T09:29:55.095 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.095 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.095 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:55.095 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.095 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.095 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T09:29:55.095 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.095 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.095 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-19T09:29:55.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T09:29:55.096 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:55.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.096 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.096 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-19T09:29:55.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.096 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T09:29:55.096 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.096 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.096 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.097 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-19T09:29:55.097 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-19T09:29:55.097 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.097 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.100 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-19T09:29:55.100 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-19T09:29:55.101 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-19T09:29:55.101 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T09:29:55.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.101 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.101 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-19T09:29:55.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-19T09:29:55.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.102 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T09:29:55.102 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.102 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.102 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.102 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-19T09:29:55.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-19T09:29:55.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.104 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.104 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.104 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.104 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.104 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.104 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T09:29:55.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.104 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.104 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.104 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-19T09:29:55.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-19T09:29:55.104 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T09:29:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-19T09:29:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-19T09:29:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-19T09:29:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T09:29:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-19T09:29:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.106 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-19T09:29:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-19T09:29:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T09:29:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-19T09:29:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.108 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-19T09:29:55.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-19T09:29:55.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.108 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T09:29:55.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.108 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-19T09:29:55.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.109 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-19T09:29:55.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-19T09:29:55.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.109 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T09:29:55.109 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.109 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.109 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-19T09:29:55.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.112 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.112 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.112 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-19T09:29:55.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-19T09:29:55.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T09:29:55.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.113 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.113 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-19T09:29:55.113 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.113 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.113 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-19T09:29:55.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-19T09:29:55.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.114 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.114 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T09:29:55.114 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.114 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.114 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-19T09:29:55.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.114 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-19T09:29:55.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-19T09:29:55.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.115 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.115 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.115 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T09:29:55.115 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-19T09:29:55.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.115 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-19T09:29:55.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-19T09:29:55.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.116 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T09:29:55.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-19T09:29:55.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.117 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-19T09:29:55.117 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-19T09:29:55.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T09:29:55.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-19T09:29:55.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.118 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.118 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-19T09:29:55.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-19T09:29:55.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T09:29:55.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.118 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.118 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-19T09:29:55.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.119 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-19T09:29:55.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-19T09:29:55.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.119 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T09:29:55.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.119 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.119 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-19T09:29:55.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.120 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-19T09:29:55.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-19T09:29:55.120 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.120 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.120 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.120 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T09:29:55.120 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.120 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.121 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.122 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-19T09:29:55.122 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.122 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.122 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-19T09:29:55.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-19T09:29:55.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.122 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:55.123 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.124 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T09:29:55.124 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.124 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.124 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-19T09:29:55.124 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T09:29:55.124 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.124 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.124 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-19T09:29:55.127 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.127 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.127 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-19T09:29:55.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.127 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-19T09:29:55.127 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-19T09:29:55.127 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.127 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.127 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T09:29:55.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.128 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-19T09:29:55.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-19T09:29:55.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:55.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.129 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-19T09:29:55.129 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.129 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.129 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.129 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-19T09:29:55.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-19T09:29:55.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T09:29:55.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.130 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.130 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T09:29:55.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.130 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-19T09:29:55.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-19T09:29:55.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-19T09:29:55.132 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.132 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.132 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.132 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.132 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.132 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.132 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T09:29:55.132 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.132 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.132 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.132 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-19T09:29:55.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-19T09:29:55.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.133 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T09:29:55.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.135 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.136 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T09:29:55.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T09:29:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.138 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.138 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.138 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.138 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.138 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.138 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.138 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.138 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.138 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.138 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.139 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.140 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.140 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.140 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.140 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.140 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.140 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.140 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T09:29:55.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T09:29:55.141 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.141 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.141 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T09:29:55.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.141 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-19T09:29:55.144 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-19T09:29:55.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.144 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.144 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.144 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T09:29:55.144 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.144 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.144 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.146 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-19T09:29:55.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-19T09:29:55.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.146 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.146 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-19T09:29:55.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.146 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.146 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.146 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-19T09:29:55.146 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-19T09:29:55.146 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.146 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T09:29:55.147 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.147 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.147 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.147 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-19T09:29:55.147 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-19T09:29:55.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.147 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-19T09:29:55.147 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.148 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T09:29:55.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.148 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.148 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-19T09:29:55.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-19T09:29:55.148 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.148 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.148 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T09:29:55.149 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.149 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T09:29:55.149 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.149 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.149 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.149 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-19T09:29:55.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-19T09:29:55.149 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-19T09:29:55.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.151 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.151 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.151 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.151 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T09:29:55.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.151 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-19T09:29:55.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-19T09:29:55.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T09:29:55.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T09:29:55.153 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-19T09:29:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-19T09:29:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T09:29:55.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.155 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-19T09:29:55.155 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-19T09:29:55.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-19T09:29:55.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.156 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-19T09:29:55.156 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.156 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.156 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-19T09:29:55.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-19T09:29:55.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.156 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.156 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.156 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.156 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.157 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T09:29:55.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.158 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.158 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-19T09:29:55.158 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-19T09:29:55.158 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.159 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.159 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T09:29:55.159 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.159 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.159 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.159 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-19T09:29:55.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-19T09:29:55.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T09:29:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-19T09:29:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-19T09:29:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.161 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.161 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T09:29:55.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.161 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.161 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.161 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-19T09:29:55.161 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-19T09:29:55.161 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T09:29:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.163 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-19T09:29:55.163 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.163 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.163 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.163 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.164 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.164 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.164 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.164 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.164 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.164 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.164 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.165 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.167 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.167 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.167 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.167 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.167 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.167 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.167 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.168 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.168 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.168 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.168 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.168 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.168 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.168 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.169 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.169 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.169 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.169 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.170 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-19T09:29:55.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-19T09:29:55.170 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.170 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.170 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-19T09:29:55.170 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.170 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T09:29:55.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.171 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.172 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.172 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.172 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.172 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-19T09:29:55.172 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.172 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.172 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.172 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T09:29:55.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T09:29:55.172 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.172 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.173 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.173 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.173 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.173 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T09:29:55.173 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T09:29:55.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T09:29:55.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-19T09:29:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T09:29:55.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T09:29:55.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.177 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.177 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.177 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.177 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.177 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.177 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T09:29:55.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T09:29:55.178 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.178 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.178 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.178 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.178 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.178 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.178 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.178 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.178 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.178 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.178 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.179 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T09:29:55.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T09:29:55.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.179 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.179 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.179 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.179 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.179 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.179 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.179 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.179 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.179 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.181 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T09:29:55.181 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T09:29:55.181 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.181 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.181 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.181 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.181 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.181 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.181 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.181 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.182 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T09:29:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T09:29:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.184 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.184 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.184 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.184 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T09:29:55.184 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T09:29:55.184 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.184 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.184 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.184 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.184 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.184 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.184 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.185 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.187 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.187 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.187 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.187 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T09:29:55.187 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T09:29:55.187 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.187 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.187 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.187 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-19T09:29:55.187 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.188 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.188 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.188 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.188 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.188 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.188 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.188 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T09:29:55.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T09:29:55.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.188 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.189 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.190 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.190 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.190 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.190 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.190 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.190 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.190 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.190 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.190 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T09:29:55.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T09:29:55.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.191 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.191 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.191 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-19T09:29:55.191 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.191 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.191 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.191 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.191 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.191 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.191 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.191 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T09:29:55.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T09:29:55.192 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.192 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.192 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.192 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.192 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.192 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.192 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.192 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.192 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.192 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T09:29:55.193 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T09:29:55.193 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.193 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.193 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.193 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.194 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.194 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T09:29:55.194 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T09:29:55.194 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.194 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.194 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.194 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.194 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.194 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.194 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.195 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.195 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.195 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T09:29:55.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T09:29:55.195 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.195 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.195 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.195 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.195 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.195 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.195 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.196 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.197 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.197 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.197 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.197 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T09:29:55.197 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T09:29:55.197 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.197 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.197 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.197 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.197 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.197 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.198 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.198 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.198 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.198 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.198 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T09:29:55.198 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T09:29:55.198 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.198 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.198 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.198 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.199 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.200 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.201 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.203 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.203 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.203 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.203 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.203 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.203 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.203 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.203 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T09:29:55.203 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T09:29:55.203 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.203 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T09:29:55.204 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T09:29:55.205 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T09:29:55.206 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.206 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.206 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.206 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.206 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.206 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.206 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.206 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.207 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T09:29:55.207 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T09:29:55.207 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.207 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.207 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.207 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T09:29:55.207 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.207 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.207 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.207 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.207 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.208 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.208 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T09:29:55.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T09:29:55.208 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.208 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.208 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.208 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.208 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.208 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.208 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.209 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.210 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.210 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.210 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T09:29:55.210 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T09:29:55.210 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.210 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.210 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.210 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-19T09:29:55.210 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.210 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.210 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.211 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.211 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.211 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.211 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.211 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T09:29:55.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T09:29:55.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.211 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.211 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.211 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.212 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.212 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.212 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.212 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.212 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.212 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T09:29:55.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T09:29:55.212 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.212 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.212 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.212 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-19T09:29:55.212 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.214 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:55.215 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T09:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T09:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.217 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:55.218 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.218 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.218 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.218 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.218 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.218 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.218 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-19T09:29:55.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-19T09:29:55.218 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.218 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.218 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.219 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.220 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.220 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.220 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.220 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.220 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.220 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.220 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.220 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.220 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-19T09:29:55.220 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-19T09:29:55.221 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.221 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.221 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.221 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.221 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.221 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.221 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.221 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-19T09:29:55.221 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.221 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.221 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-19T09:29:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-19T09:29:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-19T09:29:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.222 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.223 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-19T09:29:55.223 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-19T09:29:55.223 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.223 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.223 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.223 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.223 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.223 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.223 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.224 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.224 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.224 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.224 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.224 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.224 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.224 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.224 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.224 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.224 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.225 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.226 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.226 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.226 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.226 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.226 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.226 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.226 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.226 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.226 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.228 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.229 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.230 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.231 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.232 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.232 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.232 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.232 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.232 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-19T09:29:55.232 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-19T09:29:55.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.232 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.232 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.232 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-19T09:29:55.232 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.232 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.233 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.234 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.235 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.235 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.235 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.235 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-19T09:29:55.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-19T09:29:55.235 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.235 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.235 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-19T09:29:55.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-19T09:29:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-19T09:29:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-19T09:29:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-19T09:29:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.237 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-19T09:29:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-19T09:29:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.238 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-19T09:29:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-19T09:29:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-19T09:29:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-19T09:29:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.241 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.242 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-19T09:29:55.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-19T09:29:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-19T09:29:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T09:29:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.246 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-19T09:29:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-19T09:29:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-19T09:29:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T09:29:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.247 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-19T09:29:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-19T09:29:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-19T09:29:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-19T09:29:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-19T09:29:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-19T09:29:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-19T09:29:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T09:29:55.252 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.252 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.252 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.252 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.252 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-19T09:29:55.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-19T09:29:55.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-19T09:29:55.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T09:29:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-19T09:29:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.254 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.256 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.256 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.259 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.259 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.259 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.259 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.259 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-19T09:29:55.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-19T09:29:55.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.259 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-19T09:29:55.259 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.259 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.260 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.260 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.260 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.260 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.260 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.260 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-19T09:29:55.260 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-19T09:29:55.260 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.260 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.260 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.260 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-19T09:29:55.260 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.261 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.261 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:55.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:55.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-19T09:29:55.262 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-19T09:29:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-19T09:29:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-19T09:29:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-19T09:29:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-19T09:29:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T09:29:55.263 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.264 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.264 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.264 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.264 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-19T09:29:55.264 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-19T09:29:55.264 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.264 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.264 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.264 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-19T09:29:55.264 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.264 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.265 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T09:29:55.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.265 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.265 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-19T09:29:55.265 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-19T09:29:55.265 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.266 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.266 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-19T09:29:55.266 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.266 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.266 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.266 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.266 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:55.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:55.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-19T09:29:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-19T09:29:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T09:29:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.269 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.269 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.269 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.270 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-19T09:29:55.272 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-19T09:29:55.273 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.273 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-19T09:29:55.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.273 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.274 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-19T09:29:55.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-19T09:29:55.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.276 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.276 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.276 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:55.276 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-19T09:29:55.276 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-19T09:29:55.276 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T09:29:55.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.278 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-19T09:29:55.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-19T09:29:55.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-19T09:29:55.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.278 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:29:55.278 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-19T09:29:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.280 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.280 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.280 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.280 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.280 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.280 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.280 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.280 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.280 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.280 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.280 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.281 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.281 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.281 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.283 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.283 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.283 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.283 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.283 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.283 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.283 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.284 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.284 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.284 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.285 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.285 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.285 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-19T09:29:55.285 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.285 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.285 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.285 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.288 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.289 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.289 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.289 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.289 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.289 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.289 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.289 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.289 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.289 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.290 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.291 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.291 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-19T09:29:55.291 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.291 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.291 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.291 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-19T09:29:55.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-19T09:29:55.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.291 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-19T09:29:55.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.293 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T09:29:55.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.293 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.294 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.294 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.294 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.294 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.294 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.294 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.294 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.295 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.295 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.295 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.295 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.295 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.298 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.299 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-19T09:29:55.299 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.299 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.299 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.299 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-19T09:29:55.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-19T09:29:55.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-19T09:29:55.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.300 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-19T09:29:55.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T09:29:55.300 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.300 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.303 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.303 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.303 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.303 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.303 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.304 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-19T09:29:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-19T09:29:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-19T09:29:55.306 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T09:29:55.307 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.307 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-19T09:29:55.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-19T09:29:55.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.307 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-19T09:29:55.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.307 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-19T09:29:55.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.308 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-19T09:29:55.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-19T09:29:55.308 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.308 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-19T09:29:55.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T09:29:55.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.309 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-19T09:29:55.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-19T09:29:55.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-19T09:29:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-19T09:29:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-19T09:29:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.310 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-19T09:29:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T09:29:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T09:29:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.311 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-19T09:29:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.312 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.313 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.315 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.316 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-19T09:29:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.318 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.321 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-19T09:29:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.324 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.326 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.327 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.329 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-19T09:29:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-19T09:29:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-19T09:29:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T09:29:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.332 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.335 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-19T09:29:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-19T09:29:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-19T09:29:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-19T09:29:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-19T09:29:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-19T09:29:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.338 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-19T09:29:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.339 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.341 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.342 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.344 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-19T09:29:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.346 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.348 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.349 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.350 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-19T09:29:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-19T09:29:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-19T09:29:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-19T09:29:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-19T09:29:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-19T09:29:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T09:29:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-19T09:29:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-19T09:29:55.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-19T09:29:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T09:29:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-19T09:29:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-19T09:29:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.362 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.365 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.367 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.367 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-19T09:29:55.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-19T09:29:55.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.367 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-19T09:29:55.367 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-19T09:29:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:55.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-19T09:29:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-19T09:29:55.369 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-19T09:29:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-19T09:29:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-19T09:29:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-19T09:29:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T09:29:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T09:29:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.372 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-19T09:29:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-19T09:29:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T09:29:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T09:29:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-19T09:29:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-19T09:29:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-19T09:29:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-19T09:29:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-19T09:29:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-19T09:29:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-19T09:29:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T09:29:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-19T09:29:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-19T09:29:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-19T09:29:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-19T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-19T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-19T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-19T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-19T09:29:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-19T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-19T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-19T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-19T09:29:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-19T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-19T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-19T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.394 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-19T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-19T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.396 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-19T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-19T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-19T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-19T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.399 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-19T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-19T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-19T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-19T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.411 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-19T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.415 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-19T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-19T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-19T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-19T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.421 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.422 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.424 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.425 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.426 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-19T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-19T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-19T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-19T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-19T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-19T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-19T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.429 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-19T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-19T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-19T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-19T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.431 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-19T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-19T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.432 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-19T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-19T09:29:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-19T09:29:55.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-19T09:29:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-19T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-19T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-19T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-19T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-19T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-19T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-19T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.439 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-19T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-19T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-19T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.440 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-19T09:29:55.441 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-19T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-19T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.442 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.443 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.444 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.446 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-19T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.449 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.452 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-19T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.454 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.455 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.457 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.458 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-19T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.460 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-19T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-19T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-19T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.466 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-19T09:29:55.468 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-19T09:29:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-19T09:29:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T09:29:55.469 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-19T09:29:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-19T09:29:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T09:29:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-19T09:29:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-19T09:29:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.471 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T09:29:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-19T09:29:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-19T09:29:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.472 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T09:29:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-19T09:29:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-19T09:29:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T09:29:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-19T09:29:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-19T09:29:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-19T09:29:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-19T09:29:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:55.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-19T09:29:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-19T09:29:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-19T09:29:55.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-19T09:29:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-19T09:29:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.480 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-19T09:29:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-19T09:29:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-19T09:29:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-19T09:29:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.483 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.484 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-19T09:29:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-19T09:29:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-19T09:29:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-19T09:29:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-19T09:29:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-19T09:29:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T09:29:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-19T09:29:55.491 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.492 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.494 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.495 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-19T09:29:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.497 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.498 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T09:29:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-19T09:29:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-19T09:29:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.504 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-19T09:29:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-19T09:29:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-19T09:29:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T09:29:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-19T09:29:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T09:29:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T09:29:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.507 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-19T09:29:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-19T09:29:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-19T09:29:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-19T09:29:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-19T09:29:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-19T09:29:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:55.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.514 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.516 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-19T09:29:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-19T09:29:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-19T09:29:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.519 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T09:29:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24116, 2024-09-19T09:29:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "y", 2024-09-19T09:29:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-09-19T09:29:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-19T09:29:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-19T09:29:55.520 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T09:29:55.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.588 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.591 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-19T09:29:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-19T09:29:55.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-19T09:29:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-19T09:29:55.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-19T09:29:55.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-19T09:29:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-19T09:29:55.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-19T09:29:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-19T09:29:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-19T09:29:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-19T09:29:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-19T09:29:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-19T09:29:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-19T09:29:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-19T09:29:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-19T09:29:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T09:29:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-19T09:29:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-19T09:29:55.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-19T09:29:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-19T09:29:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-19T09:29:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-19T09:29:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.603 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T09:29:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-19T09:29:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-19T09:29:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-19T09:29:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T09:29:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:29:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-19T09:29:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-19T09:29:55.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-19T09:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-19T09:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T09:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-19T09:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-19T09:29:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-19T09:29:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-19T09:29:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-19T09:29:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T09:29:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-19T09:29:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-19T09:29:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T09:29:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T09:29:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T09:29:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-19T09:29:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-19T09:29:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-19T09:29:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-19T09:29:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-19T09:29:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-19T09:29:55.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T09:29:55.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:29:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-19T09:29:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-19T09:29:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-19T09:29:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-19T09:29:55.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.613 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.615 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-19T09:29:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-19T09:29:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T09:29:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-19T09:29:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-19T09:29:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-19T09:29:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-19T09:29:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T09:29:55.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T09:29:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-19T09:29:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-19T09:29:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-19T09:29:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T09:29:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T09:29:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-19T09:29:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-19T09:29:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T09:29:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-19T09:29:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.624 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-19T09:29:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-19T09:29:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-19T09:29:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T09:29:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-19T09:29:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-19T09:29:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-19T09:29:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T09:29:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-19T09:29:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-19T09:29:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-19T09:29:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T09:29:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T09:29:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-19T09:29:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-19T09:29:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T09:29:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-19T09:29:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T09:29:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-19T09:29:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T09:29:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-19T09:29:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-19T09:29:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-19T09:29:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-19T09:29:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-19T09:29:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T09:29:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-19T09:29:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-19T09:29:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T09:29:55.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T09:29:55.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-19T09:29:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-19T09:29:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.635 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T09:29:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-19T09:29:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-19T09:29:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T09:29:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-19T09:29:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-19T09:29:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-19T09:29:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T09:29:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-19T09:29:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-19T09:29:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-19T09:29:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T09:29:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-19T09:29:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-19T09:29:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-19T09:29:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T09:29:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-19T09:29:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-19T09:29:55.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-19T09:29:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T09:29:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-19T09:29:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-19T09:29:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-19T09:29:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T09:29:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-19T09:29:55.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-19T09:29:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-19T09:29:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T09:29:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.643 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-19T09:29:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-19T09:29:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-19T09:29:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T09:29:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-19T09:29:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-19T09:29:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-19T09:29:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T09:29:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-19T09:29:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-19T09:29:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-19T09:29:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T09:29:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-19T09:29:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-19T09:29:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-19T09:29:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T09:29:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-19T09:29:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-19T09:29:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-19T09:29:55.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T09:29:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-19T09:29:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-19T09:29:55.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-19T09:29:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T09:29:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-19T09:29:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-19T09:29:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-19T09:29:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T09:29:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-19T09:29:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-19T09:29:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-19T09:29:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T09:29:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-19T09:29:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-19T09:29:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-19T09:29:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T09:29:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-19T09:29:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-19T09:29:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T09:29:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-19T09:29:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T09:29:55.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-19T09:29:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T09:29:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-19T09:29:55.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-19T09:29:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-19T09:29:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T09:29:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-19T09:29:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-19T09:29:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T09:29:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-19T09:29:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-19T09:29:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-19T09:29:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-19T09:29:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-19T09:29:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T09:29:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T09:29:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-19T09:29:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-19T09:29:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:55.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-19T09:29:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-19T09:29:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-19T09:29:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T09:29:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-19T09:29:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-19T09:29:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-19T09:29:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-19T09:29:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T09:29:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T09:29:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T09:29:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T09:29:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T09:29:55.673 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T09:29:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T09:29:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-19T09:29:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T09:29:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-19T09:29:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-19T09:29:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-19T09:29:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-19T09:29:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-19T09:29:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T09:29:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-19T09:29:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-19T09:29:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-19T09:29:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-19T09:29:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-19T09:29:55.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-19T09:29:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-19T09:29:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-19T09:29:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T09:29:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-19T09:29:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-19T09:29:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T09:29:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T09:29:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-19T09:29:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-19T09:29:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T09:29:55.682 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T09:29:55.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-19T09:29:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-19T09:29:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T09:29:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-19T09:29:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-19T09:29:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T09:29:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-19T09:29:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-19T09:29:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-19T09:29:55.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-19T09:29:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-19T09:29:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-19T09:29:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T09:29:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-19T09:29:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-19T09:29:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T09:29:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-19T09:29:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-19T09:29:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T09:29:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-19T09:29:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-19T09:29:55.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T09:29:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-19T09:29:55.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-19T09:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T09:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-19T09:29:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.698 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-19T09:29:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-19T09:29:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-19T09:29:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T09:29:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.701 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-19T09:29:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-19T09:29:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-19T09:29:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T09:29:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-19T09:29:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T09:29:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T09:29:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T09:29:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T09:29:55.704 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T09:29:55.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T09:29:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.706 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.707 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T09:29:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T09:29:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T09:29:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T09:29:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.709 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T09:29:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T09:29:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T09:29:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T09:29:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T09:29:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T09:29:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T09:29:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T09:29:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T09:29:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T09:29:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T09:29:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T09:29:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T09:29:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T09:29:55.719 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.720 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T09:29:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T09:29:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.721 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T09:29:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T09:29:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-19T09:29:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T09:29:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T09:29:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-19T09:29:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-19T09:29:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T09:29:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T09:29:55.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-19T09:29:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.726 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T09:29:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T09:29:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T09:29:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T09:29:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T09:29:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T09:29:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T09:29:55.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T09:29:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.731 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T09:29:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T09:29:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.732 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T09:29:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T09:29:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T09:29:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T09:29:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.734 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T09:29:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T09:29:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.736 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T09:29:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T09:29:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T09:29:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T09:29:55.738 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.739 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T09:29:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T09:29:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T09:29:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T09:29:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T09:29:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T09:29:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.742 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T09:29:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T09:29:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T09:29:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T09:29:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T09:29:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T09:29:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T09:29:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T09:29:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T09:29:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-19T09:29:55.747 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T09:29:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T09:29:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.748 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-19T09:29:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.749 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T09:29:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T09:29:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.750 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T09:29:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T09:29:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-19T09:29:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T09:29:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T09:29:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.752 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T09:29:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T09:29:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.754 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T09:29:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T09:29:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T09:29:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T09:29:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.756 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T09:29:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T09:29:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-19T09:29:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-19T09:29:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.759 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-19T09:29:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-19T09:29:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.760 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-19T09:29:55.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-19T09:29:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-19T09:29:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-19T09:29:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-19T09:29:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.763 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-19T09:29:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-19T09:29:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-19T09:29:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.767 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.768 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.769 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.770 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.771 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.772 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.773 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-19T09:29:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-19T09:29:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.774 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-19T09:29:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-19T09:29:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T09:29:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.775 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-19T09:29:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-19T09:29:55.776 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.777 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.777 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.777 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-19T09:29:55.777 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.777 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.777 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.777 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.777 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.777 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.777 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.778 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-19T09:29:55.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-19T09:29:55.778 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.778 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.778 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.778 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-19T09:29:55.778 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.778 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.778 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.778 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.778 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.779 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.779 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.779 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.779 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-19T09:29:55.779 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-19T09:29:55.779 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.779 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.779 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.779 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.779 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.779 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.780 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.780 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.780 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.780 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.780 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.780 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-19T09:29:55.780 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-19T09:29:55.780 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.780 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.780 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.780 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-19T09:29:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-19T09:29:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-19T09:29:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-19T09:29:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-19T09:29:55.782 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-19T09:29:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-19T09:29:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.783 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-19T09:29:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-19T09:29:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-19T09:29:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-19T09:29:55.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-19T09:29:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.785 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-19T09:29:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-19T09:29:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-19T09:29:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-19T09:29:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.788 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.790 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.791 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.792 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-19T09:29:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-19T09:29:55.794 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-19T09:29:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T09:29:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.795 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-19T09:29:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-19T09:29:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-19T09:29:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T09:29:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-19T09:29:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-19T09:29:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-19T09:29:55.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-19T09:29:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-19T09:29:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-19T09:29:55.798 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-19T09:29:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T09:29:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-19T09:29:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-19T09:29:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T09:29:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T09:29:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-19T09:29:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-19T09:29:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.802 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-19T09:29:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T09:29:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.803 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-19T09:29:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.804 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.805 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.807 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.808 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.809 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.810 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-19T09:29:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-19T09:29:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-19T09:29:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.811 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-19T09:29:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-19T09:29:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-19T09:29:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.813 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.814 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-19T09:29:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-19T09:29:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-19T09:29:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-19T09:29:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-19T09:29:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-19T09:29:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T09:29:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-19T09:29:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-19T09:29:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-19T09:29:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T09:29:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.817 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-19T09:29:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-19T09:29:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-19T09:29:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.818 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-19T09:29:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-19T09:29:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T09:29:55.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.822 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-19T09:29:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-19T09:29:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-19T09:29:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-19T09:29:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-19T09:29:55.827 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-19T09:29:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-19T09:29:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.828 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-19T09:29:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-19T09:29:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-19T09:29:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:29:55.829 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-19T09:29:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-19T09:29:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-19T09:29:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-19T09:29:55.830 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T09:29:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-19T09:29:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-19T09:29:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.831 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-19T09:29:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.832 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-19T09:29:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-19T09:29:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-19T09:29:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:29:55.833 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-19T09:29:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.837 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.841 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.842 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-19T09:29:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.845 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.846 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.847 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.848 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.849 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.850 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-19T09:29:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-19T09:29:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-19T09:29:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-19T09:29:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T09:29:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-19T09:29:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-19T09:29:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-19T09:29:55.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T09:29:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.855 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.856 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.857 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.859 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.861 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-19T09:29:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-19T09:29:55.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-19T09:29:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-19T09:29:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-19T09:29:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T09:29:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.865 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.868 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.869 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-19T09:29:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-19T09:29:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-19T09:29:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T09:29:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-19T09:29:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-19T09:29:55.872 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-19T09:29:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-19T09:29:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-19T09:29:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-19T09:29:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-19T09:29:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.874 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T09:29:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-19T09:29:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-19T09:29:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.875 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-19T09:29:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-19T09:29:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-19T09:29:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.876 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-19T09:29:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T09:29:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T09:29:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.877 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-19T09:29:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.878 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.879 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.882 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.883 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.884 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-19T09:29:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.886 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.889 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-19T09:29:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.893 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.896 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-19T09:29:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-19T09:29:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-19T09:29:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T09:29:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.901 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.903 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.904 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.905 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.906 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-19T09:29:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-19T09:29:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.907 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-19T09:29:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-19T09:29:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-19T09:29:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-19T09:29:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.908 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-19T09:29:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.909 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.910 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.913 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-19T09:29:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.919 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.922 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-19T09:29:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.924 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.925 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.926 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.929 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-19T09:29:55.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-19T09:29:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-19T09:29:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-19T09:29:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-19T09:29:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T09:29:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-19T09:29:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-19T09:29:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-19T09:29:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.932 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T09:29:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-19T09:29:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-19T09:29:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.934 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.936 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.938 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.940 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-19T09:29:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-19T09:29:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-19T09:29:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.941 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-19T09:29:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.942 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-19T09:29:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-19T09:29:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-19T09:29:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-19T09:29:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-19T09:29:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-19T09:29:55.945 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T09:29:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T09:29:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.947 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.948 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.950 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-19T09:29:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-19T09:29:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.953 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T09:29:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T09:29:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-19T09:29:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.954 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-19T09:29:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-19T09:29:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-19T09:29:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-19T09:29:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-19T09:29:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-19T09:29:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T09:29:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.957 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-19T09:29:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-19T09:29:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-19T09:29:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T09:29:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.958 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-19T09:29:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-19T09:29:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-19T09:29:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.959 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-19T09:29:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-19T09:29:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-19T09:29:55.960 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-19T09:29:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-19T09:29:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-19T09:29:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-19T09:29:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-19T09:29:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-19T09:29:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-19T09:29:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-19T09:29:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.964 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.967 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.968 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-19T09:29:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-19T09:29:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-19T09:29:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T09:29:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.971 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-19T09:29:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-19T09:29:55.972 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.973 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-19T09:29:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-19T09:29:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-19T09:29:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.975 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-19T09:29:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-19T09:29:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.976 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.978 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.981 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.982 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-19T09:29:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-19T09:29:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.983 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.984 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-19T09:29:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-19T09:29:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.985 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-19T09:29:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.986 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.987 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.988 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.989 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.990 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.991 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T09:29:55.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T09:29:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T09:29:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.993 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-19T09:29:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:55.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.996 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:55.997 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:55.998 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:55.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.000 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-19T09:29:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-19T09:29:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-19T09:29:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.001 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T09:29:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-19T09:29:56.002 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.004 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.006 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.007 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.008 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-19T09:29:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-19T09:29:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.009 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-19T09:29:56.010 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-19T09:29:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-19T09:29:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.011 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-19T09:29:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-19T09:29:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.012 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-19T09:29:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-19T09:29:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.013 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-19T09:29:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-19T09:29:56.014 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.015 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-19T09:29:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-19T09:29:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.016 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-19T09:29:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-19T09:29:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.017 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-19T09:29:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-19T09:29:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-19T09:29:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-19T09:29:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-19T09:29:56.020 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-19T09:29:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-19T09:29:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-19T09:29:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.021 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-19T09:29:56.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-19T09:29:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.023 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-19T09:29:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-19T09:29:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.024 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.025 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-19T09:29:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-19T09:29:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-19T09:29:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T09:29:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-19T09:29:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-19T09:29:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.027 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-19T09:29:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.029 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.030 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.031 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.032 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.033 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.034 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-19T09:29:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.035 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.036 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.038 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.039 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.040 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.041 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-19T09:29:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.042 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.043 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.044 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.045 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.046 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.047 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.048 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-19T09:29:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-19T09:29:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-19T09:29:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.049 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T09:29:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:56.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-19T09:29:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.051 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.052 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.056 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-19T09:29:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.058 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-19T09:29:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-19T09:29:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T09:29:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.059 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-19T09:29:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-19T09:29:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.060 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T09:29:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-19T09:29:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-19T09:29:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T09:29:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-19T09:29:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-19T09:29:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.062 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T09:29:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-19T09:29:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-19T09:29:56.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T09:29:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.064 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-19T09:29:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-19T09:29:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.065 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-19T09:29:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-19T09:29:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.066 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-19T09:29:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-19T09:29:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T09:29:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.068 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-19T09:29:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-19T09:29:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.070 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-19T09:29:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-19T09:29:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.071 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-19T09:29:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-19T09:29:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-19T09:29:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.072 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-19T09:29:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-19T09:29:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-19T09:29:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-19T09:29:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.075 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.076 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.077 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.079 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.080 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.081 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-19T09:29:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-19T09:29:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.083 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-19T09:29:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-19T09:29:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-19T09:29:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-19T09:29:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T09:29:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.085 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-19T09:29:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.086 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.087 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.089 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.090 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-19T09:29:56.093 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.094 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.096 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-19T09:29:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-19T09:29:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-19T09:29:56.100 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T09:29:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-19T09:29:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-19T09:29:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-19T09:29:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-19T09:29:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-19T09:29:56.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T09:29:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-19T09:29:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T09:29:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T09:29:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.104 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-19T09:29:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-19T09:29:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-19T09:29:56.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-19T09:29:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-19T09:29:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-19T09:29:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.107 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.109 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.114 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-19T09:29:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-19T09:29:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.115 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-19T09:29:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-19T09:29:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-19T09:29:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-19T09:29:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-19T09:29:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T09:29:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T09:29:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "modules": [ 2024-09-19T09:29:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-19T09:29:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard", 2024-09-19T09:29:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "iostat", 2024-09-19T09:29:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "nfs", 2024-09-19T09:29:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "restful" 2024-09-19T09:29:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-19T09:29:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-19T09:29:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T09:29:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.131 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-19T09:29:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-19T09:29:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-19T09:29:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-19T09:29:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-19T09:29:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-19T09:29:56.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-19T09:29:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-19T09:29:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-19T09:29:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-19T09:29:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-19T09:29:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-19T09:29:56.135 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-19T09:29:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-19T09:29:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-19T09:29:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-19T09:29:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-19T09:29:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-19T09:29:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-19T09:29:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-19T09:29:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-19T09:29:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-19T09:29:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T09:29:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-19T09:29:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-19T09:29:56.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-19T09:29:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.141 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-19T09:29:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-19T09:29:56.142 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-19T09:29:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T09:29:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-19T09:29:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-19T09:29:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-19T09:29:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T09:29:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:29:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-19T09:29:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-19T09:29:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-19T09:29:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-19T09:29:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T09:29:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:29:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-19T09:29:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-19T09:29:56.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-19T09:29:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-19T09:29:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-19T09:29:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T09:29:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.147 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-19T09:29:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-19T09:29:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T09:29:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T09:29:56.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T09:29:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-19T09:29:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-19T09:29:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-19T09:29:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-19T09:29:56.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-19T09:29:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-19T09:29:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T09:29:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-19T09:29:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-19T09:29:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-19T09:29:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T09:29:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T09:29:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-19T09:29:56.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-19T09:29:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-19T09:29:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-19T09:29:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T09:29:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T09:29:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.152 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.153 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.154 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.156 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.157 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-19T09:29:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-19T09:29:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:56.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T09:29:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-19T09:29:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-19T09:29:56.160 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-19T09:29:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-19T09:29:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-19T09:29:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-19T09:29:56.161 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-19T09:29:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-19T09:29:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-19T09:29:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-19T09:29:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-19T09:29:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T09:29:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:56.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T09:29:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-19T09:29:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-19T09:29:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-19T09:29:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T09:29:56.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T09:29:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-19T09:29:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-19T09:29:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T09:29:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-19T09:29:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.167 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-19T09:29:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-19T09:29:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-19T09:29:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T09:29:56.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-19T09:29:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-19T09:29:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-19T09:29:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T09:29:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-19T09:29:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-19T09:29:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-19T09:29:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T09:29:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-19T09:29:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-19T09:29:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.171 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T09:29:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T09:29:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.172 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-19T09:29:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-19T09:29:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T09:29:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-19T09:29:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T09:29:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-19T09:29:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T09:29:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-19T09:29:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-19T09:29:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-19T09:29:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-19T09:29:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-19T09:29:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T09:29:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-19T09:29:56.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-19T09:29:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T09:29:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.177 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-19T09:29:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-19T09:29:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.178 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T09:29:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-19T09:29:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-19T09:29:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.179 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T09:29:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-19T09:29:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-19T09:29:56.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T09:29:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-19T09:29:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-19T09:29:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-19T09:29:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T09:29:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-19T09:29:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-19T09:29:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-19T09:29:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T09:29:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-19T09:29:56.184 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-19T09:29:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-19T09:29:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T09:29:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.185 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.186 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.186 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-19T09:29:56.186 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.186 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.186 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.186 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-19T09:29:56.186 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-19T09:29:56.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.186 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.187 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.187 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T09:29:56.187 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.187 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.187 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.187 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-19T09:29:56.187 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.187 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.187 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-19T09:29:56.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-19T09:29:56.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.188 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.188 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.188 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T09:29:56.188 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.188 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.188 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.188 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-19T09:29:56.188 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.188 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.188 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-19T09:29:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-19T09:29:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T09:29:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-19T09:29:56.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-19T09:29:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-19T09:29:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T09:29:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.190 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-19T09:29:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-19T09:29:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-19T09:29:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.191 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T09:29:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-19T09:29:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-19T09:29:56.192 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-19T09:29:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T09:29:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-19T09:29:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-19T09:29:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-19T09:29:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T09:29:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-19T09:29:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.194 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-19T09:29:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-19T09:29:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T09:29:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.195 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-19T09:29:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-19T09:29:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-19T09:29:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T09:29:56.196 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-19T09:29:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-19T09:29:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-19T09:29:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T09:29:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-19T09:29:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-19T09:29:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-19T09:29:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T09:29:56.199 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-19T09:29:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-19T09:29:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-19T09:29:56.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T09:29:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-19T09:29:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-19T09:29:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-19T09:29:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T09:29:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.202 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-19T09:29:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-19T09:29:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.203 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T09:29:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-19T09:29:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T09:29:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.204 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-19T09:29:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T09:29:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-19T09:29:56.205 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-19T09:29:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-19T09:29:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T09:29:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.206 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-19T09:29:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-19T09:29:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-19T09:29:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.207 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-19T09:29:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-19T09:29:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-19T09:29:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.208 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T09:29:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-19T09:29:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-19T09:29:56.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-19T09:29:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.210 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-19T09:29:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-19T09:29:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T09:29:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.211 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T09:29:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-19T09:29:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-19T09:29:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.212 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-19T09:29:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-19T09:29:56.213 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-19T09:29:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T09:29:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.214 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-19T09:29:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-19T09:29:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.215 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T09:29:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.216 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.217 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.218 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-19T09:29:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-19T09:29:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T09:29:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.220 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T09:29:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.221 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.222 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T09:29:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T09:29:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.224 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T09:29:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.225 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T09:29:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T09:29:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-19T09:29:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T09:29:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-19T09:29:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-19T09:29:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.227 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-19T09:29:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-19T09:29:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-19T09:29:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.228 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T09:29:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-19T09:29:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-19T09:29:56.229 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-19T09:29:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-19T09:29:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-19T09:29:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-19T09:29:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-19T09:29:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-19T09:29:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T09:29:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-19T09:29:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-19T09:29:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.232 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-19T09:29:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T09:29:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.233 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-19T09:29:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-19T09:29:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T09:29:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T09:29:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.234 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-19T09:29:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-19T09:29:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T09:29:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-19T09:29:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-19T09:29:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.236 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T09:29:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-19T09:29:56.237 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-19T09:29:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T09:29:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.238 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-19T09:29:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-19T09:29:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T09:29:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.239 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-19T09:29:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-19T09:29:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T09:29:56.240 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-19T09:29:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-19T09:29:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-19T09:29:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.241 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-19T09:29:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-19T09:29:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-19T09:29:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.242 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T09:29:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-19T09:29:56.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-19T09:29:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T09:29:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-19T09:29:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-19T09:29:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T09:29:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-19T09:29:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-19T09:29:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.246 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T09:29:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-19T09:29:56.247 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-19T09:29:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T09:29:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-19T09:29:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-19T09:29:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T09:29:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-19T09:29:56.250 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.251 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.253 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.255 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-19T09:29:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-19T09:29:56.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-19T09:29:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T09:29:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-19T09:29:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-19T09:29:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-19T09:29:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T09:29:56.259 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-19T09:29:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.260 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T09:29:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T09:29:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T09:29:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T09:29:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T09:29:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T09:29:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-19T09:29:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T09:29:56.264 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T09:29:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T09:29:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T09:29:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T09:29:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.267 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T09:29:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T09:29:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.269 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T09:29:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T09:29:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T09:29:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T09:29:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T09:29:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T09:29:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T09:29:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T09:29:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T09:29:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T09:29:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.275 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T09:29:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T09:29:56.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T09:29:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T09:29:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.278 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T09:29:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T09:29:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-19T09:29:56.279 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T09:29:56.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T09:29:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-19T09:29:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-19T09:29:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.282 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T09:29:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T09:29:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-19T09:29:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T09:29:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T09:29:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T09:29:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T09:29:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.285 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T09:29:56.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T09:29:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T09:29:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T09:29:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T09:29:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T09:29:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T09:29:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T09:29:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T09:29:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T09:29:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T09:29:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T09:29:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T09:29:56.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T09:29:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.295 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T09:29:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T09:29:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.297 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T09:29:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T09:29:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T09:29:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T09:29:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.299 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T09:29:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T09:29:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T09:29:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T09:29:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T09:29:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T09:29:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T09:29:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T09:29:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T09:29:56.304 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T09:29:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T09:29:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-19T09:29:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.306 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T09:29:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T09:29:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-19T09:29:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T09:29:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T09:29:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T09:29:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T09:29:56.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-19T09:29:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.310 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T09:29:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T09:29:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T09:29:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T09:29:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.312 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T09:29:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T09:29:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.314 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T09:29:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T09:29:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T09:29:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T09:29:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-19T09:29:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-19T09:29:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-19T09:29:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-19T09:29:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-19T09:29:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-19T09:29:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-19T09:29:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-19T09:29:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-19T09:29:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-19T09:29:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-19T09:29:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-19T09:29:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.323 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.324 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.326 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.328 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-19T09:29:56.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-19T09:29:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-19T09:29:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-19T09:29:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T09:29:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-19T09:29:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-19T09:29:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-19T09:29:56.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-19T09:29:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-19T09:29:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-19T09:29:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.339 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-19T09:29:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-19T09:29:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-19T09:29:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-19T09:29:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-19T09:29:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-19T09:29:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-19T09:29:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-19T09:29:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-19T09:29:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.343 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-19T09:29:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-19T09:29:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-19T09:29:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-19T09:29:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-19T09:29:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-19T09:29:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-19T09:29:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-19T09:29:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.346 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-19T09:29:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-19T09:29:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-19T09:29:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.349 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.352 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-19T09:29:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-19T09:29:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-19T09:29:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T09:29:56.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-19T09:29:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-19T09:29:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-19T09:29:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.356 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T09:29:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-19T09:29:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-19T09:29:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-19T09:29:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-19T09:29:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-19T09:29:56.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-19T09:29:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-19T09:29:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T09:29:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-19T09:29:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-19T09:29:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T09:29:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T09:29:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-19T09:29:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-19T09:29:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-19T09:29:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T09:29:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-19T09:29:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-19T09:29:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-19T09:29:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.370 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-19T09:29:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.371 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-19T09:29:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-19T09:29:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-19T09:29:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-19T09:29:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-19T09:29:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-19T09:29:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-19T09:29:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.374 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-19T09:29:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-19T09:29:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-19T09:29:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T09:29:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-19T09:29:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-19T09:29:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-19T09:29:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T09:29:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-19T09:29:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-19T09:29:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-19T09:29:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-19T09:29:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-19T09:29:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T09:29:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.380 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.383 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.384 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-19T09:29:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-19T09:29:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-19T09:29:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-19T09:29:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-19T09:29:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-19T09:29:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-19T09:29:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-19T09:29:56.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-19T09:29:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-19T09:29:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:29:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-19T09:29:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-19T09:29:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-19T09:29:56.390 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-19T09:29:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T09:29:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-19T09:29:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-19T09:29:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-19T09:29:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-19T09:29:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-19T09:29:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-19T09:29:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T09:29:56.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-19T09:29:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.394 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.395 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.396 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.398 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.399 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-19T09:29:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.402 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.403 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.405 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.406 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-19T09:29:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-19T09:29:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-19T09:29:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-19T09:29:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T09:29:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.409 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-19T09:29:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-19T09:29:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-19T09:29:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T09:29:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.411 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-19T09:29:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-19T09:29:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-19T09:29:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-19T09:29:56.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-19T09:29:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T09:29:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.420 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.423 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.424 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-19T09:29:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-19T09:29:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.426 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-19T09:29:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T09:29:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-19T09:29:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-19T09:29:56.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-19T09:29:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-19T09:29:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-19T09:29:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-19T09:29:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-19T09:29:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.429 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T09:29:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-19T09:29:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-19T09:29:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.430 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-19T09:29:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-19T09:29:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-19T09:29:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.431 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-19T09:29:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T09:29:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T09:29:56.432 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-19T09:29:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.433 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.434 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.437 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.438 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.439 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-19T09:29:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.441 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.442 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.444 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.446 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-19T09:29:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-19T09:29:56.453 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-19T09:29:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-19T09:29:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T09:29:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.459 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.460 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-19T09:29:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-19T09:29:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-19T09:29:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-19T09:29:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-19T09:29:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-19T09:29:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-19T09:29:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-19T09:29:56.470 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.471 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.475 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-19T09:29:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.482 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.485 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-19T09:29:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-19T09:29:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.486 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-19T09:29:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-19T09:29:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-19T09:29:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T09:29:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-19T09:29:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-19T09:29:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-19T09:29:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T09:29:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-19T09:29:56.489 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-19T09:29:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.491 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-19T09:29:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-19T09:29:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-19T09:29:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-19T09:29:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-19T09:29:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-19T09:29:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-19T09:29:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.499 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-19T09:29:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-19T09:29:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-19T09:29:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-19T09:29:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-19T09:29:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T09:29:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T09:29:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.503 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.506 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.507 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-19T09:29:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-19T09:29:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T09:29:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T09:29:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-19T09:29:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-19T09:29:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-19T09:29:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-19T09:29:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-19T09:29:56.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-19T09:29:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-19T09:29:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T09:29:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-19T09:29:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-19T09:29:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-19T09:29:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T09:29:56.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-19T09:29:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-19T09:29:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-19T09:29:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-19T09:29:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-19T09:29:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-19T09:29:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-19T09:29:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-19T09:29:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-19T09:29:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-19T09:29:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-19T09:29:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-19T09:29:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-19T09:29:56.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-19T09:29:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.519 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-19T09:29:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-19T09:29:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-19T09:29:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.526 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-19T09:29:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T09:29:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-19T09:29:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-19T09:29:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-19T09:29:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-19T09:29:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-19T09:29:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-19T09:29:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-19T09:29:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-19T09:29:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-19T09:29:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-19T09:29:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-19T09:29:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-19T09:29:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.542 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.543 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.544 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.545 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.546 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T09:29:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T09:29:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-19T09:29:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.549 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T09:29:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-19T09:29:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.550 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.551 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.552 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.554 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.555 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.556 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-19T09:29:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-19T09:29:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-19T09:29:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T09:29:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-19T09:29:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.558 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.559 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.561 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-19T09:29:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-19T09:29:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-19T09:29:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-19T09:29:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-19T09:29:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-19T09:29:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-19T09:29:56.567 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-19T09:29:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-19T09:29:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-19T09:29:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-19T09:29:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-19T09:29:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-19T09:29:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-19T09:29:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.571 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-19T09:29:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-19T09:29:56.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.574 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-19T09:29:56.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-19T09:29:56.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.574 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.574 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.574 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.574 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.574 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.575 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-19T09:29:56.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-19T09:29:56.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.575 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-19T09:29:56.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-19T09:29:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-19T09:29:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-19T09:29:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-19T09:29:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-19T09:29:56.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.578 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-19T09:29:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-19T09:29:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-19T09:29:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-19T09:29:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-19T09:29:56.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T09:29:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-19T09:29:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-19T09:29:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-19T09:29:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.587 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-19T09:29:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-19T09:29:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-19T09:29:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.603 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-19T09:29:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-19T09:29:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T09:29:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-19T09:29:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.607 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.610 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-19T09:29:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-19T09:29:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-19T09:29:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T09:29:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-19T09:29:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-19T09:29:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T09:29:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.615 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-19T09:29:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-19T09:29:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T09:29:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-19T09:29:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-19T09:29:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T09:29:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-19T09:29:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-19T09:29:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T09:29:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-19T09:29:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-19T09:29:56.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-19T09:29:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-19T09:29:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-19T09:29:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-19T09:29:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T09:29:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-19T09:29:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-19T09:29:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-19T09:29:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-19T09:29:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-19T09:29:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-19T09:29:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-19T09:29:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-19T09:29:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-19T09:29:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-19T09:29:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-19T09:29:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.629 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.630 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.633 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-19T09:29:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-19T09:29:56.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.637 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-19T09:29:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-19T09:29:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-19T09:29:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-19T09:29:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T09:29:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-19T09:29:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-19T09:29:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-19T09:29:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-19T09:29:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-19T09:29:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T09:29:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-19T09:29:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-19T09:29:56.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-19T09:29:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.656 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-19T09:29:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-19T09:29:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-19T09:29:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T09:29:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-19T09:29:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T09:29:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-19T09:29:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T09:29:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-19T09:29:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-19T09:29:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-19T09:29:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-19T09:29:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-19T09:29:56.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-19T09:29:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-19T09:29:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-19T09:29:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-19T09:29:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-19T09:29:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-19T09:29:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-19T09:29:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-19T09:29:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-19T09:29:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-19T09:29:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-19T09:29:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-19T09:29:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-19T09:29:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-19T09:29:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-19T09:29:56.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-19T09:29:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-19T09:29:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-19T09:29:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-19T09:29:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-19T09:29:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-19T09:29:56.667 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-19T09:29:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-19T09:29:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-19T09:29:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-19T09:29:56.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-19T09:29:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-19T09:29:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-19T09:29:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-19T09:29:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-19T09:29:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-19T09:29:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-19T09:29:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-19T09:29:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-19T09:29:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-19T09:29:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-19T09:29:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-19T09:29:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-19T09:29:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-19T09:29:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-19T09:29:56.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T09:29:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-19T09:29:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-19T09:29:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-19T09:29:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-19T09:29:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-19T09:29:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-19T09:29:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-19T09:29:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.673 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "services": { 2024-09-19T09:29:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard": "https://172.21.15.16:8443/" 2024-09-19T09:29:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "always_on_modules": { 2024-09-19T09:29:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "octopus": [ 2024-09-19T09:29:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-19T09:29:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-19T09:29:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-19T09:29:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-19T09:29:56.674 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-19T09:29:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-19T09:29:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-19T09:29:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-19T09:29:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-19T09:29:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-19T09:29:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "pacific": [ 2024-09-19T09:29:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-19T09:29:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-19T09:29:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-19T09:29:56.675 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-19T09:29:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-19T09:29:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-19T09:29:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-19T09:29:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-19T09:29:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-19T09:29:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-19T09:29:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "quincy": [ 2024-09-19T09:29:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-19T09:29:56.676 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-19T09:29:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-19T09:29:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-19T09:29:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-19T09:29:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-19T09:29:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-19T09:29:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-19T09:29:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-19T09:29:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-19T09:29:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "reef": [ 2024-09-19T09:29:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-19T09:29:56.677 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-19T09:29:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-19T09:29:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-19T09:29:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-19T09:29:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-19T09:29:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-19T09:29:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-19T09:29:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-19T09:29:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-19T09:29:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-19T09:29:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "squid": [ 2024-09-19T09:29:56.678 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-19T09:29:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-19T09:29:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-19T09:29:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-19T09:29:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-19T09:29:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-19T09:29:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-19T09:29:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-19T09:29:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-19T09:29:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-19T09:29:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T09:29:56.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "last_failure_osd_epoch": 3, 2024-09-19T09:29:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "active_clients": [ 2024-09-19T09:29:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-19T09:29:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-19T09:29:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-19T09:29:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-19T09:29:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 458055934 2024-09-19T09:29:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T09:29:56.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "libcephsqlite", 2024-09-19T09:29:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-19T09:29:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-19T09:29:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-19T09:29:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1630957839 2024-09-19T09:29:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T09:29:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.681 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-19T09:29:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-19T09:29:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-19T09:29:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-19T09:29:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1482490412 2024-09-19T09:29:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T09:29:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-19T09:29:56.682 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-19T09:29:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-19T09:29:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-19T09:29:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-19T09:29:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2430428028 2024-09-19T09:29:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T09:29:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:56.683 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-19T09:29:56.683 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:29:56.686 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:55 smithi097 ceph-mon[34864]: pgmap v407: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s rd, 813 KiB/s wr, 334 op/s 2024-09-19T09:29:56.686 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:55 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1402201163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:29:56.686 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:55 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1213728845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:29:56.686 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:55 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2557913436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:29:56.686 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:56 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/414189664' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T09:29:56.686 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:55 smithi142 ceph-mon[35032]: pgmap v407: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s rd, 813 KiB/s wr, 334 op/s 2024-09-19T09:29:56.686 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:55 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1402201163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:29:56.686 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:55 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1213728845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:29:56.687 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:55 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2557913436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:29:56.687 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:56 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/414189664' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T09:29:56.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.044+0000 7f422caba640 1 -- 172.21.15.16:0/414189664 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f42140a32d0 msgr2=0x7f42140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:56.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.044+0000 7f422caba640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f42140a32d0 0x7f42140a5790 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4210005ab0 tx=0x7f42100315b0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.044+0000 7f422caba640 1 -- 172.21.15.16:0/414189664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4228100a10 msgr2=0x7f42281980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:29:56.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.044+0000 7f422caba640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4228100a10 0x7f42281980a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f421800b450 tx=0x7f421800b910 comp rx=0 tx=0).stop 2024-09-19T09:29:56.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.044+0000 7f422caba640 1 -- 172.21.15.16:0/414189664 shutdown_connections 2024-09-19T09:29:56.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.044+0000 7f422caba640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f42140a32d0 0x7f42140a5790 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.044+0000 7f422caba640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42281025c0 0x7f422819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.044+0000 7f422caba640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4228101c00 0x7f42281985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.044+0000 7f422caba640 1 --2- 172.21.15.16:0/414189664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4228100a10 0x7f42281980a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:29:56.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.044+0000 7f422caba640 1 -- 172.21.15.16:0/414189664 >> 172.21.15.16:0/414189664 conn(0x7f42280fc170 msgr2=0x7f4228069410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:29:56.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.044+0000 7f422caba640 1 -- 172.21.15.16:0/414189664 shutdown_connections 2024-09-19T09:29:56.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:29:55.044+0000 7f422caba640 1 -- 172.21.15.16:0/414189664 wait complete. 2024-09-19T09:29:56.695 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T09:29:56.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T09:29:56.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:55 smithi016 ceph-mon[30979]: pgmap v407: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s rd, 813 KiB/s wr, 334 op/s 2024-09-19T09:29:56.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:55 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1402201163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T09:29:56.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:55 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1213728845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T09:29:56.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:55 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2557913436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T09:29:56.696 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:56 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/414189664' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T09:29:56.967 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:57.283 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:57 smithi016 ceph-mon[30979]: pgmap v408: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 221 KiB/s wr, 89 op/s 2024-09-19T09:29:57.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:57 smithi142 ceph-mon[35032]: pgmap v408: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 221 KiB/s wr, 89 op/s 2024-09-19T09:29:57.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:57 smithi097 ceph-mon[34864]: pgmap v408: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 221 KiB/s wr, 89 op/s 2024-09-19T09:29:57.493 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:29:57.494 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:29:57.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 265, 2024-09-19T09:29:57.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 265, 2024-09-19T09:29:57.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 91015, 2024-09-19T09:29:57.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 83447, 2024-09-19T09:29:57.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-09-19T09:29:57.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-19T09:29:57.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.028563191, 2024-09-19T09:29:57.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.008850928, 2024-09-19T09:29:57.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.008843320, 2024-09-19T09:29:57.494 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.007322224, 2024-09-19T09:29:57.495 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:29:57.495 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 264, 2024-09-19T09:29:57.495 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.016549364, 2024-09-19T09:29:57.495 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000062686 2024-09-19T09:29:57.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.495 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:29:57.495 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.495 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.495 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.495 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T09:29:57.495 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:29:57.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.496 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:29:57.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 170, 2024-09-19T09:29:57.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 228, 2024-09-19T09:29:57.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 42386, 2024-09-19T09:29:57.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 234418, 2024-09-19T09:29:57.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-19T09:29:57.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-09-19T09:29:57.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.018811861, 2024-09-19T09:29:57.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.007037436, 2024-09-19T09:29:57.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.004509512, 2024-09-19T09:29:57.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.005389243, 2024-09-19T09:29:57.496 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:29:57.496 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 226, 2024-09-19T09:29:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.008934479, 2024-09-19T09:29:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000039533 2024-09-19T09:29:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:29:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:29:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 180608 2024-09-19T09:29:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:29:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 350, 2024-09-19T09:29:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 345, 2024-09-19T09:29:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 532732, 2024-09-19T09:29:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 131098, 2024-09-19T09:29:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-09-19T09:29:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-19T09:29:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.031543672, 2024-09-19T09:29:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.011063970, 2024-09-19T09:29:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.017270229, 2024-09-19T09:29:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.005431645, 2024-09-19T09:29:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:29:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 344, 2024-09-19T09:29:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.011759862, 2024-09-19T09:29:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000034185 2024-09-19T09:29:57.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.499 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:29:57.499 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.499 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.499 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.499 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 463648, 2024-09-19T09:29:57.499 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 48736 2024-09-19T09:29:57.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.499 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-19T09:29:57.499 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-09-19T09:29:57.499 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-19T09:29:57.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.500 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-09-19T09:29:57.500 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-19T09:29:57.500 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-19T09:29:57.500 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1200, 2024-09-19T09:29:57.500 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.028862827, 2024-09-19T09:29:57.500 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000024052 2024-09-19T09:29:57.500 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:57.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.500 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-09-19T09:29:57.500 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-19T09:29:57.500 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-19T09:29:57.500 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.500 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.501 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.501 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:57.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.501 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-09-19T09:29:57.501 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-09-19T09:29:57.501 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-09-19T09:29:57.501 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-09-19T09:29:57.501 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.501 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.501 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.501 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-09-19T09:29:57.501 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-09-19T09:29:57.502 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-09-19T09:29:57.502 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-09-19T09:29:57.502 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-09-19T09:29:57.502 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-09-19T09:29:57.502 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-09-19T09:29:57.502 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 0, 2024-09-19T09:29:57.502 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-09-19T09:29:57.502 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 0, 2024-09-19T09:29:57.502 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-09-19T09:29:57.502 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 0, 2024-09-19T09:29:57.502 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-09-19T09:29:57.502 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-09-19T09:29:57.502 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-09-19T09:29:57.503 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-09-19T09:29:57.503 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-09-19T09:29:57.503 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-09-19T09:29:57.503 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-09-19T09:29:57.503 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-09-19T09:29:57.503 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:29:57.503 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-09-19T09:29:57.503 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-09-19T09:29:57.503 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-09-19T09:29:57.503 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-09-19T09:29:57.503 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-09-19T09:29:57.503 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-09-19T09:29:57.503 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-09-19T09:29:57.504 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-09-19T09:29:57.504 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-09-19T09:29:57.504 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:29:57.504 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-09-19T09:29:57.504 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-09-19T09:29:57.504 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-09-19T09:29:57.504 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-09-19T09:29:57.504 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:29:57.504 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-09-19T09:29:57.504 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:29:57.505 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-09-19T09:29:57.506 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-09-19T09:29:57.506 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.506 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-09-19T09:29:57.506 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-09-19T09:29:57.506 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-09-19T09:29:57.506 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-09-19T09:29:57.506 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-09-19T09:29:57.506 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-09-19T09:29:57.506 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-09-19T09:29:57.506 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-09-19T09:29:57.506 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:29:57.506 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-09-19T09:29:57.506 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:29:57.507 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-09-19T09:29:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-09-19T09:29:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-09-19T09:29:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-09-19T09:29:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-09-19T09:29:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-09-19T09:29:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-09-19T09:29:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 5, 2024-09-19T09:29:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-09-19T09:29:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-09-19T09:29:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-09-19T09:29:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-09-19T09:29:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4195734, 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4195426, 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 5 2024-09-19T09:29:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 3, 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 3, 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 1, 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 1, 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 0, 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 0, 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-09-19T09:29:57.510 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 48056, 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215328 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.511 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-09-19T09:29:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-09-19T09:29:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-09-19T09:29:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.513 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-09-19T09:29:57.513 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.513 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.513 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.513 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-09-19T09:29:57.513 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.513 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.513 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.513 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-09-19T09:29:57.513 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.514 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-09-19T09:29:57.515 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.515 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.515 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.515 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-09-19T09:29:57.515 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.515 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.515 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.515 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-09-19T09:29:57.515 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.516 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.516 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.516 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-09-19T09:29:57.516 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.516 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.516 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.517 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-09-19T09:29:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.519 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-09-19T09:29:57.520 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.520 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.520 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.520 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-09-19T09:29:57.520 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.520 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.520 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.520 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-09-19T09:29:57.520 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.520 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.520 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:29:57.522 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 2, 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 1, 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-09-19T09:29:57.523 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-09-19T09:29:57.524 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 1174, 2024-09-19T09:29:57.524 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-09-19T09:29:57.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.524 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-19T09:29:57.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:29:57.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-19T09:29:57.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:29:57.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:29:57.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:29:57.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:29:57.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:29:57.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:29:57.524 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:29:57.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:29:57.526 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:29:57.526 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:29:57.526 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-19T09:29:57.526 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:29:57.526 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:29:57.526 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:29:57.526 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-19T09:29:57.526 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-19T09:29:57.526 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-19T09:29:57.526 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:29:57.526 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:29:57.526 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:29:57.526 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 189239, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 48, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 7592, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 156, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:29:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:29:57.528 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-19T09:29:57.528 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-19T09:29:57.528 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 8512, 2024-09-19T09:29:57.528 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 13, 2024-09-19T09:29:57.528 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-19T09:29:57.528 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-19T09:29:57.528 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-19T09:29:57.528 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-19T09:29:57.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.528 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-09-19T09:29:57.528 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-09-19T09:29:57.528 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-09-19T09:29:57.528 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 689, 2024-09-19T09:29:57.529 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-09-19T09:29:57.529 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-09-19T09:29:57.529 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 689, 2024-09-19T09:29:57.529 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-09-19T09:29:57.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 689, 2024-09-19T09:29:57.529 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.411993314, 2024-09-19T09:29:57.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000597958 2024-09-19T09:29:57.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.529 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-09-19T09:29:57.529 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-09-19T09:29:57.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 689, 2024-09-19T09:29:57.529 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 690 2024-09-19T09:29:57.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 689, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 689, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 423, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 264, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-09-19T09:29:57.530 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-09-19T09:29:57.531 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-09-19T09:29:57.532 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 81, 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 2, 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 5, 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 5, 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-09-19T09:29:57.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-09-19T09:29:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-09-19T09:29:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:29:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-09-19T09:29:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-09-19T09:29:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-09-19T09:29:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-09-19T09:29:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-09-19T09:29:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-09-19T09:29:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:29:57.534 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-09-19T09:29:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-09-19T09:29:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-09-19T09:29:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-09-19T09:29:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-09-19T09:29:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:29:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-19T09:29:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-19T09:29:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-19T09:29:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 781, 2024-09-19T09:29:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 588212, 2024-09-19T09:29:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-19T09:29:57.535 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 781, 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 781, 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 588212, 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-19T09:29:57.536 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-19T09:29:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-19T09:29:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-19T09:29:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-19T09:29:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-19T09:29:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 689, 2024-09-19T09:29:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 1435, 2024-09-19T09:29:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 689, 2024-09-19T09:29:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1435, 2024-09-19T09:29:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-19T09:29:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.537 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 689, 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 689, 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 689, 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 689, 2024-09-19T09:29:57.538 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-19T09:29:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:29:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-19T09:29:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-19T09:29:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-19T09:29:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-19T09:29:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-19T09:29:57.539 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-19T09:29:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-19T09:29:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-19T09:29:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-19T09:29:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-19T09:29:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-19T09:29:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-19T09:29:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:57.540 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55de666f2b40": { 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:57.541 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:57.542 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:57.542 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:29:58.351 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-19T09:29:58.351 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-19T09:29:58.351 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-19T09:29:58.352 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-19T09:29:58.352 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T09:29:58.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T09:29:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:29:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:29:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:29:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-19T09:29:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-09-19T09:29:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3024, 2024-09-19T09:29:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-09-19T09:29:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-19T09:29:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-19T09:29:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.003510859, 2024-09-19T09:29:59.099 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000036560, 2024-09-19T09:29:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002329228, 2024-09-19T09:29:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:29:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:29:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-09-19T09:29:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.004140906, 2024-09-19T09:29:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001380302 2024-09-19T09:29:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:29:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:59.101 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T09:29:59.101 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:29:59.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:59.101 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:29:59.101 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 87, 2024-09-19T09:29:59.101 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 84, 2024-09-19T09:29:59.101 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 441792, 2024-09-19T09:29:59.101 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 50752, 2024-09-19T09:29:59.101 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-19T09:29:59.101 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-19T09:29:59.101 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.013475235, 2024-09-19T09:29:59.101 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.005655764, 2024-09-19T09:29:59.102 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.011045833, 2024-09-19T09:29:59.102 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:29:59.102 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:29:59.102 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 83, 2024-09-19T09:29:59.102 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.004210053, 2024-09-19T09:29:59.102 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000050723 2024-09-19T09:29:59.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:59.102 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:29:59.102 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:59.102 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:59.102 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:59.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:59.103 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 441792, 2024-09-19T09:29:59.103 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 50752 2024-09-19T09:29:59.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:59.103 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:29:59.103 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1, 2024-09-19T09:29:59.103 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 63, 2024-09-19T09:29:59.103 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 160, 2024-09-19T09:29:59.103 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 89616, 2024-09-19T09:29:59.103 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-09-19T09:29:59.103 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-19T09:29:59.103 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.005128511, 2024-09-19T09:29:59.104 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.003346345, 2024-09-19T09:29:59.104 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001094148, 2024-09-19T09:29:59.104 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:29:59.104 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:29:59.104 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 61, 2024-09-19T09:29:59.104 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.002704652, 2024-09-19T09:29:59.104 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000044338 2024-09-19T09:29:59.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:59.104 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:29:59.104 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:59.104 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:59.104 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:59.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:59.105 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:29:59.105 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 89616 2024-09-19T09:29:59.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:59.105 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-19T09:29:59.105 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-19T09:29:59.105 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-19T09:29:59.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:59.105 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-19T09:29:59.106 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:29:59.106 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-19T09:29:59.106 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:29:59.106 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:29:59.106 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:29:59.106 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:29:59.106 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:29:59.106 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:29:59.106 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:29:59.106 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:29:59.106 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:29:59.106 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:29:59.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:29:59.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:29:59.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:29:59.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-19T09:29:59.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:29:59.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-19T09:29:59.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:29:59.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:29:59.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:29:59.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:29:59.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:29:59.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:29:59.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:29:59.108 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-19T09:29:59.108 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:29:59.108 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:29:59.108 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:29:59.108 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-19T09:29:59.108 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-19T09:29:59.108 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-19T09:29:59.108 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:29:59.108 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:29:59.108 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:29:59.108 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:29:59.108 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148903, 2024-09-19T09:29:59.109 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 29, 2024-09-19T09:29:59.109 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:29:59.109 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-19T09:29:59.109 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-19T09:29:59.109 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-19T09:29:59.109 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:29:59.109 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-19T09:29:59.109 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:29:59.109 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-19T09:29:59.109 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-19T09:29:59.109 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-19T09:29:59.109 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:29:59.109 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:29:59.110 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-19T09:29:59.110 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-19T09:29:59.110 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-19T09:29:59.110 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-19T09:29:59.110 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-19T09:29:59.110 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-19T09:29:59.110 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-19T09:29:59.110 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-19T09:29:59.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:29:59.110 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:29:59.110 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-19T09:29:59.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 88, 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 429531, 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 88, 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 88, 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 429531, 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:29:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:59.112 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:29:59.112 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:29:59.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:29:59 smithi016 ceph-mon[30979]: pgmap v409: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 221 KiB/s wr, 92 op/s 2024-09-19T09:29:59.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:29:59 smithi142 ceph-mon[35032]: pgmap v409: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 221 KiB/s wr, 92 op/s 2024-09-19T09:29:59.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:29:59 smithi097 ceph-mon[34864]: pgmap v409: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 221 KiB/s wr, 92 op/s 2024-09-19T09:29:59.945 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T09:29:59.945 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T09:29:59.945 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-19T09:29:59.945 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-19T09:29:59.945 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T09:29:59.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T09:30:00.205 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:00.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:00 smithi016 ceph-mon[30979]: overall HEALTH_OK 2024-09-19T09:30:00.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:00 smithi142 ceph-mon[35032]: overall HEALTH_OK 2024-09-19T09:30:00.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:00 smithi097 ceph-mon[34864]: overall HEALTH_OK 2024-09-19T09:30:00.725 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:30:00.725 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:30:00.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 356, 2024-09-19T09:30:00.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 349, 2024-09-19T09:30:00.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 554730, 2024-09-19T09:30:00.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 132504, 2024-09-19T09:30:00.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-19T09:30:00.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-19T09:30:00.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.042372142, 2024-09-19T09:30:00.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.015437842, 2024-09-19T09:30:00.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.019270029, 2024-09-19T09:30:00.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.007942567, 2024-09-19T09:30:00.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:00.726 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 349, 2024-09-19T09:30:00.726 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.019540824, 2024-09-19T09:30:00.726 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000055990 2024-09-19T09:30:00.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:00.726 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.726 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.726 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 466944, 2024-09-19T09:30:00.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 49824 2024-09-19T09:30:00.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.727 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:30:00.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 434, 2024-09-19T09:30:00.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 494, 2024-09-19T09:30:00.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 112024, 2024-09-19T09:30:00.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 320327, 2024-09-19T09:30:00.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-19T09:30:00.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-09-19T09:30:00.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.030846604, 2024-09-19T09:30:00.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.010324507, 2024-09-19T09:30:00.727 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.008119707, 2024-09-19T09:30:00.728 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.009621923, 2024-09-19T09:30:00.728 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:00.728 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 492, 2024-09-19T09:30:00.728 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.014829767, 2024-09-19T09:30:00.728 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000030141 2024-09-19T09:30:00.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.728 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:00.728 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.728 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.728 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.728 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:30:00.728 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 182576 2024-09-19T09:30:00.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.729 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:30:00.729 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-19T09:30:00.729 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 6, 2024-09-19T09:30:00.729 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1629, 2024-09-19T09:30:00.729 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 1284, 2024-09-19T09:30:00.729 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-19T09:30:00.729 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-09-19T09:30:00.729 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001358532, 2024-09-19T09:30:00.729 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000050976, 2024-09-19T09:30:00.729 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001103836, 2024-09-19T09:30:00.729 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000103201, 2024-09-19T09:30:00.729 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:00.730 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 4, 2024-09-19T09:30:00.730 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.002123417, 2024-09-19T09:30:00.730 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000530854 2024-09-19T09:30:00.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.730 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:00.730 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.730 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.730 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.730 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:30:00.730 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:30:00.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.730 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-19T09:30:00.731 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-09-19T09:30:00.731 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-19T09:30:00.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.731 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-09-19T09:30:00.731 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-19T09:30:00.731 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-19T09:30:00.731 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1217, 2024-09-19T09:30:00.731 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.027110619, 2024-09-19T09:30:00.731 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000022276 2024-09-19T09:30:00.731 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:30:00.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.731 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-09-19T09:30:00.731 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-19T09:30:00.732 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-19T09:30:00.732 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.732 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.732 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.732 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:30:00.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.732 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-09-19T09:30:00.732 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-09-19T09:30:00.732 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-09-19T09:30:00.732 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-09-19T09:30:00.732 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.732 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.732 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 0, 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 0, 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 0, 2024-09-19T09:30:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-09-19T09:30:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-09-19T09:30:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-09-19T09:30:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-09-19T09:30:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-09-19T09:30:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-09-19T09:30:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-09-19T09:30:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-09-19T09:30:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:30:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-09-19T09:30:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-09-19T09:30:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-09-19T09:30:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-09-19T09:30:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:30:00.735 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-09-19T09:30:00.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.737 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-09-19T09:30:00.737 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-09-19T09:30:00.737 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-09-19T09:30:00.737 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-09-19T09:30:00.737 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-09-19T09:30:00.737 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-09-19T09:30:00.737 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-09-19T09:30:00.737 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-09-19T09:30:00.737 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:30:00.737 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-09-19T09:30:00.737 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-09-19T09:30:00.737 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-09-19T09:30:00.738 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 4, 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-09-19T09:30:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4195426, 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4195285, 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 4 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-09-19T09:30:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 3, 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 3, 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 1, 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 1, 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 0, 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 0, 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 48696, 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215328 2024-09-19T09:30:00.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.742 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-09-19T09:30:00.743 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.744 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.744 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.744 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-09-19T09:30:00.744 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.744 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.744 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.744 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-09-19T09:30:00.744 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.744 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.744 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.745 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.746 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-09-19T09:30:00.746 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.746 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.746 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.746 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-09-19T09:30:00.746 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.746 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.747 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.747 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-09-19T09:30:00.747 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.747 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.747 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.747 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-09-19T09:30:00.747 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.748 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.748 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.748 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-09-19T09:30:00.748 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.748 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.748 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.748 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-09-19T09:30:00.748 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.748 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.748 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-09-19T09:30:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-09-19T09:30:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-09-19T09:30:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.751 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.751 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.751 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-09-19T09:30:00.751 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.751 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.751 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.751 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-09-19T09:30:00.751 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.751 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.751 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.752 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.753 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-09-19T09:30:00.753 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.753 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.753 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.753 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-09-19T09:30:00.753 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.753 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.753 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.753 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:30:00.753 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:30:00.754 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-09-19T09:30:00.754 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.754 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.754 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.754 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:30:00.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.754 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-09-19T09:30:00.754 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 0, 2024-09-19T09:30:00.754 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 1, 2024-09-19T09:30:00.754 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 1, 2024-09-19T09:30:00.754 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2024-09-19T09:30:00.755 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-09-19T09:30:00.755 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-09-19T09:30:00.755 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-09-19T09:30:00.755 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 1006, 2024-09-19T09:30:00.755 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-09-19T09:30:00.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.756 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-19T09:30:00.756 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:30:00.756 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-19T09:30:00.756 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:30:00.756 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:30:00.756 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:30:00.756 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:30:00.756 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:30:00.756 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:30:00.756 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:30:00.756 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:30:00.757 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 197287, 2024-09-19T09:30:00.758 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 51, 2024-09-19T09:30:00.759 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:30:00.759 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-19T09:30:00.759 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-19T09:30:00.759 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-19T09:30:00.759 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:30:00.759 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-19T09:30:00.759 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:30:00.759 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-19T09:30:00.759 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 7592, 2024-09-19T09:30:00.759 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 156, 2024-09-19T09:30:00.759 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:30:00.759 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:30:00.759 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-19T09:30:00.760 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-19T09:30:00.760 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 8512, 2024-09-19T09:30:00.760 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 13, 2024-09-19T09:30:00.760 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-19T09:30:00.760 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-19T09:30:00.760 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-19T09:30:00.760 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-19T09:30:00.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.760 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-09-19T09:30:00.760 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-09-19T09:30:00.760 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-09-19T09:30:00.760 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 697, 2024-09-19T09:30:00.760 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 697, 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 697, 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.346994376, 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000497839 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 697, 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 698 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 697, 2024-09-19T09:30:00.761 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 697, 2024-09-19T09:30:00.762 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-09-19T09:30:00.762 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-09-19T09:30:00.762 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-09-19T09:30:00.762 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 429, 2024-09-19T09:30:00.762 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-09-19T09:30:00.762 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 266, 2024-09-19T09:30:00.762 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-09-19T09:30:00.762 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-09-19T09:30:00.762 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-09-19T09:30:00.762 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-09-19T09:30:00.762 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-09-19T09:30:00.762 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-09-19T09:30:00.762 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-09-19T09:30:00.763 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:30:00.764 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-09-19T09:30:00.764 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-09-19T09:30:00.764 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-09-19T09:30:00.764 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-09-19T09:30:00.764 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-09-19T09:30:00.764 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 81, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-09-19T09:30:00.765 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 2, 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 6, 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 6, 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-09-19T09:30:00.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:30:00.767 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-19T09:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-19T09:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-19T09:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 790, 2024-09-19T09:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 589600, 2024-09-19T09:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 790, 2024-09-19T09:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-19T09:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-19T09:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 790, 2024-09-19T09:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 589600, 2024-09-19T09:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-19T09:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.769 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.769 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:30:00.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.769 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-09-19T09:30:00.769 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-19T09:30:00.769 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-19T09:30:00.769 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-19T09:30:00.769 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-19T09:30:00.769 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-19T09:30:00.769 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:00.769 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:00.769 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 697, 2024-09-19T09:30:00.770 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 1127, 2024-09-19T09:30:00.770 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 697, 2024-09-19T09:30:00.770 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1127, 2024-09-19T09:30:00.770 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-19T09:30:00.770 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.770 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.770 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.770 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:30:00.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.770 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-09-19T09:30:00.770 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-19T09:30:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-09-19T09:30:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-19T09:30:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-19T09:30:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-19T09:30:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 697, 2024-09-19T09:30:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 697, 2024-09-19T09:30:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 697, 2024-09-19T09:30:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 697, 2024-09-19T09:30:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-19T09:30:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.772 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.772 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:30:00.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.772 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:30:00.772 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-19T09:30:00.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-19T09:30:00.772 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-19T09:30:00.772 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-19T09:30:00.772 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-19T09:30:00.772 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:00.773 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:00.773 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-19T09:30:00.773 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-19T09:30:00.773 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-19T09:30:00.773 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-19T09:30:00.773 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-19T09:30:00.773 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.773 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.773 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.773 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:30:00.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:00.773 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x5632a8d02b40": { 2024-09-19T09:30:00.773 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:00.774 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:00.775 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:30:00.775 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:30:00.775 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:30:01.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:01 smithi142 ceph-mon[35032]: pgmap v410: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 15 KiB/s wr, 22 op/s 2024-09-19T09:30:01.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:01 smithi016 ceph-mon[30979]: pgmap v410: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 15 KiB/s wr, 22 op/s 2024-09-19T09:30:01.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:01 smithi097 ceph-mon[34864]: pgmap v410: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 15 KiB/s wr, 22 op/s 2024-09-19T09:30:01.596 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-19T09:30:01.597 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-19T09:30:01.597 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-19T09:30:01.597 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-19T09:30:01.597 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T09:30:01.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T09:30:01.858 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:02.324 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-19T09:30:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:30:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 92, 2024-09-19T09:30:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 87, 2024-09-19T09:30:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 445088, 2024-09-19T09:30:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 51840, 2024-09-19T09:30:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-19T09:30:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-19T09:30:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.015542791, 2024-09-19T09:30:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.004948387, 2024-09-19T09:30:02.325 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.012746506, 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 87, 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.008651923, 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000099447 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 445088, 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 51840 2024-09-19T09:30:02.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1, 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 65, 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 160, 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 89968, 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.005578932, 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.003758065, 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001120529, 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:02.327 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 63, 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.003093609, 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000049104 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 89968 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:30:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-09-19T09:30:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-09-19T09:30:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-09-19T09:30:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-09-19T09:30:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-19T09:30:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-19T09:30:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.000419807, 2024-09-19T09:30:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T09:30:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000315129, 2024-09-19T09:30:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T09:30:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:02.329 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-19T09:30:02.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:30:02.331 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:30:02.332 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:30:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:30:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-19T09:30:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:30:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:30:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:30:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-19T09:30:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-19T09:30:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-19T09:30:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:30:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:30:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:30:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:30:02.333 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148823, 2024-09-19T09:30:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 27, 2024-09-19T09:30:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:30:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-19T09:30:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-19T09:30:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-19T09:30:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:30:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-19T09:30:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:30:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-19T09:30:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-19T09:30:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-19T09:30:02.334 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-19T09:30:02.335 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 89, 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 429661, 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 89, 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 89, 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 429661, 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-19T09:30:02.336 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-19T09:30:03.022 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T09:30:03.022 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T09:30:03.022 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-19T09:30:03.022 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-19T09:30:03.022 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T09:30:03.023 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T09:30:03.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:03 smithi016 ceph-mon[30979]: pgmap v411: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-19T09:30:03.283 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:30:03.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:03 smithi142 ceph-mon[35032]: pgmap v411: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-19T09:30:03.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:03 smithi097 ceph-mon[34864]: pgmap v411: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-19T09:30:03.749 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-19T09:30:03.749 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:30:03.749 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 7840, 2024-09-19T09:30:03.749 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 12732, 2024-09-19T09:30:03.749 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 2964529, 2024-09-19T09:30:03.750 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 9235089, 2024-09-19T09:30:03.750 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 16, 2024-09-19T09:30:03.750 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-09-19T09:30:03.750 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.583777355, 2024-09-19T09:30:03.750 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.203620452, 2024-09-19T09:30:03.750 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.307161816, 2024-09-19T09:30:03.750 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.120759111, 2024-09-19T09:30:03.750 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:03.750 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 12728, 2024-09-19T09:30:03.751 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.881700553, 2024-09-19T09:30:03.751 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000069272 2024-09-19T09:30:03.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:03.751 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 14359, 2024-09-19T09:30:03.751 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.002594140, 2024-09-19T09:30:03.751 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000180 2024-09-19T09:30:03.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 564192, 2024-09-19T09:30:03.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 87328 2024-09-19T09:30:03.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.751 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:30:03.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 3381, 2024-09-19T09:30:03.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 3723, 2024-09-19T09:30:03.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 997758, 2024-09-19T09:30:03.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 5799509, 2024-09-19T09:30:03.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 11, 2024-09-19T09:30:03.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 7, 2024-09-19T09:30:03.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.234011359, 2024-09-19T09:30:03.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.088550611, 2024-09-19T09:30:03.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.076260106, 2024-09-19T09:30:03.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.063251943, 2024-09-19T09:30:03.753 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:03.753 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3722, 2024-09-19T09:30:03.753 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.570958754, 2024-09-19T09:30:03.753 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000153401 2024-09-19T09:30:03.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.753 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:03.753 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1250, 2024-09-19T09:30:03.753 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000274050, 2024-09-19T09:30:03.753 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000219 2024-09-19T09:30:03.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.753 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-19T09:30:03.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 187200 2024-09-19T09:30:03.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.754 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:30:03.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 3196, 2024-09-19T09:30:03.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 3752, 2024-09-19T09:30:03.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 781829, 2024-09-19T09:30:03.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 6236002, 2024-09-19T09:30:03.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 68, 2024-09-19T09:30:03.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-09-19T09:30:03.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.260816559, 2024-09-19T09:30:03.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.100945677, 2024-09-19T09:30:03.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.107495430, 2024-09-19T09:30:03.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.047536212, 2024-09-19T09:30:03.755 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:03.755 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3753, 2024-09-19T09:30:03.755 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.435296226, 2024-09-19T09:30:03.755 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000115986 2024-09-19T09:30:03.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.755 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:03.755 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2552, 2024-09-19T09:30:03.755 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000518953, 2024-09-19T09:30:03.756 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000203 2024-09-19T09:30:03.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.756 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:30:03.756 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:30:03.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.757 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-09-19T09:30:03.757 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-09-19T09:30:03.757 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-09-19T09:30:03.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.757 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-09-19T09:30:03.757 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-19T09:30:03.757 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-19T09:30:03.757 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4550, 2024-09-19T09:30:03.757 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.134123359, 2024-09-19T09:30:03.757 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000469038 2024-09-19T09:30:03.758 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:03.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.758 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-09-19T09:30:03.758 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-19T09:30:03.758 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-19T09:30:03.758 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5630, 2024-09-19T09:30:03.758 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.185385336, 2024-09-19T09:30:03.758 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000032928 2024-09-19T09:30:03.758 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:03.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.759 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-09-19T09:30:03.759 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 2477, 2024-09-19T09:30:03.759 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 2477, 2024-09-19T09:30:03.759 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-09-19T09:30:03.759 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2477, 2024-09-19T09:30:03.759 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 33.731659702, 2024-09-19T09:30:03.759 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.013617949 2024-09-19T09:30:03.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.759 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-09-19T09:30:03.759 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-09-19T09:30:03.759 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-09-19T09:30:03.759 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-09-19T09:30:03.760 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 77, 2024-09-19T09:30:03.760 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-09-19T09:30:03.760 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-09-19T09:30:03.760 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 213, 2024-09-19T09:30:03.760 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 5, 2024-09-19T09:30:03.760 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 208, 2024-09-19T09:30:03.760 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-09-19T09:30:03.760 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 205, 2024-09-19T09:30:03.760 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-09-19T09:30:03.760 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 2, 2024-09-19T09:30:03.760 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 2, 2024-09-19T09:30:03.760 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 2, 2024-09-19T09:30:03.760 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 2525, 2024-09-19T09:30:03.761 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 2500, 2024-09-19T09:30:03.761 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-09-19T09:30:03.761 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-09-19T09:30:03.761 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:30:03.761 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-09-19T09:30:03.761 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-09-19T09:30:03.761 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 146, 2024-09-19T09:30:03.761 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-09-19T09:30:03.761 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-09-19T09:30:03.761 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-09-19T09:30:03.761 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-09-19T09:30:03.761 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 36, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 4, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 234, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 36, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 24, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 3, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 21, 2024-09-19T09:30:03.762 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 12, 2024-09-19T09:30:03.763 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 2, 2024-09-19T09:30:03.763 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 44, 2024-09-19T09:30:03.763 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 3653, 2024-09-19T09:30:03.763 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:30:03.763 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:30:03.763 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:30:03.763 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 2461, 2024-09-19T09:30:03.763 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 1212, 2024-09-19T09:30:03.763 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 14, 2024-09-19T09:30:03.763 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 1244 2024-09-19T09:30:03.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.764 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-09-19T09:30:03.764 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 31, 2024-09-19T09:30:03.764 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-09-19T09:30:03.764 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-09-19T09:30:03.764 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-09-19T09:30:03.764 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 52, 2024-09-19T09:30:03.764 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-09-19T09:30:03.764 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-09-19T09:30:03.764 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:30:03.765 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 1212, 2024-09-19T09:30:03.765 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 1212, 2024-09-19T09:30:03.765 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-09-19T09:30:03.765 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-09-19T09:30:03.765 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-09-19T09:30:03.765 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:30:03.765 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:30:03.765 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-09-19T09:30:03.765 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-09-19T09:30:03.765 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 3, 2024-09-19T09:30:03.765 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 38, 2024-09-19T09:30:03.766 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 3, 2024-09-19T09:30:03.766 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-09-19T09:30:03.766 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-09-19T09:30:03.766 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:30:03.766 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-09-19T09:30:03.766 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-09-19T09:30:03.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.766 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-09-19T09:30:03.766 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-09-19T09:30:03.767 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 6124, 2024-09-19T09:30:03.767 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 3185, 2024-09-19T09:30:03.767 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 3185, 2024-09-19T09:30:03.767 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 2939, 2024-09-19T09:30:03.767 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-09-19T09:30:03.767 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 3097, 2024-09-19T09:30:03.767 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 19, 2024-09-19T09:30:03.767 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 14, 2024-09-19T09:30:03.767 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 14, 2024-09-19T09:30:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 5, 2024-09-19T09:30:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 1, 2024-09-19T09:30:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-09-19T09:30:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 4, 2024-09-19T09:30:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 11037950, 2024-09-19T09:30:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 16789743, 2024-09-19T09:30:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-09-19T09:30:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-09-19T09:30:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2456, 2024-09-19T09:30:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3.835974932, 2024-09-19T09:30:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001561879 2024-09-19T09:30:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-09-19T09:30:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-09-19T09:30:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 23, 2024-09-19T09:30:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 1238, 2024-09-19T09:30:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 1215, 2024-09-19T09:30:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 18, 2024-09-19T09:30:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 24, 2024-09-19T09:30:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 6, 2024-09-19T09:30:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 213, 2024-09-19T09:30:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 1243, 2024-09-19T09:30:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 1030, 2024-09-19T09:30:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 2, 2024-09-19T09:30:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 1226, 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 1224, 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 61952, 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 231712 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 2744, 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 2477, 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 10, 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1210, 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.067793103, 2024-09-19T09:30:03.770 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000882473 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 18, 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.010030288, 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000557238 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.771 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.772 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-09-19T09:30:03.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8, 2024-09-19T09:30:03.772 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.003112799, 2024-09-19T09:30:03.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000389099 2024-09-19T09:30:03.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.772 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-09-19T09:30:03.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.772 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.773 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-09-19T09:30:03.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-09-19T09:30:03.773 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000347085, 2024-09-19T09:30:03.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000347085 2024-09-19T09:30:03.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.773 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-09-19T09:30:03.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.773 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.773 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-09-19T09:30:03.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.774 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.774 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.774 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-09-19T09:30:03.774 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.774 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.774 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.774 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-09-19T09:30:03.774 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.775 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.775 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.775 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-09-19T09:30:03.776 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 12, 2024-09-19T09:30:03.776 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 17.610520316, 2024-09-19T09:30:03.776 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 1.467543359 2024-09-19T09:30:03.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.776 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-09-19T09:30:03.776 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.776 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.776 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.777 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-09-19T09:30:03.777 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.777 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.777 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.777 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-09-19T09:30:03.777 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-09-19T09:30:03.777 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000192673, 2024-09-19T09:30:03.777 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000192673 2024-09-19T09:30:03.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.778 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-09-19T09:30:03.778 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5, 2024-09-19T09:30:03.778 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.007831654, 2024-09-19T09:30:03.778 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001566330 2024-09-19T09:30:03.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.778 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-09-19T09:30:03.778 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-09-19T09:30:03.778 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.002050209, 2024-09-19T09:30:03.778 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001025104 2024-09-19T09:30:03.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.778 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-09-19T09:30:03.778 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.778 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.779 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.779 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-09-19T09:30:03.779 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.779 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.779 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.779 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-09-19T09:30:03.779 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6, 2024-09-19T09:30:03.779 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.032235952, 2024-09-19T09:30:03.779 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.005372658 2024-09-19T09:30:03.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.780 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-09-19T09:30:03.780 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.780 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.780 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.780 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-09-19T09:30:03.780 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.781 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.781 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.781 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-09-19T09:30:03.781 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-09-19T09:30:03.781 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000897350, 2024-09-19T09:30:03.781 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000897350 2024-09-19T09:30:03.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.781 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-09-19T09:30:03.781 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.781 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.781 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.782 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-09-19T09:30:03.782 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.782 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.782 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.782 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-09-19T09:30:03.782 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.782 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.782 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.782 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-09-19T09:30:03.783 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 7, 2024-09-19T09:30:03.783 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.019594223, 2024-09-19T09:30:03.783 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002799174 2024-09-19T09:30:03.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.783 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-09-19T09:30:03.783 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.783 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.783 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.783 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-09-19T09:30:03.783 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1206, 2024-09-19T09:30:03.783 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 14.977054050, 2024-09-19T09:30:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.012418784 2024-09-19T09:30:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:30:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:30:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-09-19T09:30:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-09-19T09:30:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-09-19T09:30:03.784 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 2, 2024-09-19T09:30:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 1, 2024-09-19T09:30:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-09-19T09:30:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-09-19T09:30:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 2250, 2024-09-19T09:30:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 2250, 2024-09-19T09:30:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 161, 2024-09-19T09:30:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-09-19T09:30:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-09-19T09:30:03.785 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:30:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-09-19T09:30:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:30:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:30:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:30:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:30:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:30:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:30:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:30:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:30:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:30:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:30:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:30:03.786 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:30:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:30:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-09-19T09:30:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:30:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-09-19T09:30:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:30:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:30:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:30:03.787 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:30:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:30:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:30:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:30:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-09-19T09:30:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:30:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:30:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:30:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-09-19T09:30:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-09-19T09:30:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-09-19T09:30:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:30:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:30:03.788 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:30:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:30:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 294853, 2024-09-19T09:30:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 100, 2024-09-19T09:30:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 88, 2024-09-19T09:30:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 1, 2024-09-19T09:30:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-09-19T09:30:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-09-19T09:30:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:30:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-09-19T09:30:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:30:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-09-19T09:30:03.789 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7592, 2024-09-19T09:30:03.790 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 156, 2024-09-19T09:30:03.790 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:30:03.790 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:30:03.790 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-09-19T09:30:03.790 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-09-19T09:30:03.790 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 237314, 2024-09-19T09:30:03.790 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2262, 2024-09-19T09:30:03.790 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-09-19T09:30:03.790 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-09-19T09:30:03.790 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-09-19T09:30:03.790 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-09-19T09:30:03.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.790 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-09-19T09:30:03.791 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-09-19T09:30:03.791 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-09-19T09:30:03.791 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 7011, 2024-09-19T09:30:03.791 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 13065384, 2024-09-19T09:30:03.791 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 32, 2024-09-19T09:30:03.791 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 7011, 2024-09-19T09:30:03.791 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-09-19T09:30:03.791 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 7011, 2024-09-19T09:30:03.791 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 29.330499262, 2024-09-19T09:30:03.791 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.004183497 2024-09-19T09:30:03.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.791 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-09-19T09:30:03.791 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-09-19T09:30:03.792 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 7011, 2024-09-19T09:30:03.792 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 7650 2024-09-19T09:30:03.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.792 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 7011, 2024-09-19T09:30:03.792 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 75, 2024-09-19T09:30:03.792 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 6936, 2024-09-19T09:30:03.792 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-09-19T09:30:03.792 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-09-19T09:30:03.792 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 56, 2024-09-19T09:30:03.792 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 231, 2024-09-19T09:30:03.792 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 36, 2024-09-19T09:30:03.792 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 2489, 2024-09-19T09:30:03.792 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 66, 2024-09-19T09:30:03.793 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-09-19T09:30:03.793 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-09-19T09:30:03.793 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-09-19T09:30:03.793 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-09-19T09:30:03.793 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 4014, 2024-09-19T09:30:03.793 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-09-19T09:30:03.793 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-09-19T09:30:03.793 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-09-19T09:30:03.793 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 39, 2024-09-19T09:30:03.793 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 462, 2024-09-19T09:30:03.793 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:30:03.794 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-09-19T09:30:03.794 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:30:03.794 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-09-19T09:30:03.794 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-09-19T09:30:03.794 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-09-19T09:30:03.794 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:30:03.794 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-09-19T09:30:03.795 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:30:03.795 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-09-19T09:30:03.795 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-09-19T09:30:03.795 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-09-19T09:30:03.795 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-09-19T09:30:03.795 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-09-19T09:30:03.795 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-09-19T09:30:03.795 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-09-19T09:30:03.795 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-09-19T09:30:03.795 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-09-19T09:30:03.795 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-09-19T09:30:03.796 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-09-19T09:30:03.796 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-09-19T09:30:03.796 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-09-19T09:30:03.796 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-09-19T09:30:03.796 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-09-19T09:30:03.796 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-09-19T09:30:03.796 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-09-19T09:30:03.796 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 81, 2024-09-19T09:30:03.796 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-09-19T09:30:03.796 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 2, 2024-09-19T09:30:03.796 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 12, 2024-09-19T09:30:03.796 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 12, 2024-09-19T09:30:03.796 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-09-19T09:30:03.797 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-09-19T09:30:03.797 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 227, 2024-09-19T09:30:03.797 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-09-19T09:30:03.797 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 30 2024-09-19T09:30:03.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.797 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-09-19T09:30:03.798 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-09-19T09:30:03.798 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:30:03.798 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-09-19T09:30:03.798 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-09-19T09:30:03.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.798 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-09-19T09:30:03.798 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 43 2024-09-19T09:30:03.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.798 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-09-19T09:30:03.799 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-09-19T09:30:03.799 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 9, 2024-09-19T09:30:03.799 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-09-19T09:30:03.799 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 4, 2024-09-19T09:30:03.799 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 3498, 2024-09-19T09:30:03.799 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 1212, 2024-09-19T09:30:03.799 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-09-19T09:30:03.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.799 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:30:03.799 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:03.799 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-19T09:30:03.799 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:03.799 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 14413, 2024-09-19T09:30:03.800 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 3430792, 2024-09-19T09:30:03.800 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:03.800 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 14413, 2024-09-19T09:30:03.800 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-19T09:30:03.800 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-19T09:30:03.800 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 14413, 2024-09-19T09:30:03.800 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 3430792, 2024-09-19T09:30:03.800 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:03.800 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.800 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.800 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.800 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:03.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.801 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-09-19T09:30:03.801 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:03.801 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-19T09:30:03.801 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:03.801 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-19T09:30:03.801 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-19T09:30:03.801 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:03.801 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:03.801 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 7011, 2024-09-19T09:30:03.801 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 13187796, 2024-09-19T09:30:03.802 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 7011, 2024-09-19T09:30:03.802 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 13187796, 2024-09-19T09:30:03.802 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:03.802 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.803 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.803 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.803 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:03.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.803 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-09-19T09:30:03.803 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:03.803 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-09-19T09:30:03.803 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:03.803 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-19T09:30:03.803 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-19T09:30:03.804 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:03.804 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:03.804 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 7011, 2024-09-19T09:30:03.804 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 7011, 2024-09-19T09:30:03.804 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 7011, 2024-09-19T09:30:03.804 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 7011, 2024-09-19T09:30:03.804 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:03.804 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.804 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.804 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 1212, 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 122412, 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 1212, 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 33, 2024-09-19T09:30:03.805 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 122412, 2024-09-19T09:30:03.806 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:03.806 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.806 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.806 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.806 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:03.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:03.806 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55f5b0980960": { 2024-09-19T09:30:03.806 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:03.807 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-19T09:30:03.807 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:03.807 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 6124, 2024-09-19T09:30:03.807 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 12595439, 2024-09-19T09:30:03.807 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:03.807 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 6124, 2024-09-19T09:30:03.807 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-19T09:30:03.807 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-19T09:30:03.807 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2456, 2024-09-19T09:30:03.808 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 12595439, 2024-09-19T09:30:03.808 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:03.808 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:03.808 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:03.808 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:03.808 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:03.808 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:03.808 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-19T09:30:04.672 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-19T09:30:04.672 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-19T09:30:04.672 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=31 2024-09-19T09:30:04.672 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-19T09:30:04.672 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T09:30:04.673 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T09:30:04.924 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:30:05.315 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:05 smithi097 ceph-mon[34864]: pgmap v412: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-09-19T09:30:05.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:05 smithi142 ceph-mon[35032]: pgmap v412: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-09-19T09:30:05.418 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-19T09:30:05.418 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:30:05.418 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 713, 2024-09-19T09:30:05.418 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 715, 2024-09-19T09:30:05.418 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 205445, 2024-09-19T09:30:05.418 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 226741, 2024-09-19T09:30:05.418 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2024-09-19T09:30:05.418 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-09-19T09:30:05.418 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.058712208, 2024-09-19T09:30:05.418 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.019305783, 2024-09-19T09:30:05.418 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.015268169, 2024-09-19T09:30:05.419 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.018736219, 2024-09-19T09:30:05.419 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:05.419 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 713, 2024-09-19T09:30:05.419 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.025851105, 2024-09-19T09:30:05.419 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000036256 2024-09-19T09:30:05.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.419 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:05.419 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.419 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.419 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.419 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:30:05.419 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:30:05.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.420 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:30:05.420 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 7, 2024-09-19T09:30:05.420 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 68, 2024-09-19T09:30:05.420 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 3740, 2024-09-19T09:30:05.420 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 185866, 2024-09-19T09:30:05.420 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-09-19T09:30:05.420 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-09-19T09:30:05.420 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.006819853, 2024-09-19T09:30:05.421 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.002726755, 2024-09-19T09:30:05.421 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.003332705, 2024-09-19T09:30:05.421 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000054969, 2024-09-19T09:30:05.421 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:05.421 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 67, 2024-09-19T09:30:05.421 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.008882493, 2024-09-19T09:30:05.421 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000132574 2024-09-19T09:30:05.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.421 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:05.421 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.421 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.421 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.422 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-19T09:30:05.422 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 185168 2024-09-19T09:30:05.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.422 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:30:05.422 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 102, 2024-09-19T09:30:05.422 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 93, 2024-09-19T09:30:05.422 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 490940, 2024-09-19T09:30:05.422 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 52894, 2024-09-19T09:30:05.422 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 3, 2024-09-19T09:30:05.422 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-09-19T09:30:05.423 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.016279177, 2024-09-19T09:30:05.423 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.005988301, 2024-09-19T09:30:05.423 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.015710568, 2024-09-19T09:30:05.423 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000031241, 2024-09-19T09:30:05.423 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:05.423 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 92, 2024-09-19T09:30:05.423 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.007082577, 2024-09-19T09:30:05.423 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000076984 2024-09-19T09:30:05.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.423 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:05.423 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.423 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.423 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.424 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 490672, 2024-09-19T09:30:05.424 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 52576 2024-09-19T09:30:05.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.424 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-09-19T09:30:05.424 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-09-19T09:30:05.424 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-09-19T09:30:05.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.424 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-09-19T09:30:05.425 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-19T09:30:05.425 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-19T09:30:05.425 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1251, 2024-09-19T09:30:05.425 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.029971521, 2024-09-19T09:30:05.425 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000023958 2024-09-19T09:30:05.425 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:05.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.425 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-09-19T09:30:05.425 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-19T09:30:05.425 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-19T09:30:05.425 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.425 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.425 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:05.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.426 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-09-19T09:30:05.426 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-09-19T09:30:05.426 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-09-19T09:30:05.426 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-09-19T09:30:05.426 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.426 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.426 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.427 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-09-19T09:30:05.427 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-09-19T09:30:05.427 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-09-19T09:30:05.427 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-09-19T09:30:05.427 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2024-09-19T09:30:05.427 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-09-19T09:30:05.427 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-09-19T09:30:05.427 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 0, 2024-09-19T09:30:05.427 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-09-19T09:30:05.427 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 0, 2024-09-19T09:30:05.427 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-09-19T09:30:05.428 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 0, 2024-09-19T09:30:05.428 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-09-19T09:30:05.428 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-09-19T09:30:05.428 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-09-19T09:30:05.428 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 1, 2024-09-19T09:30:05.428 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-09-19T09:30:05.428 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-09-19T09:30:05.428 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-09-19T09:30:05.428 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-09-19T09:30:05.428 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:30:05.428 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-09-19T09:30:05.429 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-09-19T09:30:05.429 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-09-19T09:30:05.429 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-09-19T09:30:05.429 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-09-19T09:30:05.429 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-09-19T09:30:05.429 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-09-19T09:30:05.429 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-09-19T09:30:05.429 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-09-19T09:30:05.430 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:30:05.430 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-09-19T09:30:05.430 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-09-19T09:30:05.430 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-09-19T09:30:05.430 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-09-19T09:30:05.430 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:30:05.430 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-09-19T09:30:05.430 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:30:05.430 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:30:05.431 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:30:05.431 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:30:05.431 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:30:05.431 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-09-19T09:30:05.431 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:30:05.431 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:30:05.431 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:30:05.431 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:30:05.431 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:30:05.431 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:30:05.432 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-09-19T09:30:05.432 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:30:05.432 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-09-19T09:30:05.432 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-09-19T09:30:05.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.432 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-09-19T09:30:05.432 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-09-19T09:30:05.432 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-09-19T09:30:05.432 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-09-19T09:30:05.432 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-09-19T09:30:05.432 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-09-19T09:30:05.433 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-09-19T09:30:05.433 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-09-19T09:30:05.433 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:30:05.433 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-09-19T09:30:05.433 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-09-19T09:30:05.433 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-09-19T09:30:05.433 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-09-19T09:30:05.433 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-09-19T09:30:05.433 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:30:05.433 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:30:05.433 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-09-19T09:30:05.433 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-09-19T09:30:05.433 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-09-19T09:30:05.434 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 4, 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4195426, 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 4195285, 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-09-19T09:30:05.435 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.436 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.436 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.436 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 4 2024-09-19T09:30:05.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.436 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-09-19T09:30:05.436 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 3, 2024-09-19T09:30:05.436 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 3, 2024-09-19T09:30:05.436 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 0, 2024-09-19T09:30:05.436 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 1, 2024-09-19T09:30:05.436 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 1, 2024-09-19T09:30:05.436 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 0, 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 0, 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 0, 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 0, 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 48204, 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215328 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-09-19T09:30:05.437 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-09-19T09:30:05.438 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-09-19T09:30:05.438 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-09-19T09:30:05.438 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.438 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.438 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.438 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-09-19T09:30:05.438 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.438 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.438 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.439 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.440 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-09-19T09:30:05.441 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.442 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.442 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.442 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-09-19T09:30:05.442 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.442 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.443 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.443 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-09-19T09:30:05.443 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.443 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.443 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.443 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-09-19T09:30:05.443 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.443 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.443 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.444 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-09-19T09:30:05.444 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.444 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.444 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.444 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-09-19T09:30:05.444 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.444 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.444 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.444 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-09-19T09:30:05.444 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.444 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.445 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.445 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-09-19T09:30:05.445 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.445 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.445 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.445 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-09-19T09:30:05.445 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.445 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.445 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.446 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-09-19T09:30:05.446 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.446 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.446 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.446 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-09-19T09:30:05.446 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.446 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.446 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.446 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-09-19T09:30:05.446 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.446 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.447 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.447 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-09-19T09:30:05.447 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.447 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.447 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.447 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-09-19T09:30:05.447 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.447 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.447 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.447 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.449 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-09-19T09:30:05.449 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.449 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.449 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.449 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-09-19T09:30:05.449 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.449 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.449 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.449 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:30:05.450 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:30:05.450 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-09-19T09:30:05.450 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.450 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.450 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.450 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:05.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.450 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-09-19T09:30:05.450 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 0, 2024-09-19T09:30:05.450 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 1, 2024-09-19T09:30:05.450 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 1, 2024-09-19T09:30:05.450 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-09-19T09:30:05.450 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-09-19T09:30:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-09-19T09:30:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-09-19T09:30:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 1240, 2024-09-19T09:30:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-09-19T09:30:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-09-19T09:30:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:30:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-09-19T09:30:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:30:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:30:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:30:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:30:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:30:05.452 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:30:05.452 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:30:05.452 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:30:05.452 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:30:05.452 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:30:05.452 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:30:05.452 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:30:05.452 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:30:05.452 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-09-19T09:30:05.453 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:30:05.453 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-09-19T09:30:05.453 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:30:05.453 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:30:05.453 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:30:05.453 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:30:05.453 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:30:05.453 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:30:05.453 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:30:05.453 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-09-19T09:30:05.453 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:30:05.453 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:30:05.453 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:30:05.454 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-09-19T09:30:05.454 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-09-19T09:30:05.454 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-09-19T09:30:05.454 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:30:05.454 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:30:05.454 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:30:05.454 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:30:05.454 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 181239, 2024-09-19T09:30:05.454 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 46, 2024-09-19T09:30:05.454 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:30:05.454 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-09-19T09:30:05.455 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-09-19T09:30:05.455 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-09-19T09:30:05.455 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:30:05.455 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-09-19T09:30:05.455 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:30:05.455 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-09-19T09:30:05.455 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7592, 2024-09-19T09:30:05.455 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 156, 2024-09-19T09:30:05.455 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:30:05.455 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:30:05.455 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-09-19T09:30:05.455 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-09-19T09:30:05.455 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 8512, 2024-09-19T09:30:05.456 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 13, 2024-09-19T09:30:05.456 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-09-19T09:30:05.456 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-09-19T09:30:05.456 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-09-19T09:30:05.456 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-09-19T09:30:05.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.456 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-09-19T09:30:05.456 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-09-19T09:30:05.456 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-09-19T09:30:05.456 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 716, 2024-09-19T09:30:05.457 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 21, 2024-09-19T09:30:05.457 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 3, 2024-09-19T09:30:05.457 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 716, 2024-09-19T09:30:05.457 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-09-19T09:30:05.457 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 716, 2024-09-19T09:30:05.457 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.338994224, 2024-09-19T09:30:05.457 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000473455 2024-09-19T09:30:05.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.457 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-09-19T09:30:05.457 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-09-19T09:30:05.458 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 716, 2024-09-19T09:30:05.458 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 717 2024-09-19T09:30:05.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.458 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 716, 2024-09-19T09:30:05.458 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 716, 2024-09-19T09:30:05.458 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2024-09-19T09:30:05.458 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-09-19T09:30:05.458 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-09-19T09:30:05.458 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 441, 2024-09-19T09:30:05.459 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2024-09-19T09:30:05.459 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 273, 2024-09-19T09:30:05.459 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2024-09-19T09:30:05.459 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2024-09-19T09:30:05.459 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-09-19T09:30:05.459 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-09-19T09:30:05.459 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-09-19T09:30:05.459 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-09-19T09:30:05.459 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2024-09-19T09:30:05.459 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-09-19T09:30:05.459 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-09-19T09:30:05.460 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-09-19T09:30:05.460 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 1, 2024-09-19T09:30:05.460 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2024-09-19T09:30:05.460 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:30:05.460 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-09-19T09:30:05.460 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:30:05.460 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-09-19T09:30:05.460 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-09-19T09:30:05.460 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-09-19T09:30:05.460 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:30:05.461 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-09-19T09:30:05.461 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:30:05.461 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-09-19T09:30:05.461 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-09-19T09:30:05.461 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-09-19T09:30:05.461 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-09-19T09:30:05.461 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-09-19T09:30:05.461 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-09-19T09:30:05.461 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-09-19T09:30:05.461 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-09-19T09:30:05.461 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-09-19T09:30:05.462 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-09-19T09:30:05.462 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-09-19T09:30:05.462 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-09-19T09:30:05.462 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-09-19T09:30:05.462 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-09-19T09:30:05.462 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-09-19T09:30:05.462 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-09-19T09:30:05.462 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-09-19T09:30:05.462 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 81, 2024-09-19T09:30:05.462 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-09-19T09:30:05.462 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 2, 2024-09-19T09:30:05.462 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 4, 2024-09-19T09:30:05.462 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 4, 2024-09-19T09:30:05.463 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-09-19T09:30:05.463 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-09-19T09:30:05.463 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2024-09-19T09:30:05.463 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-09-19T09:30:05.463 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2024-09-19T09:30:05.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.463 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-09-19T09:30:05.463 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2024-09-19T09:30:05.464 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:30:05.464 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-09-19T09:30:05.464 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-09-19T09:30:05.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.464 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-09-19T09:30:05.464 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-09-19T09:30:05.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.464 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-09-19T09:30:05.464 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-09-19T09:30:05.464 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:30:05.464 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-09-19T09:30:05.464 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-09-19T09:30:05.465 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-09-19T09:30:05.465 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-09-19T09:30:05.465 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-09-19T09:30:05.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.465 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:30:05.465 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:05.465 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-19T09:30:05.465 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:05.465 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 818, 2024-09-19T09:30:05.466 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 618327, 2024-09-19T09:30:05.466 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:05.466 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 818, 2024-09-19T09:30:05.466 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-19T09:30:05.466 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-19T09:30:05.466 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 818, 2024-09-19T09:30:05.466 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 618327, 2024-09-19T09:30:05.466 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:05.467 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.467 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.467 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.467 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:05.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.467 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-09-19T09:30:05.467 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:05.467 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-19T09:30:05.467 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:05.467 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-19T09:30:05.467 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-19T09:30:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 716, 2024-09-19T09:30:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 1127, 2024-09-19T09:30:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 716, 2024-09-19T09:30:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1127, 2024-09-19T09:30:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.468 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-09-19T09:30:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-09-19T09:30:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-19T09:30:05.469 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-19T09:30:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 716, 2024-09-19T09:30:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 716, 2024-09-19T09:30:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 716, 2024-09-19T09:30:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 716, 2024-09-19T09:30:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:05.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:30:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-19T09:30:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-19T09:30:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-19T09:30:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-19T09:30:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-19T09:30:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-09-19T09:30:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-09-19T09:30:05.471 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55c3e9658b40": { 2024-09-19T09:30:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-19T09:30:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-19T09:30:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-19T09:30:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:05.472 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-19T09:30:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-19T09:30:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-09-19T09:30:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-09-19T09:30:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:05.473 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:05.473 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-19T09:30:05.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:05 smithi016 ceph-mon[30979]: pgmap v412: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-09-19T09:30:06.112 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T09:30:06.112 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T09:30:06.112 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-19T09:30:06.112 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-19T09:30:06.112 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T09:30:06.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T09:30:06.358 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:30:06.871 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-19T09:30:06.871 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:30:06.871 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 171, 2024-09-19T09:30:06.872 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 161, 2024-09-19T09:30:06.872 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 514004, 2024-09-19T09:30:06.872 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 173008, 2024-09-19T09:30:06.872 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 19, 2024-09-19T09:30:06.872 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-09-19T09:30:06.872 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.025069229, 2024-09-19T09:30:06.872 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.006939364, 2024-09-19T09:30:06.872 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.020791844, 2024-09-19T09:30:06.872 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000414457, 2024-09-19T09:30:06.872 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:06.872 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 160, 2024-09-19T09:30:06.872 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.020846266, 2024-09-19T09:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000130289 2024-09-19T09:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 39, 2024-09-19T09:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000006879, 2024-09-19T09:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000176 2024-09-19T09:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 498720, 2024-09-19T09:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 57472 2024-09-19T09:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 54, 2024-09-19T09:30:06.873 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 112, 2024-09-19T09:30:06.874 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 29326, 2024-09-19T09:30:06.874 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 270795, 2024-09-19T09:30:06.874 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 12, 2024-09-19T09:30:06.874 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-09-19T09:30:06.874 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.012681959, 2024-09-19T09:30:06.874 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.004151070, 2024-09-19T09:30:06.874 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.007506923, 2024-09-19T09:30:06.874 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000429117, 2024-09-19T09:30:06.874 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:06.874 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 113, 2024-09-19T09:30:06.874 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.029483791, 2024-09-19T09:30:06.874 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000260918 2024-09-19T09:30:06.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.875 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:06.875 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 48, 2024-09-19T09:30:06.875 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000009537, 2024-09-19T09:30:06.875 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000198 2024-09-19T09:30:06.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.875 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-19T09:30:06.875 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 186608 2024-09-19T09:30:06.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.875 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:30:06.875 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 819, 2024-09-19T09:30:06.875 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 534, 2024-09-19T09:30:06.875 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 244773, 2024-09-19T09:30:06.875 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 149163, 2024-09-19T09:30:06.876 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 8, 2024-09-19T09:30:06.876 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-09-19T09:30:06.876 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.063253940, 2024-09-19T09:30:06.876 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.027956015, 2024-09-19T09:30:06.876 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.040723077, 2024-09-19T09:30:06.876 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.001772103, 2024-09-19T09:30:06.876 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:06.876 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 530, 2024-09-19T09:30:06.876 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.028581238, 2024-09-19T09:30:06.876 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000053926 2024-09-19T09:30:06.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.876 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:06.876 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1300, 2024-09-19T09:30:06.877 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000318938, 2024-09-19T09:30:06.877 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000245 2024-09-19T09:30:06.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.877 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:30:06.877 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:30:06.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.877 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-09-19T09:30:06.877 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-09-19T09:30:06.877 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-09-19T09:30:06.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.877 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-09-19T09:30:06.877 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-19T09:30:06.877 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 51, 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001346208, 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000026396 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000007230, 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000003615 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.878 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-09-19T09:30:06.879 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 3, 2024-09-19T09:30:06.879 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 3, 2024-09-19T09:30:06.879 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-09-19T09:30:06.879 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2024-09-19T09:30:06.879 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 5.544324392, 2024-09-19T09:30:06.879 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 1.848108130 2024-09-19T09:30:06.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.879 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-09-19T09:30:06.879 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-09-19T09:30:06.879 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-09-19T09:30:06.879 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-09-19T09:30:06.879 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 11, 2024-09-19T09:30:06.880 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-09-19T09:30:06.880 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-09-19T09:30:06.880 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 17, 2024-09-19T09:30:06.880 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 1, 2024-09-19T09:30:06.880 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 16, 2024-09-19T09:30:06.880 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-09-19T09:30:06.880 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 13, 2024-09-19T09:30:06.880 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 8, 2024-09-19T09:30:06.880 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 1, 2024-09-19T09:30:06.880 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 1, 2024-09-19T09:30:06.880 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 3, 2024-09-19T09:30:06.880 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 24, 2024-09-19T09:30:06.880 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 14, 2024-09-19T09:30:06.881 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-09-19T09:30:06.881 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 4, 2024-09-19T09:30:06.881 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:30:06.881 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-09-19T09:30:06.881 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 2, 2024-09-19T09:30:06.881 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-09-19T09:30:06.881 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-09-19T09:30:06.881 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-09-19T09:30:06.881 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-09-19T09:30:06.881 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-09-19T09:30:06.881 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-09-19T09:30:06.881 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-09-19T09:30:06.881 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:30:06.882 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-09-19T09:30:06.882 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 4, 2024-09-19T09:30:06.882 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 234, 2024-09-19T09:30:06.882 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-09-19T09:30:06.882 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:30:06.882 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-09-19T09:30:06.882 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:30:06.882 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:30:06.882 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:30:06.882 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:30:06.882 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:30:06.882 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-09-19T09:30:06.882 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:30:06.883 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 2, 2024-09-19T09:30:06.883 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 1, 2024-09-19T09:30:06.883 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:30:06.883 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:30:06.883 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:30:06.883 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 2, 2024-09-19T09:30:06.883 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:30:06.883 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 1, 2024-09-19T09:30:06.883 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-09-19T09:30:06.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.883 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-09-19T09:30:06.883 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-09-19T09:30:06.883 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 20, 2024-09-19T09:30:06.884 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 12, 2024-09-19T09:30:06.884 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 13, 2024-09-19T09:30:06.884 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-09-19T09:30:06.884 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-09-19T09:30:06.884 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-09-19T09:30:06.884 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:30:06.884 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-09-19T09:30:06.884 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-09-19T09:30:06.884 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-09-19T09:30:06.884 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-09-19T09:30:06.884 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-09-19T09:30:06.884 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:30:06.885 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:30:06.885 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-09-19T09:30:06.885 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-09-19T09:30:06.885 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 3, 2024-09-19T09:30:06.885 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 22, 2024-09-19T09:30:06.885 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:30:06.885 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-09-19T09:30:06.885 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-09-19T09:30:06.885 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:30:06.885 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-09-19T09:30:06.885 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-09-19T09:30:06.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 5, 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 5, 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 2, 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-09-19T09:30:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-09-19T09:30:06.887 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-09-19T09:30:06.887 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-09-19T09:30:06.887 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4195734, 2024-09-19T09:30:06.887 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-09-19T09:30:06.887 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-09-19T09:30:06.887 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4, 2024-09-19T09:30:06.887 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.008269860, 2024-09-19T09:30:06.887 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002067465 2024-09-19T09:30:06.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.887 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-09-19T09:30:06.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.887 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-09-19T09:30:06.887 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 21, 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 27, 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 6, 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 20, 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 27, 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 7, 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 17, 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 25, 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 8, 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 1, 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 2, 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 1, 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 52548, 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 223520 2024-09-19T09:30:06.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.889 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-09-19T09:30:06.889 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 5, 2024-09-19T09:30:06.889 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-09-19T09:30:06.889 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 3, 2024-09-19T09:30:06.889 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 7, 2024-09-19T09:30:06.889 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-09-19T09:30:06.889 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-09-19T09:30:06.889 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.889 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.889 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.889 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-09-19T09:30:06.889 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.890 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.890 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.890 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-09-19T09:30:06.890 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.890 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.890 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.890 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-09-19T09:30:06.890 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.890 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.890 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 5.544219695, 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 2.772109847 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000104697, 2024-09-19T09:30:06.891 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000104697 2024-09-19T09:30:06.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.892 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-09-19T09:30:06.892 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.892 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.892 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.892 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-09-19T09:30:06.892 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.892 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.892 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.892 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-09-19T09:30:06.893 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.893 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.893 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.893 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-09-19T09:30:06.893 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.893 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.893 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.893 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-09-19T09:30:06.893 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.893 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-09-19T09:30:06.894 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.895 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.895 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.895 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-09-19T09:30:06.895 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.895 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.895 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.895 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-09-19T09:30:06.895 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.895 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.896 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.896 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-09-19T09:30:06.896 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.896 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.896 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.896 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-09-19T09:30:06.896 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.896 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.897 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.897 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-09-19T09:30:06.897 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.897 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.897 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.897 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-09-19T09:30:06.897 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.897 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.897 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.898 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-09-19T09:30:06.898 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.898 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.898 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.898 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-09-19T09:30:06.898 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.898 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.898 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.898 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-09-19T09:30:06.898 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.898 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-09-19T09:30:06.899 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.900 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.900 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.900 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-09-19T09:30:06.900 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.900 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.900 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.900 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-09-19T09:30:06.900 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.900 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.900 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.901 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:30:06.901 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:30:06.901 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-09-19T09:30:06.901 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.901 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.901 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:06.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.901 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-09-19T09:30:06.901 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-09-19T09:30:06.902 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 2, 2024-09-19T09:30:06.902 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 1, 2024-09-19T09:30:06.902 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-09-19T09:30:06.902 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-09-19T09:30:06.902 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 2, 2024-09-19T09:30:06.902 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 2, 2024-09-19T09:30:06.902 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 143, 2024-09-19T09:30:06.902 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-09-19T09:30:06.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.902 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-09-19T09:30:06.902 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:30:06.902 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-09-19T09:30:06.902 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:30:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:30:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:30:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:30:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:30:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:30:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:30:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:30:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:30:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:30:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:30:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:30:06.903 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:30:06.904 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:30:06.905 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-09-19T09:30:06.905 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-09-19T09:30:06.905 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-09-19T09:30:06.905 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:30:06.905 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:30:06.905 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:30:06.905 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:30:06.905 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 254455, 2024-09-19T09:30:06.905 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 87, 2024-09-19T09:30:06.905 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:30:06.905 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-09-19T09:30:06.905 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-09-19T09:30:06.905 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7592, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 156, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 85493, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 338, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-09-19T09:30:06.906 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-09-19T09:30:06.907 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-09-19T09:30:06.907 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-09-19T09:30:06.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.907 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-09-19T09:30:06.907 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-09-19T09:30:06.907 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-09-19T09:30:06.907 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 43, 2024-09-19T09:30:06.907 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 12448, 2024-09-19T09:30:06.907 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 33, 2024-09-19T09:30:06.907 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 43, 2024-09-19T09:30:06.907 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-09-19T09:30:06.907 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 43, 2024-09-19T09:30:06.908 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.200996583, 2024-09-19T09:30:06.908 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.004674339 2024-09-19T09:30:06.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.908 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-09-19T09:30:06.908 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-09-19T09:30:06.908 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 43, 2024-09-19T09:30:06.908 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 69 2024-09-19T09:30:06.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.908 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 43, 2024-09-19T09:30:06.908 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 0, 2024-09-19T09:30:06.909 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 43, 2024-09-19T09:30:06.909 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-09-19T09:30:06.909 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-09-19T09:30:06.909 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 0, 2024-09-19T09:30:06.909 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 10, 2024-09-19T09:30:06.909 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 0, 2024-09-19T09:30:06.909 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 4, 2024-09-19T09:30:06.909 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 7, 2024-09-19T09:30:06.909 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-09-19T09:30:06.909 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-09-19T09:30:06.909 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-09-19T09:30:06.909 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 6, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 0, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 20, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:30:06.910 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-09-19T09:30:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:30:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-09-19T09:30:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-09-19T09:30:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-09-19T09:30:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-09-19T09:30:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-09-19T09:30:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-09-19T09:30:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-09-19T09:30:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-09-19T09:30:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-09-19T09:30:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-09-19T09:30:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-09-19T09:30:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 81, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 2, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 8, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 8, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 19, 2024-09-19T09:30:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-09-19T09:30:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 3 2024-09-19T09:30:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-09-19T09:30:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-09-19T09:30:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:30:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-09-19T09:30:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-09-19T09:30:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-09-19T09:30:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 25 2024-09-19T09:30:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-09-19T09:30:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-09-19T09:30:06.914 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:30:06.914 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-09-19T09:30:06.914 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-09-19T09:30:06.914 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-09-19T09:30:06.914 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-09-19T09:30:06.914 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-09-19T09:30:06.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.914 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:30:06.914 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:06.914 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-19T09:30:06.915 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:06.915 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 1040, 2024-09-19T09:30:06.915 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 684783, 2024-09-19T09:30:06.915 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:06.915 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 1040, 2024-09-19T09:30:06.915 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-19T09:30:06.915 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-19T09:30:06.915 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 1040, 2024-09-19T09:30:06.915 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 684783, 2024-09-19T09:30:06.915 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:06.915 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.915 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 43, 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 12448, 2024-09-19T09:30:06.916 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 43, 2024-09-19T09:30:06.917 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 12448, 2024-09-19T09:30:06.917 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:06.917 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.917 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.917 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.917 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:06.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.917 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-09-19T09:30:06.917 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:06.917 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-09-19T09:30:06.917 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:06.917 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-19T09:30:06.917 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-19T09:30:06.918 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:06.918 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:06.918 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 43, 2024-09-19T09:30:06.918 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 43, 2024-09-19T09:30:06.918 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 43, 2024-09-19T09:30:06.918 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 43, 2024-09-19T09:30:06.918 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:06.918 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.918 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.918 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.918 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:06.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.918 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.919 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.920 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.920 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:06.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:06.920 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x563b36da6960": { 2024-09-19T09:30:06.920 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:06.920 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-19T09:30:06.920 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:06.920 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 5, 2024-09-19T09:30:06.920 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 1430, 2024-09-19T09:30:06.920 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:06.920 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 5, 2024-09-19T09:30:06.920 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-19T09:30:06.920 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-19T09:30:06.921 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 4, 2024-09-19T09:30:06.921 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1430, 2024-09-19T09:30:06.921 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:06.921 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:06.921 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:06.921 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:06.921 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:06.921 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:06.921 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-19T09:30:07.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:07 smithi142 ceph-mon[35032]: pgmap v413: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T09:30:07.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:07 smithi097 ceph-mon[34864]: pgmap v413: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T09:30:07.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:07 smithi016 ceph-mon[30979]: pgmap v413: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T09:30:07.831 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-19T09:30:07.831 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-19T09:30:07.832 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-19T09:30:07.832 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=20 2024-09-19T09:30:07.832 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T09:30:07.832 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T09:30:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T09:30:08.571 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-19T09:30:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:30:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 803, 2024-09-19T09:30:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 524, 2024-09-19T09:30:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 238113, 2024-09-19T09:30:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 243686, 2024-09-19T09:30:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 18, 2024-09-19T09:30:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 8, 2024-09-19T09:30:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.064801899, 2024-09-19T09:30:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.027452264, 2024-09-19T09:30:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.039733470, 2024-09-19T09:30:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.001706706, 2024-09-19T09:30:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 520, 2024-09-19T09:30:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.032740540, 2024-09-19T09:30:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000062962 2024-09-19T09:30:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1271, 2024-09-19T09:30:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000228881, 2024-09-19T09:30:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000180 2024-09-19T09:30:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:30:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:30:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:30:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 49, 2024-09-19T09:30:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 110, 2024-09-19T09:30:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 26049, 2024-09-19T09:30:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 264581, 2024-09-19T09:30:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 12, 2024-09-19T09:30:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-09-19T09:30:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.011005255, 2024-09-19T09:30:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.004015234, 2024-09-19T09:30:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.005882392, 2024-09-19T09:30:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000377047, 2024-09-19T09:30:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 112, 2024-09-19T09:30:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.020952816, 2024-09-19T09:30:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000187078 2024-09-19T09:30:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 51, 2024-09-19T09:30:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000010605, 2024-09-19T09:30:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000207 2024-09-19T09:30:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:30:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 186864 2024-09-19T09:30:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:30:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 167, 2024-09-19T09:30:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 159, 2024-09-19T09:30:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 509420, 2024-09-19T09:30:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 80699, 2024-09-19T09:30:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 9, 2024-09-19T09:30:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-09-19T09:30:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.024748469, 2024-09-19T09:30:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.008203505, 2024-09-19T09:30:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.022442133, 2024-09-19T09:30:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000179420, 2024-09-19T09:30:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 157, 2024-09-19T09:30:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.018737688, 2024-09-19T09:30:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000119348 2024-09-19T09:30:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 40, 2024-09-19T09:30:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000007795, 2024-09-19T09:30:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000194 2024-09-19T09:30:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 496544, 2024-09-19T09:30:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 61184 2024-09-19T09:30:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-09-19T09:30:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-09-19T09:30:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-09-19T09:30:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-09-19T09:30:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-19T09:30:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-19T09:30:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 38, 2024-09-19T09:30:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001686909, 2024-09-19T09:30:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000044392 2024-09-19T09:30:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-09-19T09:30:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-19T09:30:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-19T09:30:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-09-19T09:30:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000009820, 2024-09-19T09:30:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000004910 2024-09-19T09:30:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-09-19T09:30:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 3, 2024-09-19T09:30:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 1, 2024-09-19T09:30:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-09-19T09:30:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-09-19T09:30:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000183523, 2024-09-19T09:30:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000183523 2024-09-19T09:30:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-09-19T09:30:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 2, 2024-09-19T09:30:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-09-19T09:30:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-09-19T09:30:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 11, 2024-09-19T09:30:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-09-19T09:30:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-09-19T09:30:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 17, 2024-09-19T09:30:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 1, 2024-09-19T09:30:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 16, 2024-09-19T09:30:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-09-19T09:30:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 13, 2024-09-19T09:30:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 7, 2024-09-19T09:30:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-09-19T09:30:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-09-19T09:30:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 3, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 22, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 15, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 1, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 5, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-09-19T09:30:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 4, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 234, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-09-19T09:30:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:30:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:30:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:30:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:30:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:30:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:30:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-09-19T09:30:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:30:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-09-19T09:30:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 1 2024-09-19T09:30:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-09-19T09:30:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-09-19T09:30:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 18, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 11, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 13, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:30:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-09-19T09:30:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-09-19T09:30:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 3, 2024-09-19T09:30:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 22, 2024-09-19T09:30:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:30:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-09-19T09:30:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-09-19T09:30:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:30:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-09-19T09:30:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-09-19T09:30:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-09-19T09:30:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-09-19T09:30:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 5, 2024-09-19T09:30:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-09-19T09:30:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-09-19T09:30:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 5, 2024-09-19T09:30:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-09-19T09:30:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-09-19T09:30:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 2, 2024-09-19T09:30:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-09-19T09:30:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-09-19T09:30:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-09-19T09:30:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-09-19T09:30:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-09-19T09:30:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-09-19T09:30:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-09-19T09:30:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4195734, 2024-09-19T09:30:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-09-19T09:30:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-09-19T09:30:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4, 2024-09-19T09:30:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.472419709, 2024-09-19T09:30:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.118104927 2024-09-19T09:30:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-09-19T09:30:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-09-19T09:30:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 21, 2024-09-19T09:30:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 26, 2024-09-19T09:30:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 5, 2024-09-19T09:30:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 20, 2024-09-19T09:30:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 26, 2024-09-19T09:30:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 6, 2024-09-19T09:30:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 17, 2024-09-19T09:30:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 24, 2024-09-19T09:30:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 7, 2024-09-19T09:30:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-09-19T09:30:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 1, 2024-09-19T09:30:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 1, 2024-09-19T09:30:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 50500, 2024-09-19T09:30:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 223520 2024-09-19T09:30:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-09-19T09:30:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 4, 2024-09-19T09:30:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-09-19T09:30:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 3, 2024-09-19T09:30:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 7, 2024-09-19T09:30:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-09-19T09:30:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-09-19T09:30:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-09-19T09:30:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-09-19T09:30:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000183523, 2024-09-19T09:30:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000183523 2024-09-19T09:30:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-09-19T09:30:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-09-19T09:30:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-09-19T09:30:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-09-19T09:30:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-09-19T09:30:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-09-19T09:30:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.594 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-09-19T09:30:08.594 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.595 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.595 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.595 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-09-19T09:30:08.595 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.595 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.595 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-09-19T09:30:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-09-19T09:30:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-09-19T09:30:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-09-19T09:30:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-09-19T09:30:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-09-19T09:30:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-09-19T09:30:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-09-19T09:30:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-09-19T09:30:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-09-19T09:30:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-09-19T09:30:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-09-19T09:30:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-09-19T09:30:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-09-19T09:30:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-09-19T09:30:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-09-19T09:30:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:30:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:30:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-09-19T09:30:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-09-19T09:30:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-09-19T09:30:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 2, 2024-09-19T09:30:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 1, 2024-09-19T09:30:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-09-19T09:30:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-09-19T09:30:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 1, 2024-09-19T09:30:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 1, 2024-09-19T09:30:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 169, 2024-09-19T09:30:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-09-19T09:30:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-09-19T09:30:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:30:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-09-19T09:30:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-09-19T09:30:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:30:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-09-19T09:30:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:30:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:30:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:30:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:30:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:30:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:30:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:30:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-09-19T09:30:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:30:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:30:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:30:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 230321, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 78, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:30:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7592, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 156, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 85173, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 342, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-09-19T09:30:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 39, 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 11642, 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 33, 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 39, 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 39, 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.671988560, 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.017230475 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-09-19T09:30:08.612 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-09-19T09:30:08.612 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 39, 2024-09-19T09:30:08.612 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 61 2024-09-19T09:30:08.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.612 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 39, 2024-09-19T09:30:08.612 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 0, 2024-09-19T09:30:08.612 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 39, 2024-09-19T09:30:08.612 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-09-19T09:30:08.612 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-09-19T09:30:08.612 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 0, 2024-09-19T09:30:08.612 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 10, 2024-09-19T09:30:08.612 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 0, 2024-09-19T09:30:08.612 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 4, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 6, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 6, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 0, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 20, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-09-19T09:30:08.613 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-09-19T09:30:08.614 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 81, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 2, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 8, 2024-09-19T09:30:08.615 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 8, 2024-09-19T09:30:08.616 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-09-19T09:30:08.616 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-09-19T09:30:08.616 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 14, 2024-09-19T09:30:08.616 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-09-19T09:30:08.616 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 1 2024-09-19T09:30:08.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.616 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-09-19T09:30:08.616 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-09-19T09:30:08.616 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:30:08.616 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-09-19T09:30:08.616 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-09-19T09:30:08.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.616 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-09-19T09:30:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 25 2024-09-19T09:30:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-09-19T09:30:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-09-19T09:30:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:30:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-09-19T09:30:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-09-19T09:30:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-09-19T09:30:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-09-19T09:30:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-09-19T09:30:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:30:08.617 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 1015, 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 672312, 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 1015, 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 1015, 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 672312, 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.618 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:08.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.619 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-09-19T09:30:08.619 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:08.619 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-19T09:30:08.619 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:08.619 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-19T09:30:08.619 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-19T09:30:08.620 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:08.620 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:08.620 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 39, 2024-09-19T09:30:08.620 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 11642, 2024-09-19T09:30:08.620 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 39, 2024-09-19T09:30:08.620 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 11642, 2024-09-19T09:30:08.620 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:08.620 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.620 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.620 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.620 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:08.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.620 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-09-19T09:30:08.621 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:08.621 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-09-19T09:30:08.621 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:08.621 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-19T09:30:08.621 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-19T09:30:08.621 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:08.621 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:08.621 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 39, 2024-09-19T09:30:08.621 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 39, 2024-09-19T09:30:08.621 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 39, 2024-09-19T09:30:08.621 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 39, 2024-09-19T09:30:08.621 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:08.621 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-19T09:30:08.622 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x56173d0e2960": { 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 5, 2024-09-19T09:30:08.623 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 1430, 2024-09-19T09:30:08.624 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:08.624 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 5, 2024-09-19T09:30:08.624 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-19T09:30:08.624 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-19T09:30:08.624 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 4, 2024-09-19T09:30:08.624 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1430, 2024-09-19T09:30:08.624 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-19T09:30:08.624 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-19T09:30:08.624 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-19T09:30:08.624 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-19T09:30:08.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:08.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-19T09:30:08.624 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-19T09:30:09.350 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:09 smithi097 ceph-mon[34864]: pgmap v414: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:30:09.377 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-19T09:30:09.377 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-19T09:30:09.377 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-19T09:30:09.377 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=18 2024-09-19T09:30:09.377 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T09:30:09.377 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T09:30:09.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:09 smithi142 ceph-mon[35032]: pgmap v414: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:30:09.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:09 smithi016 ceph-mon[30979]: pgmap v414: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T09:30:09.634 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:30:10.224 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-19T09:30:10.225 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:30:10.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 569, 2024-09-19T09:30:10.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 569, 2024-09-19T09:30:10.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 4365153, 2024-09-19T09:30:10.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 180128, 2024-09-19T09:30:10.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 6, 2024-09-19T09:30:10.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-09-19T09:30:10.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.055662180, 2024-09-19T09:30:10.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.015666231, 2024-09-19T09:30:10.225 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.018983308, 2024-09-19T09:30:10.226 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.015846433, 2024-09-19T09:30:10.226 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:10.226 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 568, 2024-09-19T09:30:10.226 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.026973959, 2024-09-19T09:30:10.226 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000047489 2024-09-19T09:30:10.226 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.226 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:10.226 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.226 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.226 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.226 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.226 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T09:30:10.226 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:30:10.227 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.227 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:30:10.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 161, 2024-09-19T09:30:10.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 152, 2024-09-19T09:30:10.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 4713785, 2024-09-19T09:30:10.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 71564, 2024-09-19T09:30:10.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-09-19T09:30:10.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-09-19T09:30:10.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.022088127, 2024-09-19T09:30:10.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.007757535, 2024-09-19T09:30:10.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.016600205, 2024-09-19T09:30:10.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.001400340, 2024-09-19T09:30:10.227 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:10.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 151, 2024-09-19T09:30:10.228 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.010307995, 2024-09-19T09:30:10.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000068264 2024-09-19T09:30:10.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.228 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:10.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.228 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.228 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.228 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 490832, 2024-09-19T09:30:10.228 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 52736 2024-09-19T09:30:10.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.228 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 253, 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 316, 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 4256831, 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 266473, 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.029596565, 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.011281830, 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.007722665, 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.007734299, 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 314, 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.013102323, 2024-09-19T09:30:10.229 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000041727 2024-09-19T09:30:10.230 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.230 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:10.230 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.230 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.230 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.230 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.230 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:30:10.230 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 186336 2024-09-19T09:30:10.230 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.230 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-09-19T09:30:10.230 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-09-19T09:30:10.230 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-09-19T09:30:10.230 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.231 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-09-19T09:30:10.231 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-19T09:30:10.231 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-19T09:30:10.231 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1430, 2024-09-19T09:30:10.231 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.039900725, 2024-09-19T09:30:10.231 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000027902 2024-09-19T09:30:10.231 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:10.231 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.231 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-09-19T09:30:10.231 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-19T09:30:10.231 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-19T09:30:10.231 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.231 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-09-19T09:30:10.232 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 1233, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 337, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 896, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 751, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 2, 2024-09-19T09:30:10.233 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-09-19T09:30:10.234 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-09-19T09:30:10.234 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-09-19T09:30:10.234 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-09-19T09:30:10.234 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:30:10.234 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-09-19T09:30:10.234 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-09-19T09:30:10.234 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-09-19T09:30:10.234 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-09-19T09:30:10.234 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-09-19T09:30:10.234 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-09-19T09:30:10.234 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-09-19T09:30:10.234 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-09-19T09:30:10.234 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 234, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-09-19T09:30:10.235 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-09-19T09:30:10.236 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-09-19T09:30:10.237 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-09-19T09:30:10.237 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-09-19T09:30:10.237 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-09-19T09:30:10.237 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-09-19T09:30:10.237 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:30:10.237 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-09-19T09:30:10.237 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-09-19T09:30:10.237 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-09-19T09:30:10.237 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-09-19T09:30:10.238 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-09-19T09:30:10.238 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:30:10.238 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:30:10.238 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-09-19T09:30:10.238 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-09-19T09:30:10.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:30:10.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:30:10.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:30:10.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-09-19T09:30:10.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-09-19T09:30:10.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:30:10.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-09-19T09:30:10.238 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-09-19T09:30:10.239 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.239 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-09-19T09:30:10.239 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-09-19T09:30:10.239 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-09-19T09:30:10.239 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-09-19T09:30:10.239 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-09-19T09:30:10.239 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 6124, 2024-09-19T09:30:10.239 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-09-19T09:30:10.239 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-09-19T09:30:10.239 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-09-19T09:30:10.239 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-09-19T09:30:10.239 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-09-19T09:30:10.239 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 5, 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 12, 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 11037950, 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 16789743, 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 16787246, 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 6124 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.240 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-09-19T09:30:10.241 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 21, 2024-09-19T09:30:10.241 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 1233, 2024-09-19T09:30:10.241 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 1212, 2024-09-19T09:30:10.241 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 16, 2024-09-19T09:30:10.241 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 22, 2024-09-19T09:30:10.241 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 6, 2024-09-19T09:30:10.241 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 1233, 2024-09-19T09:30:10.241 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 1240, 2024-09-19T09:30:10.241 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 7, 2024-09-19T09:30:10.241 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-09-19T09:30:10.241 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-09-19T09:30:10.241 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-09-19T09:30:10.241 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 50172, 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 215328 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.242 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.245 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-09-19T09:30:10.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.245 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.245 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.245 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-09-19T09:30:10.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.245 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.245 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.245 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-09-19T09:30:10.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.246 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.246 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.246 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.246 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-09-19T09:30:10.246 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.246 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.246 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.246 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.246 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-09-19T09:30:10.247 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.247 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.247 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.247 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.247 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-09-19T09:30:10.247 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.247 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.247 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.247 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.247 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-09-19T09:30:10.247 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.247 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.248 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-09-19T09:30:10.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.248 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.248 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-09-19T09:30:10.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.248 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.248 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-09-19T09:30:10.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.249 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.249 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-09-19T09:30:10.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.249 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.249 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-09-19T09:30:10.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.250 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.250 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.250 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-09-19T09:30:10.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.250 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.250 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.250 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-09-19T09:30:10.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.251 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.251 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.251 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-09-19T09:30:10.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.251 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.251 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.251 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-09-19T09:30:10.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.251 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.251 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.252 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-09-19T09:30:10.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.252 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.252 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.252 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-09-19T09:30:10.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.252 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.252 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.252 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-09-19T09:30:10.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.252 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.253 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.253 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-09-19T09:30:10.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.253 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.253 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.253 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-09-19T09:30:10.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.253 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.253 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.253 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-09-19T09:30:10.254 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.254 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.254 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.254 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.254 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:30:10.254 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:30:10.254 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-09-19T09:30:10.254 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.254 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.254 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.254 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:10.254 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 2, 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 1405, 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:30:10.255 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-09-19T09:30:10.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:30:10.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:30:10.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:30:10.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:30:10.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:30:10.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:30:10.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:30:10.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:30:10.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:30:10.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:30:10.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:30:10.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:30:10.256 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:30:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-09-19T09:30:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:30:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-09-19T09:30:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:30:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:30:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:30:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:30:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:30:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:30:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:30:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-09-19T09:30:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:30:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:30:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:30:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-09-19T09:30:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-09-19T09:30:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-09-19T09:30:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:30:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:30:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:30:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:30:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 189279, 2024-09-19T09:30:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 49, 2024-09-19T09:30:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:30:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-09-19T09:30:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-09-19T09:30:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-09-19T09:30:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:30:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-09-19T09:30:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:30:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-09-19T09:30:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7592, 2024-09-19T09:30:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 156, 2024-09-19T09:30:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:30:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:30:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-09-19T09:30:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-09-19T09:30:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 935418, 2024-09-19T09:30:10.259 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 5199, 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 876, 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 26, 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 4, 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 876, 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-09-19T09:30:10.260 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 876, 2024-09-19T09:30:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.460999112, 2024-09-19T09:30:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000526254 2024-09-19T09:30:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-09-19T09:30:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-09-19T09:30:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 876, 2024-09-19T09:30:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 877 2024-09-19T09:30:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 876, 2024-09-19T09:30:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 876, 2024-09-19T09:30:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-09-19T09:30:10.261 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-09-19T09:30:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-09-19T09:30:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 543, 2024-09-19T09:30:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-09-19T09:30:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 330, 2024-09-19T09:30:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-09-19T09:30:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-09-19T09:30:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-09-19T09:30:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-09-19T09:30:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-09-19T09:30:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-09-19T09:30:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-09-19T09:30:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-09-19T09:30:10.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 2, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-09-19T09:30:10.263 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-09-19T09:30:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-09-19T09:30:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-09-19T09:30:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-09-19T09:30:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-09-19T09:30:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-09-19T09:30:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-09-19T09:30:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-09-19T09:30:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-09-19T09:30:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-09-19T09:30:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-09-19T09:30:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-09-19T09:30:10.264 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-09-19T09:30:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-09-19T09:30:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-09-19T09:30:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-09-19T09:30:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 81, 2024-09-19T09:30:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-09-19T09:30:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 2, 2024-09-19T09:30:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 5, 2024-09-19T09:30:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 5, 2024-09-19T09:30:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-09-19T09:30:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-09-19T09:30:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-09-19T09:30:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-09-19T09:30:10.265 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-09-19T09:30:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-09-19T09:30:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-09-19T09:30:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:30:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-09-19T09:30:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-09-19T09:30:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-09-19T09:30:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-09-19T09:30:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-09-19T09:30:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-09-19T09:30:10.266 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 0, 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 979, 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 13239435, 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:10.267 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 979, 2024-09-19T09:30:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-19T09:30:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-19T09:30:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 979, 2024-09-19T09:30:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 13239435, 2024-09-19T09:30:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-09-19T09:30:10.268 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-19T09:30:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 876, 2024-09-19T09:30:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 12595809, 2024-09-19T09:30:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 876, 2024-09-19T09:30:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 12595809, 2024-09-19T09:30:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.269 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 876, 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 876, 2024-09-19T09:30:10.270 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 876, 2024-09-19T09:30:10.271 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 876, 2024-09-19T09:30:10.271 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:10.271 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.271 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.271 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.271 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:10.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.271 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:30:10.271 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:10.271 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-19T09:30:10.271 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:10.271 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:10.271 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:10.272 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:10.272 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:10.272 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-19T09:30:10.272 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-19T09:30:10.272 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-09-19T09:30:10.272 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-09-19T09:30:10.272 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:10.272 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.272 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.272 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.272 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:10.272 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:10.273 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x5639a03d4b40": { 2024-09-19T09:30:10.273 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:10.273 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-19T09:30:10.273 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:10.273 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:10.273 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:10.273 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:10.273 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:10.273 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-19T09:30:10.273 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-19T09:30:10.273 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-09-19T09:30:10.273 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-09-19T09:30:10.273 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:10.274 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:10.274 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:10.274 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:10.274 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:10.274 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:10.274 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-19T09:30:11.121 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-19T09:30:11.121 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-19T09:30:11.121 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-19T09:30:11.121 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-19T09:30:11.121 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T09:30:11.121 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T09:30:11.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:11 smithi142 ceph-mon[35032]: pgmap v415: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T09:30:11.378 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:30:11.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:11 smithi097 ceph-mon[34864]: pgmap v415: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T09:30:11.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:11 smithi016 ceph-mon[30979]: pgmap v415: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-19T09:30:11.898 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-19T09:30:11.898 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:30:11.899 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 280, 2024-09-19T09:30:11.899 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 282, 2024-09-19T09:30:11.899 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 75365, 2024-09-19T09:30:11.899 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 89104, 2024-09-19T09:30:11.899 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 6, 2024-09-19T09:30:11.899 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-09-19T09:30:11.899 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.024651158, 2024-09-19T09:30:11.899 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.007246263, 2024-09-19T09:30:11.899 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.007127654, 2024-09-19T09:30:11.899 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.007170552, 2024-09-19T09:30:11.899 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:11.900 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 280, 2024-09-19T09:30:11.900 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.012239444, 2024-09-19T09:30:11.900 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000043712 2024-09-19T09:30:11.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.900 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:11.900 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.900 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.900 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.900 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:30:11.900 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:30:11.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.900 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:30:11.901 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 288, 2024-09-19T09:30:11.901 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 277, 2024-09-19T09:30:11.901 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 538762, 2024-09-19T09:30:11.901 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 111545, 2024-09-19T09:30:11.901 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-09-19T09:30:11.901 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-09-19T09:30:11.901 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.033643039, 2024-09-19T09:30:11.901 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.010262349, 2024-09-19T09:30:11.901 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.020989561, 2024-09-19T09:30:11.901 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.006048056, 2024-09-19T09:30:11.901 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:11.901 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 277, 2024-09-19T09:30:11.902 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.016636798, 2024-09-19T09:30:11.902 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000060060 2024-09-19T09:30:11.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.902 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:11.902 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.902 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.902 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.902 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 493968, 2024-09-19T09:30:11.902 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 53664 2024-09-19T09:30:11.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.902 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:30:11.903 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 274, 2024-09-19T09:30:11.903 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 338, 2024-09-19T09:30:11.903 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 91888, 2024-09-19T09:30:11.903 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 274942, 2024-09-19T09:30:11.903 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-09-19T09:30:11.903 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-09-19T09:30:11.903 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.033108016, 2024-09-19T09:30:11.903 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.012035743, 2024-09-19T09:30:11.903 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.008615514, 2024-09-19T09:30:11.903 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.008865299, 2024-09-19T09:30:11.903 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:11.903 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 336, 2024-09-19T09:30:11.903 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.018603574, 2024-09-19T09:30:11.904 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000055367 2024-09-19T09:30:11.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.904 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:11.904 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.904 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.904 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.904 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T09:30:11.904 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 188128 2024-09-19T09:30:11.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.904 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-09-19T09:30:11.904 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-09-19T09:30:11.905 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-09-19T09:30:11.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.905 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-09-19T09:30:11.905 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-19T09:30:11.905 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-19T09:30:11.905 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1257, 2024-09-19T09:30:11.905 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.034737296, 2024-09-19T09:30:11.905 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000027635 2024-09-19T09:30:11.905 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-09-19T09:30:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-19T09:30:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-19T09:30:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-09-19T09:30:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-09-19T09:30:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 0, 2024-09-19T09:30:11.907 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 0, 2024-09-19T09:30:11.908 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 0, 2024-09-19T09:30:11.908 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-09-19T09:30:11.908 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 0, 2024-09-19T09:30:11.908 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-09-19T09:30:11.908 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-09-19T09:30:11.908 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-09-19T09:30:11.908 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 1, 2024-09-19T09:30:11.908 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-09-19T09:30:11.908 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-09-19T09:30:11.908 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-09-19T09:30:11.908 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-09-19T09:30:11.908 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:30:11.908 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-09-19T09:30:11.909 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-09-19T09:30:11.909 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-09-19T09:30:11.909 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-09-19T09:30:11.909 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-09-19T09:30:11.909 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-09-19T09:30:11.909 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-09-19T09:30:11.909 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-09-19T09:30:11.909 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-09-19T09:30:11.909 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:30:11.909 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-09-19T09:30:11.909 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 0, 2024-09-19T09:30:11.909 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 0, 2024-09-19T09:30:11.910 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-09-19T09:30:11.910 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:30:11.910 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-09-19T09:30:11.910 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:30:11.910 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:30:11.910 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:30:11.910 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:30:11.910 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:30:11.910 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-09-19T09:30:11.910 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:30:11.910 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:30:11.910 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:30:11.910 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-09-19T09:30:11.911 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-09-19T09:30:11.912 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-09-19T09:30:11.912 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:30:11.912 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-09-19T09:30:11.912 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-09-19T09:30:11.912 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-09-19T09:30:11.912 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-09-19T09:30:11.912 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-09-19T09:30:11.912 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:30:11.912 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:30:11.912 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-09-19T09:30:11.912 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-09-19T09:30:11.913 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-09-19T09:30:11.914 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 5, 2024-09-19T09:30:11.914 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-09-19T09:30:11.914 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-09-19T09:30:11.914 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-09-19T09:30:11.914 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-09-19T09:30:11.914 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-09-19T09:30:11.914 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 2, 2024-09-19T09:30:11.914 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 2, 2024-09-19T09:30:11.914 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-09-19T09:30:11.914 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-09-19T09:30:11.914 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-09-19T09:30:11.914 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 4195734, 2024-09-19T09:30:11.915 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 4195426, 2024-09-19T09:30:11.915 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-09-19T09:30:11.915 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.915 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.915 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.915 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.915 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 5 2024-09-19T09:30:11.915 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.915 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-09-19T09:30:11.915 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 3, 2024-09-19T09:30:11.915 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 3, 2024-09-19T09:30:11.916 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 0, 2024-09-19T09:30:11.916 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 1, 2024-09-19T09:30:11.916 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 1, 2024-09-19T09:30:11.916 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 0, 2024-09-19T09:30:11.916 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 0, 2024-09-19T09:30:11.916 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 0, 2024-09-19T09:30:11.916 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 0, 2024-09-19T09:30:11.916 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-09-19T09:30:11.916 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-09-19T09:30:11.916 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 50568, 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 215328 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.918 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-09-19T09:30:11.918 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.918 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.918 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.918 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-09-19T09:30:11.918 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.918 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.918 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.918 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-09-19T09:30:11.918 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.918 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-09-19T09:30:11.919 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.920 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.920 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.920 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-09-19T09:30:11.920 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.920 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.920 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.921 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-09-19T09:30:11.921 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.921 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.921 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.921 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-09-19T09:30:11.921 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.921 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.921 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.921 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-09-19T09:30:11.921 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.921 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.922 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.922 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.922 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-09-19T09:30:11.922 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.922 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.923 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.923 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-09-19T09:30:11.923 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.923 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.923 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.923 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-09-19T09:30:11.923 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.923 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.923 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.924 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-09-19T09:30:11.924 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.924 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.924 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.924 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-09-19T09:30:11.924 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.924 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.924 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.924 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-09-19T09:30:11.924 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.924 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.925 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.925 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.925 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-09-19T09:30:11.925 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.925 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.925 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.925 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.925 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-09-19T09:30:11.925 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.925 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.926 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.926 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-09-19T09:30:11.926 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.926 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.926 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.926 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-09-19T09:30:11.926 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.926 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.926 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.926 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-09-19T09:30:11.927 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.927 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.927 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.927 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-09-19T09:30:11.927 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.927 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.927 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.927 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-09-19T09:30:11.927 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.927 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.927 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.928 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.929 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.929 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-09-19T09:30:11.929 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.929 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.929 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.929 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-09-19T09:30:11.929 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.929 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.929 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.929 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:30:11.930 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:30:11.930 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-09-19T09:30:11.930 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.930 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.930 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.930 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:11.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.930 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-09-19T09:30:11.930 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-09-19T09:30:11.930 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 2, 2024-09-19T09:30:11.930 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-09-19T09:30:11.931 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-09-19T09:30:11.931 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-09-19T09:30:11.931 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-09-19T09:30:11.931 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-09-19T09:30:11.931 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 1408, 2024-09-19T09:30:11.931 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-09-19T09:30:11.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.931 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-09-19T09:30:11.931 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:30:11.931 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-09-19T09:30:11.931 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:30:11.932 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:30:11.932 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:30:11.932 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:30:11.932 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:30:11.932 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:30:11.932 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:30:11.932 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:30:11.932 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:30:11.932 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:30:11.932 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:30:11.932 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:30:11.932 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:30:11.932 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-09-19T09:30:11.933 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:30:11.933 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-09-19T09:30:11.933 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:30:11.933 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:30:11.933 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:30:11.933 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:30:11.933 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:30:11.933 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:30:11.933 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:30:11.933 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-09-19T09:30:11.933 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:30:11.933 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:30:11.933 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 189239, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 48, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-09-19T09:30:11.934 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7592, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 156, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 8512, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 13, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-09-19T09:30:11.935 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-09-19T09:30:11.936 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-09-19T09:30:11.936 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.936 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-09-19T09:30:11.936 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-09-19T09:30:11.936 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-09-19T09:30:11.936 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 735, 2024-09-19T09:30:11.936 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 21, 2024-09-19T09:30:11.936 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 3, 2024-09-19T09:30:11.936 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 735, 2024-09-19T09:30:11.936 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-09-19T09:30:11.936 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 735, 2024-09-19T09:30:11.936 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.346999320, 2024-09-19T09:30:11.937 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000472107 2024-09-19T09:30:11.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.937 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-09-19T09:30:11.937 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-09-19T09:30:11.937 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 735, 2024-09-19T09:30:11.937 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 736 2024-09-19T09:30:11.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.937 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 735, 2024-09-19T09:30:11.937 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 735, 2024-09-19T09:30:11.937 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-09-19T09:30:11.937 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-09-19T09:30:11.937 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-09-19T09:30:11.937 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 453, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 280, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2024-09-19T09:30:11.938 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-09-19T09:30:11.939 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:30:11.939 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-09-19T09:30:11.939 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:30:11.939 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-09-19T09:30:11.939 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-09-19T09:30:11.939 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-09-19T09:30:11.939 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:30:11.939 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-09-19T09:30:11.939 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:30:11.939 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-09-19T09:30:11.939 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-09-19T09:30:11.939 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-09-19T09:30:11.939 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-09-19T09:30:11.940 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-09-19T09:30:11.940 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-09-19T09:30:11.940 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-09-19T09:30:11.940 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-09-19T09:30:11.940 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-09-19T09:30:11.940 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-09-19T09:30:11.940 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-09-19T09:30:11.940 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-09-19T09:30:11.940 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-09-19T09:30:11.940 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-09-19T09:30:11.940 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-09-19T09:30:11.940 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-09-19T09:30:11.940 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-09-19T09:30:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 81, 2024-09-19T09:30:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-09-19T09:30:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 2, 2024-09-19T09:30:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 5, 2024-09-19T09:30:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 5, 2024-09-19T09:30:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-09-19T09:30:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-09-19T09:30:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-09-19T09:30:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-09-19T09:30:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-09-19T09:30:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-09-19T09:30:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-09-19T09:30:11.942 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:30:11.942 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-09-19T09:30:11.942 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-09-19T09:30:11.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.942 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-09-19T09:30:11.942 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-09-19T09:30:11.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.942 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-09-19T09:30:11.942 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-09-19T09:30:11.942 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:30:11.942 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-09-19T09:30:11.942 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-09-19T09:30:11.942 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 0, 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 838, 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 622365, 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 838, 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-19T09:30:11.943 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 838, 2024-09-19T09:30:11.944 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 622365, 2024-09-19T09:30:11.944 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:11.944 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.944 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.944 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.944 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:11.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.944 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-09-19T09:30:11.944 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:11.944 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-19T09:30:11.944 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:11.944 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:11.944 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:11.945 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:11.945 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:11.945 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 735, 2024-09-19T09:30:11.945 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 1435, 2024-09-19T09:30:11.945 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 735, 2024-09-19T09:30:11.945 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1435, 2024-09-19T09:30:11.945 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:11.945 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.945 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.945 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.945 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:11.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.945 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 735, 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 735, 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 735, 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 735, 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.946 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.947 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.947 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:11.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.947 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:30:11.947 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:11.947 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-19T09:30:11.947 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:11.947 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:11.947 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:11.947 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:11.947 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:11.947 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-19T09:30:11.947 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-19T09:30:11.948 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-09-19T09:30:11.948 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-09-19T09:30:11.948 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:11.948 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.948 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.948 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.948 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:11.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:11.948 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x562232174b40": { 2024-09-19T09:30:11.948 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:11.948 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-19T09:30:11.948 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:11.948 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:11.949 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-19T09:30:12.785 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-19T09:30:12.786 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-19T09:30:12.786 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-19T09:30:12.786 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-19T09:30:12.786 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T09:30:12.786 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T09:30:13.073 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:30:13.328 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:13 smithi142 ceph-mon[35032]: pgmap v416: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:30:13.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:13 smithi097 ceph-mon[34864]: pgmap v416: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:30:13.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:13 smithi016 ceph-mon[30979]: pgmap v416: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:30:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-19T09:30:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:30:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 933, 2024-09-19T09:30:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 645, 2024-09-19T09:30:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 738245, 2024-09-19T09:30:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 274164, 2024-09-19T09:30:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 17, 2024-09-19T09:30:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2024-09-19T09:30:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.072137826, 2024-09-19T09:30:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.024898596, 2024-09-19T09:30:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.056019657, 2024-09-19T09:30:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.001930453, 2024-09-19T09:30:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 645, 2024-09-19T09:30:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.033402234, 2024-09-19T09:30:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000051786 2024-09-19T09:30:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1271, 2024-09-19T09:30:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000230192, 2024-09-19T09:30:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000181 2024-09-19T09:30:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 504576, 2024-09-19T09:30:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 57344 2024-09-19T09:30:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:30:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 32, 2024-09-19T09:30:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 95, 2024-09-19T09:30:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 14889, 2024-09-19T09:30:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 229119, 2024-09-19T09:30:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 12, 2024-09-19T09:30:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-09-19T09:30:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.009571013, 2024-09-19T09:30:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.003511238, 2024-09-19T09:30:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.005332258, 2024-09-19T09:30:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000198231, 2024-09-19T09:30:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 93, 2024-09-19T09:30:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.022986996, 2024-09-19T09:30:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000247172 2024-09-19T09:30:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 37, 2024-09-19T09:30:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000024251, 2024-09-19T09:30:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000655 2024-09-19T09:30:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-19T09:30:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 188688 2024-09-19T09:30:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:30:13.595 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 46, 2024-09-19T09:30:13.595 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 48, 2024-09-19T09:30:13.595 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 25595, 2024-09-19T09:30:13.595 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 58192, 2024-09-19T09:30:13.595 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 10, 2024-09-19T09:30:13.595 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2024-09-19T09:30:13.595 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.006139789, 2024-09-19T09:30:13.595 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.001220148, 2024-09-19T09:30:13.595 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.004620817, 2024-09-19T09:30:13.595 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000533284, 2024-09-19T09:30:13.595 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:13.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 46, 2024-09-19T09:30:13.596 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.023759227, 2024-09-19T09:30:13.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000516504 2024-09-19T09:30:13.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.596 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:13.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 60, 2024-09-19T09:30:13.596 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000010698, 2024-09-19T09:30:13.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000178 2024-09-19T09:30:13.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.596 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T09:30:13.596 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T09:30:13.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.596 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-09-19T09:30:13.597 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-09-19T09:30:13.597 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-09-19T09:30:13.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.597 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-09-19T09:30:13.597 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-19T09:30:13.597 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-19T09:30:13.597 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 38, 2024-09-19T09:30:13.597 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.001649227, 2024-09-19T09:30:13.597 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000043400 2024-09-19T09:30:13.597 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:13.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.597 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-09-19T09:30:13.597 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-19T09:30:13.598 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-19T09:30:13.598 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2, 2024-09-19T09:30:13.598 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000012254, 2024-09-19T09:30:13.598 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000006127 2024-09-19T09:30:13.598 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:13.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.598 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-09-19T09:30:13.598 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-09-19T09:30:13.598 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-09-19T09:30:13.598 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-09-19T09:30:13.598 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.598 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.598 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.599 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-09-19T09:30:13.599 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-09-19T09:30:13.599 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-09-19T09:30:13.599 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-09-19T09:30:13.599 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 11, 2024-09-19T09:30:13.599 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-09-19T09:30:13.599 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-09-19T09:30:13.599 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 17, 2024-09-19T09:30:13.599 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 1, 2024-09-19T09:30:13.599 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 16, 2024-09-19T09:30:13.599 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-09-19T09:30:13.599 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 13, 2024-09-19T09:30:13.600 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 7, 2024-09-19T09:30:13.600 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-09-19T09:30:13.600 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-09-19T09:30:13.600 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 3, 2024-09-19T09:30:13.600 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 18, 2024-09-19T09:30:13.600 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 14, 2024-09-19T09:30:13.600 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-09-19T09:30:13.600 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 4, 2024-09-19T09:30:13.600 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:30:13.600 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-09-19T09:30:13.600 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-09-19T09:30:13.600 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 234, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-09-19T09:30:13.601 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:30:13.602 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:30:13.602 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:30:13.602 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:30:13.602 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:30:13.602 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-09-19T09:30:13.602 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:30:13.602 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:30:13.602 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:30:13.602 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:30:13.602 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:30:13.602 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:30:13.602 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-09-19T09:30:13.602 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:30:13.603 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-09-19T09:30:13.603 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-09-19T09:30:13.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.603 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-09-19T09:30:13.603 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-09-19T09:30:13.603 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 20, 2024-09-19T09:30:13.603 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 12, 2024-09-19T09:30:13.603 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 13, 2024-09-19T09:30:13.603 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-09-19T09:30:13.603 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-09-19T09:30:13.603 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-09-19T09:30:13.603 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:30:13.603 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-09-19T09:30:13.604 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-09-19T09:30:13.604 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-09-19T09:30:13.604 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-09-19T09:30:13.604 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-09-19T09:30:13.604 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:30:13.604 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:30:13.604 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-09-19T09:30:13.604 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-09-19T09:30:13.604 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 3, 2024-09-19T09:30:13.604 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 22, 2024-09-19T09:30:13.604 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:30:13.604 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-09-19T09:30:13.605 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-09-19T09:30:13.605 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:30:13.605 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-09-19T09:30:13.605 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-09-19T09:30:13.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.605 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-09-19T09:30:13.605 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-09-19T09:30:13.605 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 4, 2024-09-19T09:30:13.605 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-09-19T09:30:13.605 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-09-19T09:30:13.605 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 4, 2024-09-19T09:30:13.605 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-09-19T09:30:13.605 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-09-19T09:30:13.606 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 2, 2024-09-19T09:30:13.606 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-09-19T09:30:13.606 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-09-19T09:30:13.606 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 2, 2024-09-19T09:30:13.606 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 2, 2024-09-19T09:30:13.606 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-09-19T09:30:13.606 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-09-19T09:30:13.606 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-09-19T09:30:13.606 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 4195426, 2024-09-19T09:30:13.606 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-09-19T09:30:13.606 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-09-19T09:30:13.606 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3, 2024-09-19T09:30:13.606 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.006179930, 2024-09-19T09:30:13.607 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002059976 2024-09-19T09:30:13.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.607 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-09-19T09:30:13.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.607 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-09-19T09:30:13.607 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 21, 2024-09-19T09:30:13.607 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 26, 2024-09-19T09:30:13.607 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 5, 2024-09-19T09:30:13.607 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 20, 2024-09-19T09:30:13.607 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 27, 2024-09-19T09:30:13.608 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 7, 2024-09-19T09:30:13.608 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 17, 2024-09-19T09:30:13.608 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 24, 2024-09-19T09:30:13.608 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 7, 2024-09-19T09:30:13.608 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-09-19T09:30:13.608 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-09-19T09:30:13.608 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-09-19T09:30:13.608 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 49892, 2024-09-19T09:30:13.608 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 223520 2024-09-19T09:30:13.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.608 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-09-19T09:30:13.609 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-09-19T09:30:13.609 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-09-19T09:30:13.609 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-09-19T09:30:13.609 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 4, 2024-09-19T09:30:13.609 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-09-19T09:30:13.609 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-09-19T09:30:13.609 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.609 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.609 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.609 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-09-19T09:30:13.609 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.609 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.610 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.610 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-09-19T09:30:13.610 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.610 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.610 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.610 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-09-19T09:30:13.610 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.610 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.610 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.610 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-09-19T09:30:13.611 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.611 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.611 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.611 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-09-19T09:30:13.611 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.611 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.611 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.611 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-09-19T09:30:13.611 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.611 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.611 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.612 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-09-19T09:30:13.612 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.612 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.612 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.612 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-09-19T09:30:13.612 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.612 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.612 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.612 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-09-19T09:30:13.613 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.613 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.613 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.613 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-09-19T09:30:13.613 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.613 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.613 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.613 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-09-19T09:30:13.613 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.614 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.614 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.614 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-09-19T09:30:13.614 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.614 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.614 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.615 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-09-19T09:30:13.615 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.615 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.615 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.615 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-09-19T09:30:13.615 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.615 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.615 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.615 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-09-19T09:30:13.616 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.616 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.616 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.616 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-09-19T09:30:13.616 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.616 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.616 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.616 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-09-19T09:30:13.616 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.616 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-09-19T09:30:13.617 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.618 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.618 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.618 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-09-19T09:30:13.618 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.618 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.618 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.618 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-09-19T09:30:13.618 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.618 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.618 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.619 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-09-19T09:30:13.619 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.619 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.619 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.619 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-09-19T09:30:13.619 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.619 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.619 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.619 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-09-19T09:30:13.619 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.620 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.620 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.620 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-09-19T09:30:13.620 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.620 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.620 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.620 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-09-19T09:30:13.620 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.620 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.620 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.621 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-09-19T09:30:13.621 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.621 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.621 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.621 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:30:13.621 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:30:13.621 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-09-19T09:30:13.621 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.621 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.621 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.621 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:13.622 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.622 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-09-19T09:30:13.622 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 0, 2024-09-19T09:30:13.622 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 1, 2024-09-19T09:30:13.622 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-09-19T09:30:13.622 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 0, 2024-09-19T09:30:13.622 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-09-19T09:30:13.622 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-09-19T09:30:13.622 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-09-19T09:30:13.622 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 88, 2024-09-19T09:30:13.622 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-09-19T09:30:13.622 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.623 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-09-19T09:30:13.623 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:30:13.623 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-09-19T09:30:13.623 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:30:13.623 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:30:13.623 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:30:13.623 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:30:13.623 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:30:13.623 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:30:13.623 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:30:13.623 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:30:13.623 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:30:13.623 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:30:13.624 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:30:13.624 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:30:13.624 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:30:13.624 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-09-19T09:30:13.624 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:30:13.624 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-09-19T09:30:13.624 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:30:13.624 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:30:13.624 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:30:13.624 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:30:13.624 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:30:13.624 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:30:13.625 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:30:13.625 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-09-19T09:30:13.625 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:30:13.625 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:30:13.625 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:30:13.625 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-09-19T09:30:13.625 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-09-19T09:30:13.625 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-09-19T09:30:13.625 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:30:13.625 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:30:13.625 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:30:13.625 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:30:13.625 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 262423, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 89, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7592, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 156, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:30:13.626 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-09-19T09:30:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-09-19T09:30:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 85117, 2024-09-19T09:30:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 335, 2024-09-19T09:30:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-09-19T09:30:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-09-19T09:30:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-09-19T09:30:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-09-19T09:30:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-09-19T09:30:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-09-19T09:30:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-09-19T09:30:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 38, 2024-09-19T09:30:13.627 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 11334, 2024-09-19T09:30:13.628 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 30, 2024-09-19T09:30:13.628 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 38, 2024-09-19T09:30:13.628 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-09-19T09:30:13.628 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 38, 2024-09-19T09:30:13.628 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.203999559, 2024-09-19T09:30:13.628 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.005368409 2024-09-19T09:30:13.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.628 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-09-19T09:30:13.628 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-09-19T09:30:13.628 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 38, 2024-09-19T09:30:13.628 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 60 2024-09-19T09:30:13.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.628 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 38, 2024-09-19T09:30:13.629 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 0, 2024-09-19T09:30:13.629 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 38, 2024-09-19T09:30:13.629 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-09-19T09:30:13.629 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-09-19T09:30:13.629 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2024-09-19T09:30:13.629 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 10, 2024-09-19T09:30:13.629 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 0, 2024-09-19T09:30:13.629 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 3, 2024-09-19T09:30:13.629 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 6, 2024-09-19T09:30:13.629 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-09-19T09:30:13.629 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-09-19T09:30:13.629 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 6, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 0, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 20, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-09-19T09:30:13.630 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:30:13.631 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-09-19T09:30:13.631 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:30:13.631 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-09-19T09:30:13.631 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-09-19T09:30:13.631 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-09-19T09:30:13.631 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-09-19T09:30:13.631 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-09-19T09:30:13.631 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-09-19T09:30:13.631 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-09-19T09:30:13.631 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-09-19T09:30:13.631 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-09-19T09:30:13.631 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-09-19T09:30:13.631 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-09-19T09:30:13.632 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-09-19T09:30:13.632 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-09-19T09:30:13.632 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-09-19T09:30:13.632 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-09-19T09:30:13.632 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-09-19T09:30:13.632 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-09-19T09:30:13.632 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 81, 2024-09-19T09:30:13.632 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-09-19T09:30:13.632 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 2, 2024-09-19T09:30:13.632 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 9, 2024-09-19T09:30:13.632 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 9, 2024-09-19T09:30:13.632 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-09-19T09:30:13.632 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-09-19T09:30:13.633 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 14, 2024-09-19T09:30:13.633 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 0, 2024-09-19T09:30:13.633 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 1 2024-09-19T09:30:13.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.633 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-09-19T09:30:13.633 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-09-19T09:30:13.633 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:30:13.633 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-09-19T09:30:13.633 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-09-19T09:30:13.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.633 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-09-19T09:30:13.633 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 25 2024-09-19T09:30:13.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 0, 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:13.634 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 1007, 2024-09-19T09:30:13.635 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 678375, 2024-09-19T09:30:13.635 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:13.635 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 1007, 2024-09-19T09:30:13.635 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-19T09:30:13.635 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-19T09:30:13.635 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1007, 2024-09-19T09:30:13.635 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 678375, 2024-09-19T09:30:13.635 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:13.635 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.635 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.635 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.635 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 38, 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 11334, 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 38, 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 11334, 2024-09-19T09:30:13.636 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:13.637 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.637 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.637 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.637 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:13.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.637 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-09-19T09:30:13.637 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:13.637 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-09-19T09:30:13.637 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:13.637 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:13.637 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:13.637 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:13.637 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:13.638 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 38, 2024-09-19T09:30:13.638 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 38, 2024-09-19T09:30:13.638 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 38, 2024-09-19T09:30:13.638 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 38, 2024-09-19T09:30:13.638 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:13.638 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.638 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.638 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.638 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:13.638 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.638 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:30:13.638 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:13.638 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.639 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:13.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:13.640 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55cb41fc0960": { 2024-09-19T09:30:13.640 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:13.640 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-19T09:30:13.640 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:13.640 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 4, 2024-09-19T09:30:13.640 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 1122, 2024-09-19T09:30:13.640 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:13.640 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 4, 2024-09-19T09:30:13.640 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-19T09:30:13.640 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-19T09:30:13.640 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 3, 2024-09-19T09:30:13.640 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1122, 2024-09-19T09:30:13.641 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:13.641 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:13.641 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:13.641 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:13.641 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:13.641 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:13.641 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-19T09:30:14.240 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-19T09:30:14.240 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-19T09:30:14.240 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-19T09:30:14.240 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=20 2024-09-19T09:30:14.240 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T09:30:14.241 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T09:30:14.485 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T09:30:15.011 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-19T09:30:15.011 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T09:30:15.011 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 932, 2024-09-19T09:30:15.011 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 649, 2024-09-19T09:30:15.011 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 728312, 2024-09-19T09:30:15.011 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 187432, 2024-09-19T09:30:15.012 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 8, 2024-09-19T09:30:15.012 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-09-19T09:30:15.012 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.067895639, 2024-09-19T09:30:15.012 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.024553979, 2024-09-19T09:30:15.012 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.052335799, 2024-09-19T09:30:15.012 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.001940841, 2024-09-19T09:30:15.012 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:15.012 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 648, 2024-09-19T09:30:15.012 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.033813107, 2024-09-19T09:30:15.012 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000052180 2024-09-19T09:30:15.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.012 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:15.012 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1281, 2024-09-19T09:30:15.013 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000216904, 2024-09-19T09:30:15.013 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000169 2024-09-19T09:30:15.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.013 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 482576, 2024-09-19T09:30:15.013 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 60576 2024-09-19T09:30:15.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.013 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T09:30:15.013 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 22, 2024-09-19T09:30:15.013 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 92, 2024-09-19T09:30:15.013 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 4750, 2024-09-19T09:30:15.013 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 297159, 2024-09-19T09:30:15.013 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 19, 2024-09-19T09:30:15.013 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2024-09-19T09:30:15.014 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.010965480, 2024-09-19T09:30:15.014 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.003345744, 2024-09-19T09:30:15.014 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.006226701, 2024-09-19T09:30:15.014 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000282169, 2024-09-19T09:30:15.014 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:15.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 88, 2024-09-19T09:30:15.014 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.014970433, 2024-09-19T09:30:15.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000170118 2024-09-19T09:30:15.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.014 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:15.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 15, 2024-09-19T09:30:15.014 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000001898, 2024-09-19T09:30:15.014 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000126 2024-09-19T09:30:15.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.015 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-19T09:30:15.015 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 189568 2024-09-19T09:30:15.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.015 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T09:30:15.015 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 56, 2024-09-19T09:30:15.015 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 54, 2024-09-19T09:30:15.015 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 29853, 2024-09-19T09:30:15.015 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 79783, 2024-09-19T09:30:15.015 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 12, 2024-09-19T09:30:15.015 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2024-09-19T09:30:15.015 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.009198482, 2024-09-19T09:30:15.015 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.001348440, 2024-09-19T09:30:15.016 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.007420569, 2024-09-19T09:30:15.016 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000534236, 2024-09-19T09:30:15.016 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T09:30:15.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 55, 2024-09-19T09:30:15.016 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.031516482, 2024-09-19T09:30:15.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000573026 2024-09-19T09:30:15.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.016 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-09-19T09:30:15.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 52, 2024-09-19T09:30:15.016 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000009699, 2024-09-19T09:30:15.016 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000186 2024-09-19T09:30:15.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.016 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T09:30:15.017 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T09:30:15.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.017 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-09-19T09:30:15.017 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-09-19T09:30:15.017 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-09-19T09:30:15.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.017 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-09-19T09:30:15.017 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-19T09:30:15.017 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-19T09:30:15.017 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 49, 2024-09-19T09:30:15.017 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.001508881, 2024-09-19T09:30:15.017 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000030793 2024-09-19T09:30:15.017 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-09-19T09:30:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-09-19T09:30:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-09-19T09:30:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2, 2024-09-19T09:30:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000011240, 2024-09-19T09:30:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000005620 2024-09-19T09:30:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-09-19T09:30:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-09-19T09:30:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-09-19T09:30:15.018 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-09-19T09:30:15.019 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.019 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.019 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.019 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.019 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-09-19T09:30:15.019 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-09-19T09:30:15.019 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-09-19T09:30:15.019 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-09-19T09:30:15.019 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 11, 2024-09-19T09:30:15.019 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-09-19T09:30:15.019 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-09-19T09:30:15.019 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 17, 2024-09-19T09:30:15.019 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 1, 2024-09-19T09:30:15.020 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 16, 2024-09-19T09:30:15.020 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-09-19T09:30:15.020 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 13, 2024-09-19T09:30:15.020 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 7, 2024-09-19T09:30:15.020 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-09-19T09:30:15.020 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-09-19T09:30:15.020 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 3, 2024-09-19T09:30:15.020 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 18, 2024-09-19T09:30:15.020 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 14, 2024-09-19T09:30:15.020 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-09-19T09:30:15.020 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 4, 2024-09-19T09:30:15.020 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-09-19T09:30:15.020 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-09-19T09:30:15.021 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-09-19T09:30:15.021 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-09-19T09:30:15.021 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-09-19T09:30:15.021 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-09-19T09:30:15.021 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-09-19T09:30:15.021 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-09-19T09:30:15.021 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-09-19T09:30:15.021 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-09-19T09:30:15.021 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-09-19T09:30:15.021 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-09-19T09:30:15.021 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2024-09-19T09:30:15.021 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 234, 2024-09-19T09:30:15.021 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T09:30:15.022 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 20, 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 12, 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 13, 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-09-19T09:30:15.023 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 3, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 22, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-09-19T09:30:15.024 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-09-19T09:30:15.025 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-09-19T09:30:15.025 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-09-19T09:30:15.025 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-09-19T09:30:15.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.025 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-09-19T09:30:15.025 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-09-19T09:30:15.025 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 4, 2024-09-19T09:30:15.025 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-09-19T09:30:15.025 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-09-19T09:30:15.025 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 4, 2024-09-19T09:30:15.025 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-09-19T09:30:15.025 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-09-19T09:30:15.025 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 2, 2024-09-19T09:30:15.026 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-09-19T09:30:15.026 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-09-19T09:30:15.026 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 2, 2024-09-19T09:30:15.026 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 2, 2024-09-19T09:30:15.026 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-09-19T09:30:15.026 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-09-19T09:30:15.026 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-09-19T09:30:15.026 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 4195426, 2024-09-19T09:30:15.027 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-09-19T09:30:15.027 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-09-19T09:30:15.027 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3, 2024-09-19T09:30:15.027 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.005877333, 2024-09-19T09:30:15.027 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001959111 2024-09-19T09:30:15.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.027 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-09-19T09:30:15.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.027 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-09-19T09:30:15.027 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 21, 2024-09-19T09:30:15.027 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 26, 2024-09-19T09:30:15.027 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 5, 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 20, 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 27, 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 7, 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 17, 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 24, 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 7, 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 50428, 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 223520 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-09-19T09:30:15.028 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 4, 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.030 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-09-19T09:30:15.030 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.030 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.030 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.030 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-09-19T09:30:15.030 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.030 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.030 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.030 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-09-19T09:30:15.030 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.030 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.031 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.031 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-09-19T09:30:15.031 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.031 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.031 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.031 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-09-19T09:30:15.031 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.031 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.031 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.031 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-09-19T09:30:15.032 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.032 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.032 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.032 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-09-19T09:30:15.032 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.032 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.032 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.032 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-09-19T09:30:15.032 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.033 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.033 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.033 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-09-19T09:30:15.033 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.033 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.033 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.033 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-09-19T09:30:15.033 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.033 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.033 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.034 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.035 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-09-19T09:30:15.035 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.035 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.035 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.035 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-09-19T09:30:15.035 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.035 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.035 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.035 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-09-19T09:30:15.035 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.036 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.036 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.036 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.036 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-09-19T09:30:15.036 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.036 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.036 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.036 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.036 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-09-19T09:30:15.036 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.036 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.036 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.036 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.037 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.038 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-09-19T09:30:15.038 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.038 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.038 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.038 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-09-19T09:30:15.038 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.038 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.038 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.038 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-09-19T09:30:15.038 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.039 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-09-19T09:30:15.040 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 0, 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 1, 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 0, 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 87, 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T09:30:15.041 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-09-19T09:30:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T09:30:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-09-19T09:30:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T09:30:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T09:30:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T09:30:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T09:30:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T09:30:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-09-19T09:30:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T09:30:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T09:30:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-09-19T09:30:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-09-19T09:30:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-09-19T09:30:15.043 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-09-19T09:30:15.043 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T09:30:15.043 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T09:30:15.043 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T09:30:15.043 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T09:30:15.043 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T09:30:15.043 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-09-19T09:30:15.043 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-09-19T09:30:15.043 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-09-19T09:30:15.043 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T09:30:15.044 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T09:30:15.044 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-09-19T09:30:15.044 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-09-19T09:30:15.044 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-09-19T09:30:15.044 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-09-19T09:30:15.044 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T09:30:15.044 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-09-19T09:30:15.044 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T09:30:15.044 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-09-19T09:30:15.044 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 254375, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 86, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 7592, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 156, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-09-19T09:30:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-09-19T09:30:15.046 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-09-19T09:30:15.046 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 85117, 2024-09-19T09:30:15.046 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 335, 2024-09-19T09:30:15.046 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-09-19T09:30:15.046 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-09-19T09:30:15.046 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-09-19T09:30:15.046 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-09-19T09:30:15.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.046 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-09-19T09:30:15.046 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-09-19T09:30:15.046 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-09-19T09:30:15.046 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 38, 2024-09-19T09:30:15.047 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 11334, 2024-09-19T09:30:15.047 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 33, 2024-09-19T09:30:15.047 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 38, 2024-09-19T09:30:15.047 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-09-19T09:30:15.047 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 38, 2024-09-19T09:30:15.047 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.215999536, 2024-09-19T09:30:15.047 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.005684198 2024-09-19T09:30:15.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.047 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-09-19T09:30:15.047 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-09-19T09:30:15.047 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 38, 2024-09-19T09:30:15.047 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 60 2024-09-19T09:30:15.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.048 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 38, 2024-09-19T09:30:15.048 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 0, 2024-09-19T09:30:15.048 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 38, 2024-09-19T09:30:15.048 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-09-19T09:30:15.048 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-09-19T09:30:15.048 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2024-09-19T09:30:15.048 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 10, 2024-09-19T09:30:15.048 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 0, 2024-09-19T09:30:15.048 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 3, 2024-09-19T09:30:15.048 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 6, 2024-09-19T09:30:15.048 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-09-19T09:30:15.048 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-09-19T09:30:15.049 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-09-19T09:30:15.049 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-09-19T09:30:15.049 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 6, 2024-09-19T09:30:15.049 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-09-19T09:30:15.049 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-09-19T09:30:15.049 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-09-19T09:30:15.049 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 0, 2024-09-19T09:30:15.049 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 20, 2024-09-19T09:30:15.049 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-09-19T09:30:15.049 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-09-19T09:30:15.050 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-09-19T09:30:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-09-19T09:30:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-09-19T09:30:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-09-19T09:30:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-09-19T09:30:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-09-19T09:30:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-09-19T09:30:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-09-19T09:30:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-09-19T09:30:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-09-19T09:30:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-09-19T09:30:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 81, 2024-09-19T09:30:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-09-19T09:30:15.051 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 2, 2024-09-19T09:30:15.052 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 9, 2024-09-19T09:30:15.052 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 9, 2024-09-19T09:30:15.052 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-09-19T09:30:15.052 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-09-19T09:30:15.052 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 14, 2024-09-19T09:30:15.052 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 0, 2024-09-19T09:30:15.052 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 1 2024-09-19T09:30:15.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.052 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-09-19T09:30:15.052 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-09-19T09:30:15.052 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-09-19T09:30:15.052 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-09-19T09:30:15.052 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 25 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 0, 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.053 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 1006, 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 662528, 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 1006, 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1006, 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 662528, 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.054 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.055 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:15.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.055 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-09-19T09:30:15.055 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:15.055 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-09-19T09:30:15.055 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:15.055 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:15.055 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:15.055 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:15.055 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:15.055 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 38, 2024-09-19T09:30:15.055 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 11334, 2024-09-19T09:30:15.056 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 38, 2024-09-19T09:30:15.056 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 11334, 2024-09-19T09:30:15.056 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:15.056 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.056 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.056 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.056 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:15.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.056 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-09-19T09:30:15.056 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:15.056 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-09-19T09:30:15.056 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:15.056 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:15.057 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:15.057 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:15.057 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:15.057 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 38, 2024-09-19T09:30:15.057 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 38, 2024-09-19T09:30:15.057 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 38, 2024-09-19T09:30:15.057 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 38, 2024-09-19T09:30:15.057 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:15.057 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.057 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.057 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.057 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:15.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.058 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-09-19T09:30:15.058 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:15.058 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-19T09:30:15.058 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:15.058 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-09-19T09:30:15.058 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-09-19T09:30:15.058 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:15.058 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-09-19T09:30:15.058 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-19T09:30:15.058 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-19T09:30:15.058 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-09-19T09:30:15.058 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-09-19T09:30:15.058 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x56503d4f0960": { 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 4, 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 1122, 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 4, 2024-09-19T09:30:15.059 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-09-19T09:30:15.060 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-09-19T09:30:15.060 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 3, 2024-09-19T09:30:15.060 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1122, 2024-09-19T09:30:15.060 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-09-19T09:30:15.060 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-09-19T09:30:15.060 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-09-19T09:30:15.060 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-09-19T09:30:15.060 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:15.060 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-19T09:30:15.060 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-19T09:30:15.307 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:15 smithi142 ceph-mon[35032]: pgmap v417: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:30:15.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:15 smithi097 ceph-mon[34864]: pgmap v417: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:30:15.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:15 smithi016 ceph-mon[30979]: pgmap v417: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-19T09:30:15.880 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-19T09:30:15.880 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-19T09:30:15.880 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-19T09:30:15.880 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=20 2024-09-19T09:30:15.880 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T09:30:15.891 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T09:30:15.891 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:30:15.891 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T09:30:15.915 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-19T09:30:15.915 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T09:30:15.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T09:30:16.284 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:16.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.740+0000 7f28847e9640 1 -- 172.21.15.16:0/1238262454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c106f20 msgr2=0x7f287c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:16.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.740+0000 7f28847e9640 1 --2- 172.21.15.16:0/1238262454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c106f20 0x7f287c10eaa0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2874009ca0 tx=0x7f287402a8c0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.740+0000 7f28847e9640 1 -- 172.21.15.16:0/1238262454 shutdown_connections 2024-09-19T09:30:16.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.740+0000 7f28847e9640 1 --2- 172.21.15.16:0/1238262454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f287c1027a0 0x7f287c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.740+0000 7f28847e9640 1 --2- 172.21.15.16:0/1238262454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c106f20 0x7f287c10eaa0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.740+0000 7f28847e9640 1 --2- 172.21.15.16:0/1238262454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f287c101df0 0x7f287c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.740+0000 7f28847e9640 1 -- 172.21.15.16:0/1238262454 >> 172.21.15.16:0/1238262454 conn(0x7f287c0fd8f0 msgr2=0x7f287c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:16.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.740+0000 7f28847e9640 1 -- 172.21.15.16:0/1238262454 shutdown_connections 2024-09-19T09:30:16.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.740+0000 7f28847e9640 1 -- 172.21.15.16:0/1238262454 wait complete. 2024-09-19T09:30:16.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f28847e9640 1 Processor -- start 2024-09-19T09:30:16.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f28847e9640 1 -- start start 2024-09-19T09:30:16.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f28847e9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f287c101df0 0x7f287c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:16.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f28847e9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f287c1027a0 0x7f287c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:16.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f28847e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c106f20 0x7f287c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:16.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f28847e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f287c111f80 con 0x7f287c106f20 2024-09-19T09:30:16.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f28847e9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f287c111e00 con 0x7f287c1027a0 2024-09-19T09:30:16.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f28847e9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f287c112100 con 0x7f287c101df0 2024-09-19T09:30:16.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f2882d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c106f20 0x7f287c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f2882d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c106f20 0x7f287c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40948/0 (socket says 172.21.15.16:40948) 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f2882d86640 1 -- 172.21.15.16:0/3781884466 learned_addr learned my addr 172.21.15.16:0/3781884466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f2881d84640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f287c1027a0 0x7f287c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f2882585640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f287c101df0 0x7f287c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f2882d86640 1 -- 172.21.15.16:0/3781884466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f287c101df0 msgr2=0x7f287c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f2882d86640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f287c101df0 0x7f287c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f2882d86640 1 -- 172.21.15.16:0/3781884466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f287c1027a0 msgr2=0x7f287c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f2882d86640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f287c1027a0 0x7f287c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.741+0000 7f2882d86640 1 -- 172.21.15.16:0/3781884466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c1a6880 con 0x7f287c106f20 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.742+0000 7f2882d86640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c106f20 0x7f287c074f70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2874031a20 tx=0x7f2874031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.742+0000 7f28737fe640 1 -- 172.21.15.16:0/3781884466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874003c30 con 0x7f287c106f20 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.742+0000 7f28737fe640 1 -- 172.21.15.16:0/3781884466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2874003dd0 con 0x7f287c106f20 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.742+0000 7f28847e9640 1 -- 172.21.15.16:0/3781884466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287c1a6b10 con 0x7f287c106f20 2024-09-19T09:30:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.742+0000 7f28737fe640 1 -- 172.21.15.16:0/3781884466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287402fcf0 con 0x7f287c106f20 2024-09-19T09:30:16.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.742+0000 7f28847e9640 1 -- 172.21.15.16:0/3781884466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287c1a7050 con 0x7f287c106f20 2024-09-19T09:30:16.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.743+0000 7f28847e9640 1 -- 172.21.15.16:0/3781884466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287c1021d0 con 0x7f287c106f20 2024-09-19T09:30:16.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.744+0000 7f28737fe640 1 -- 172.21.15.16:0/3781884466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f2874004050 con 0x7f287c106f20 2024-09-19T09:30:16.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.745+0000 7f28737fe640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f28680a3240 0x7f28680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:16.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.745+0000 7f2882585640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f28680a3240 0x7f28680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:16.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.745+0000 7f28737fe640 1 -- 172.21.15.16:0/3781884466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f28740f0ef0 con 0x7f287c106f20 2024-09-19T09:30:16.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.745+0000 7f2882585640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f28680a3240 0x7f28680a5700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2878005fd0 tx=0x7f2878005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:16.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.749+0000 7f28737fe640 1 -- 172.21.15.16:0/3781884466 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f28740bdc60 con 0x7f287c106f20 2024-09-19T09:30:16.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.939+0000 7f28847e9640 1 -- 172.21.15.16:0/3781884466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f287c10a100 con 0x7f287c106f20 2024-09-19T09:30:16.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.940+0000 7f28737fe640 1 -- 172.21.15.16:0/3781884466 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v81) ==== 81+0+589 (secure 0 0 0) 0x7f287402fe90 con 0x7f287c106f20 2024-09-19T09:30:16.941 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1680178094 2024-09-20T09:18:41.244076+0000 2024-09-19T09:30:16.942 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3710267992 2024-09-20T09:18:41.244076+0000 2024-09-19T09:30:16.942 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1680178094 2024-09-20T09:18:41.244076+0000 2024-09-19T09:30:16.942 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3740275561 2024-09-20T09:18:41.244076+0000 2024-09-19T09:30:16.942 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2755421444 2024-09-20T09:18:41.244076+0000 2024-09-19T09:30:16.942 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/183909517 2024-09-20T09:18:17.070034+0000 2024-09-19T09:30:16.942 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/357390846 2024-09-20T09:18:17.070034+0000 2024-09-19T09:30:16.942 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/183909517 2024-09-20T09:18:17.070034+0000 2024-09-19T09:30:16.942 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3104087402 2024-09-20T09:18:17.070034+0000 2024-09-19T09:30:16.942 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3546947161 2024-09-20T09:18:17.070034+0000 2024-09-19T09:30:16.942 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-09-19T09:30:16.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.942+0000 7f28847e9640 1 -- 172.21.15.16:0/3781884466 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f28680a3240 msgr2=0x7f28680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:16.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.942+0000 7f28847e9640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f28680a3240 0x7f28680a5700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2878005fd0 tx=0x7f2878005850 comp rx=0 tx=0).stop 2024-09-19T09:30:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.942+0000 7f28847e9640 1 -- 172.21.15.16:0/3781884466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c106f20 msgr2=0x7f287c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.942+0000 7f28847e9640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c106f20 0x7f287c074f70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2874031a20 tx=0x7f2874031a50 comp rx=0 tx=0).stop 2024-09-19T09:30:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.942+0000 7f28847e9640 1 -- 172.21.15.16:0/3781884466 shutdown_connections 2024-09-19T09:30:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.942+0000 7f28847e9640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c106f20 0x7f287c074f70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.942+0000 7f28847e9640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f287c1027a0 0x7f287c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.942+0000 7f28847e9640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f28680a3240 0x7f28680a5700 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.942+0000 7f28847e9640 1 --2- 172.21.15.16:0/3781884466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f287c101df0 0x7f287c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.942+0000 7f28847e9640 1 -- 172.21.15.16:0/3781884466 >> 172.21.15.16:0/3781884466 conn(0x7f287c0fd8f0 msgr2=0x7f287c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.942+0000 7f28847e9640 1 -- 172.21.15.16:0/3781884466 shutdown_connections 2024-09-19T09:30:16.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:16.942+0000 7f28847e9640 1 -- 172.21.15.16:0/3781884466 wait complete. 2024-09-19T09:30:17.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:17 smithi016 ceph-mon[30979]: pgmap v418: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:30:17.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:17 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3781884466' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T09:30:17.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:17 smithi142 ceph-mon[35032]: pgmap v418: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:30:17.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:17 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3781884466' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T09:30:17.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:17 smithi097 ceph-mon[34864]: pgmap v418: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T09:30:17.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:17 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3781884466' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T09:30:17.706 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T09:30:17.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:30:17.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T09:30:17.752 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T09:30:17.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:30:17.752 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T09:30:17.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:30:17.784 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-19T09:30:17.840 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T09:30:17.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:30:17.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T09:30:17.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T09:30:17.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-19T09:30:18.005 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T09:30:18.016 INFO:tasks.cephadm:Teardown begin 2024-09-19T09:30:18.016 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T09:30:18.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:30:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.737+0000 7f4ea803b640 1 -- 172.21.15.16:0/3753507650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea01036e0 msgr2=0x7f4ea0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.737+0000 7f4ea803b640 1 --2- 172.21.15.16:0/3753507650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea01036e0 0x7f4ea0107b90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009930 tx=0x7f4e9402a420 comp rx=0 tx=0).stop 2024-09-19T09:30:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.737+0000 7f4ea803b640 1 -- 172.21.15.16:0/3753507650 shutdown_connections 2024-09-19T09:30:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.737+0000 7f4ea803b640 1 --2- 172.21.15.16:0/3753507650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea01082a0 0x7f4ea010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.737+0000 7f4ea803b640 1 --2- 172.21.15.16:0/3753507650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea01036e0 0x7f4ea0107b90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.737+0000 7f4ea803b640 1 --2- 172.21.15.16:0/3753507650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0102d30 0x7f4ea0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.737+0000 7f4ea803b640 1 -- 172.21.15.16:0/3753507650 >> 172.21.15.16:0/3753507650 conn(0x7f4ea00fc7d0 msgr2=0x7f4ea00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.737+0000 7f4ea803b640 1 -- 172.21.15.16:0/3753507650 shutdown_connections 2024-09-19T09:30:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.737+0000 7f4ea803b640 1 -- 172.21.15.16:0/3753507650 wait complete. 2024-09-19T09:30:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.737+0000 7f4ea803b640 1 Processor -- start 2024-09-19T09:30:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea803b640 1 -- start start 2024-09-19T09:30:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea803b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0102d30 0x7f4ea0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea803b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea01036e0 0x7f4ea0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea803b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea01082a0 0x7f4ea0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea803b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea01120c0 con 0x7f4ea01082a0 2024-09-19T09:30:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea803b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea0111f40 con 0x7f4ea0102d30 2024-09-19T09:30:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea803b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea0112240 con 0x7f4ea01036e0 2024-09-19T09:30:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea65d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea01082a0 0x7f4ea0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea5dd7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0102d30 0x7f4ea0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea65d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea01082a0 0x7f4ea0101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40962/0 (socket says 172.21.15.16:40962) 2024-09-19T09:30:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea65d8640 1 -- 172.21.15.16:0/2089407207 learned_addr learned my addr 172.21.15.16:0/2089407207 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:30:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea55d6640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea01036e0 0x7f4ea0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea65d8640 1 -- 172.21.15.16:0/2089407207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea01036e0 msgr2=0x7f4ea0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea65d8640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea01036e0 0x7f4ea0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea65d8640 1 -- 172.21.15.16:0/2089407207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0102d30 msgr2=0x7f4ea0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea65d8640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0102d30 0x7f4ea0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea65d8640 1 -- 172.21.15.16:0/2089407207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea01a6a80 con 0x7f4ea01082a0 2024-09-19T09:30:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.738+0000 7f4ea5dd7640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0102d30 0x7f4ea0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T09:30:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.739+0000 7f4ea65d8640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea01082a0 0x7f4ea0101410 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009fd0 tx=0x7f4e9800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.739+0000 7f4e9effd640 1 -- 172.21.15.16:0/2089407207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98011830 con 0x7f4ea01082a0 2024-09-19T09:30:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.739+0000 7f4ea803b640 1 -- 172.21.15.16:0/2089407207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea01a6d70 con 0x7f4ea01082a0 2024-09-19T09:30:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.739+0000 7f4e9effd640 1 -- 172.21.15.16:0/2089407207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e98016070 con 0x7f4ea01082a0 2024-09-19T09:30:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.739+0000 7f4e9effd640 1 -- 172.21.15.16:0/2089407207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800f2e0 con 0x7f4ea01082a0 2024-09-19T09:30:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.739+0000 7f4ea803b640 1 -- 172.21.15.16:0/2089407207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea01a7280 con 0x7f4ea01082a0 2024-09-19T09:30:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.740+0000 7f4ea803b640 1 -- 172.21.15.16:0/2089407207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea0103110 con 0x7f4ea01082a0 2024-09-19T09:30:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.741+0000 7f4e9effd640 1 -- 172.21.15.16:0/2089407207 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4e980119d0 con 0x7f4ea01082a0 2024-09-19T09:30:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.742+0000 7f4e9effd640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4e880a3240 0x7f4e880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.742+0000 7f4ea5dd7640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4e880a3240 0x7f4e880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.742+0000 7f4e9effd640 1 -- 172.21.15.16:0/2089407207 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f4e980cee90 con 0x7f4ea01082a0 2024-09-19T09:30:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.742+0000 7f4ea5dd7640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4e880a3240 0x7f4e880a5700 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000f650 tx=0x7f4e900075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.746+0000 7f4e9effd640 1 -- 172.21.15.16:0/2089407207 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4e9809bbb0 con 0x7f4ea01082a0 2024-09-19T09:30:18.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.960+0000 7f4ea803b640 1 -- 172.21.15.16:0/2089407207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ea010a2b0 con 0x7f4ea01082a0 2024-09-19T09:30:18.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.960+0000 7f4e9effd640 1 -- 172.21.15.16:0/2089407207 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f4e9809b4b0 con 0x7f4ea01082a0 2024-09-19T09:30:18.962 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:30:18.962 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:30:18.962 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:30:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.963+0000 7f4ea803b640 1 -- 172.21.15.16:0/2089407207 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4e880a3240 msgr2=0x7f4e880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.963+0000 7f4ea803b640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4e880a3240 0x7f4e880a5700 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000f650 tx=0x7f4e900075b0 comp rx=0 tx=0).stop 2024-09-19T09:30:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.963+0000 7f4ea803b640 1 -- 172.21.15.16:0/2089407207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea01082a0 msgr2=0x7f4ea0101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.963+0000 7f4ea803b640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea01082a0 0x7f4ea0101410 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009fd0 tx=0x7f4e9800b8e0 comp rx=0 tx=0).stop 2024-09-19T09:30:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.963+0000 7f4ea803b640 1 -- 172.21.15.16:0/2089407207 shutdown_connections 2024-09-19T09:30:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.963+0000 7f4ea803b640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f4e880a3240 0x7f4e880a5700 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.963+0000 7f4ea803b640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea01036e0 0x7f4ea0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.963+0000 7f4ea803b640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea01082a0 0x7f4ea0101410 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.963+0000 7f4ea803b640 1 --2- 172.21.15.16:0/2089407207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea0102d30 0x7f4ea0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.963+0000 7f4ea803b640 1 -- 172.21.15.16:0/2089407207 >> 172.21.15.16:0/2089407207 conn(0x7f4ea00fc7d0 msgr2=0x7f4ea0106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.963+0000 7f4ea803b640 1 -- 172.21.15.16:0/2089407207 shutdown_connections 2024-09-19T09:30:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:18.963+0000 7f4ea803b640 1 -- 172.21.15.16:0/2089407207 wait complete. 2024-09-19T09:30:19.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:19 smithi016 ceph-mon[30979]: pgmap v419: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:30:19.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:19 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2089407207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:19.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:19 smithi142 ceph-mon[35032]: pgmap v419: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:30:19.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:19 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2089407207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:19.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:19 smithi097 ceph-mon[34864]: pgmap v419: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T09:30:19.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:19 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2089407207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:19.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T09:30:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.445+0000 7f1d77d6e640 1 -- 172.21.15.16:0/673559218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d70100ca0 msgr2=0x7f1d70106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.445+0000 7f1d77d6e640 1 --2- 172.21.15.16:0/673559218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d70100ca0 0x7f1d70106840 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009930 tx=0x7f1d6002a420 comp rx=0 tx=0).stop 2024-09-19T09:30:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.446+0000 7f1d77d6e640 1 -- 172.21.15.16:0/673559218 shutdown_connections 2024-09-19T09:30:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.446+0000 7f1d77d6e640 1 --2- 172.21.15.16:0/673559218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d70106f50 0x7f1d7010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.446+0000 7f1d77d6e640 1 --2- 172.21.15.16:0/673559218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d70100ca0 0x7f1d70106840 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.446+0000 7f1d77d6e640 1 --2- 172.21.15.16:0/673559218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d701002f0 0x7f1d701006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.446+0000 7f1d77d6e640 1 -- 172.21.15.16:0/673559218 >> 172.21.15.16:0/673559218 conn(0x7f1d70077210 msgr2=0x7f1d700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.446+0000 7f1d77d6e640 1 -- 172.21.15.16:0/673559218 shutdown_connections 2024-09-19T09:30:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.446+0000 7f1d77d6e640 1 -- 172.21.15.16:0/673559218 wait complete. 2024-09-19T09:30:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.446+0000 7f1d77d6e640 1 Processor -- start 2024-09-19T09:30:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.446+0000 7f1d77d6e640 1 -- start start 2024-09-19T09:30:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.446+0000 7f1d77d6e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d701002f0 0x7f1d70101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d77d6e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d70100ca0 0x7f1d701017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d77d6e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d70106f50 0x7f1d70101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d77d6e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70111fa0 con 0x7f1d701002f0 2024-09-19T09:30:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d77d6e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70111e20 con 0x7f1d70106f50 2024-09-19T09:30:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d77d6e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70112120 con 0x7f1d70100ca0 2024-09-19T09:30:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d75b0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d701002f0 0x7f1d70101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d75b0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d701002f0 0x7f1d70101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40984/0 (socket says 172.21.15.16:40984) 2024-09-19T09:30:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d75b0a640 1 -- 172.21.15.16:0/2911665948 learned_addr learned my addr 172.21.15.16:0/2911665948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:30:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d75309640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d70100ca0 0x7f1d701017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d7630b640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d70106f50 0x7f1d70101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d75b0a640 1 -- 172.21.15.16:0/2911665948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d70100ca0 msgr2=0x7f1d701017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:20.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d75b0a640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d70100ca0 0x7f1d701017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d75b0a640 1 -- 172.21.15.16:0/2911665948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d70106f50 msgr2=0x7f1d70101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d75b0a640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d70106f50 0x7f1d70101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d75b0a640 1 -- 172.21.15.16:0/2911665948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d701a6960 con 0x7f1d701002f0 2024-09-19T09:30:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d75b0a640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d701002f0 0x7f1d70101270 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00d950 tx=0x7f1d6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d66ffd640 1 -- 172.21.15.16:0/2911665948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c017070 con 0x7f1d701002f0 2024-09-19T09:30:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d66ffd640 1 -- 172.21.15.16:0/2911665948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d6c009c30 con 0x7f1d701002f0 2024-09-19T09:30:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.447+0000 7f1d77d6e640 1 -- 172.21.15.16:0/2911665948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d701a6c50 con 0x7f1d701002f0 2024-09-19T09:30:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.448+0000 7f1d66ffd640 1 -- 172.21.15.16:0/2911665948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00f6b0 con 0x7f1d701002f0 2024-09-19T09:30:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.448+0000 7f1d77d6e640 1 -- 172.21.15.16:0/2911665948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d701a71b0 con 0x7f1d701002f0 2024-09-19T09:30:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.449+0000 7f1d77d6e640 1 -- 172.21.15.16:0/2911665948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d701006d0 con 0x7f1d701002f0 2024-09-19T09:30:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.450+0000 7f1d66ffd640 1 -- 172.21.15.16:0/2911665948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f1d6c0109d0 con 0x7f1d701002f0 2024-09-19T09:30:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.451+0000 7f1d66ffd640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d5c0a32c0 0x7f1d5c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.451+0000 7f1d75309640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d5c0a32c0 0x7f1d5c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.451+0000 7f1d66ffd640 1 -- 172.21.15.16:0/2911665948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f1d6c0cf4c0 con 0x7f1d701002f0 2024-09-19T09:30:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.451+0000 7f1d75309640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d5c0a32c0 0x7f1d5c0a5780 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1d60004770 tx=0x7f1d600058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.454+0000 7f1d66ffd640 1 -- 172.21.15.16:0/2911665948 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1d6c009de0 con 0x7f1d701002f0 2024-09-19T09:30:20.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.668+0000 7f1d77d6e640 1 -- 172.21.15.16:0/2911665948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d701027a0 con 0x7f1d701002f0 2024-09-19T09:30:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.669+0000 7f1d66ffd640 1 -- 172.21.15.16:0/2911665948 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f1d701027a0 con 0x7f1d701002f0 2024-09-19T09:30:20.670 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:30:20.670 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-19T09:27:18:867820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3514257756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3514257756},{"type":"v1","addr":"172.21.15.16:6835","nonce":3514257756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2828730634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2828730634},{"type":"v1","addr":"172.21.15.16:6837","nonce":2828730634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T09:25:29.829112+0000","modified":"2024-09-19T09:27:18.729819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24484,"mds_2":24470,"mds_3":24428,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3573603820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3573603820},{"type":"v1","addr":"172.21.15.16:6839","nonce":3573603820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/4017062522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":4017062522},{"type":"v1","addr":"172.21.15.16:6841","nonce":4017062522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6833/3906366829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3906366829},{"type":"v1","addr":"172.21.15.142:6833","nonce":3906366829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.97:6833/108652274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":108652274},{"type":"v1","addr":"172.21.15.97:6833","nonce":108652274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2657443138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2657443138},{"type":"v1","addr":"172.21.15.142:6835","nonce":2657443138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/4135704970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4135704970},{"type":"v1","addr":"172.21.15.97:6835","nonce":4135704970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6837/81091251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":81091251},{"type":"v1","addr":"172.21.15.142:6837","nonce":81091251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6837/2018284957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2018284957},{"type":"v1","addr":"172.21.15.97:6837","nonce":2018284957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6839/3822134597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":3822134597},{"type":"v1","addr":"172.21.15.142:6839","nonce":3822134597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6839/3225392797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3225392797},{"type":"v1","addr":"172.21.15.97:6839","nonce":3225392797}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24424,24428,24470,24484,24500]},"id":1}]} 2024-09-19T09:30:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-19T09:30:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.671+0000 7f1d77d6e640 1 -- 172.21.15.16:0/2911665948 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d5c0a32c0 msgr2=0x7f1d5c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.671+0000 7f1d77d6e640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d5c0a32c0 0x7f1d5c0a5780 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1d60004770 tx=0x7f1d600058d0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.671+0000 7f1d77d6e640 1 -- 172.21.15.16:0/2911665948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d701002f0 msgr2=0x7f1d70101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.671+0000 7f1d77d6e640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d701002f0 0x7f1d70101270 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00d950 tx=0x7f1d6c00de10 comp rx=0 tx=0).stop 2024-09-19T09:30:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.671+0000 7f1d77d6e640 1 -- 172.21.15.16:0/2911665948 shutdown_connections 2024-09-19T09:30:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.671+0000 7f1d77d6e640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d5c0a32c0 0x7f1d5c0a5780 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.671+0000 7f1d77d6e640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d70100ca0 0x7f1d701017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.671+0000 7f1d77d6e640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d70106f50 0x7f1d70101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.671+0000 7f1d77d6e640 1 --2- 172.21.15.16:0/2911665948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d701002f0 0x7f1d70101270 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.671+0000 7f1d77d6e640 1 -- 172.21.15.16:0/2911665948 >> 172.21.15.16:0/2911665948 conn(0x7f1d70077210 msgr2=0x7f1d700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.671+0000 7f1d77d6e640 1 -- 172.21.15.16:0/2911665948 shutdown_connections 2024-09-19T09:30:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:20.671+0000 7f1d77d6e640 1 -- 172.21.15.16:0/2911665948 wait complete. 2024-09-19T09:30:21.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:21 smithi142 ceph-mon[35032]: pgmap v420: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-19T09:30:21.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:21 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2911665948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:21.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:21 smithi097 ceph-mon[34864]: pgmap v420: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-19T09:30:21.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:21 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2911665948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:21.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:21 smithi016 ceph-mon[30979]: pgmap v420: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-19T09:30:21.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:21 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2911665948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T09:30:21.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T09:30:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.311+0000 7f1d2bd53640 1 -- 172.21.15.16:0/4262167010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d24100e20 msgr2=0x7f1d24101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.311+0000 7f1d2bd53640 1 --2- 172.21.15.16:0/4262167010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d24100e20 0x7f1d24101200 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009900 tx=0x7f1d2002a480 comp rx=0 tx=0).stop 2024-09-19T09:30:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.311+0000 7f1d2bd53640 1 -- 172.21.15.16:0/4262167010 shutdown_connections 2024-09-19T09:30:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.311+0000 7f1d2bd53640 1 --2- 172.21.15.16:0/4262167010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d24102ef0 0x7f1d2410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.311+0000 7f1d2bd53640 1 --2- 172.21.15.16:0/4262167010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24102570 0x7f1d241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.311+0000 7f1d2bd53640 1 --2- 172.21.15.16:0/4262167010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d24100e20 0x7f1d24101200 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.311+0000 7f1d2bd53640 1 -- 172.21.15.16:0/4262167010 >> 172.21.15.16:0/4262167010 conn(0x7f1d240fc7d0 msgr2=0x7f1d240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.311+0000 7f1d2bd53640 1 -- 172.21.15.16:0/4262167010 shutdown_connections 2024-09-19T09:30:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.311+0000 7f1d2bd53640 1 -- 172.21.15.16:0/4262167010 wait complete. 2024-09-19T09:30:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.311+0000 7f1d2bd53640 1 Processor -- start 2024-09-19T09:30:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d2bd53640 1 -- start start 2024-09-19T09:30:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d2bd53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d24100e20 0x7f1d24198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d2bd53640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d24102570 0x7f1d24198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d2bd53640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24102ef0 0x7f1d2419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d2bd53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24110fc0 con 0x7f1d24100e20 2024-09-19T09:30:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d2bd53640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24110e40 con 0x7f1d24102ef0 2024-09-19T09:30:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d2bd53640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24111140 con 0x7f1d24102570 2024-09-19T09:30:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d29aef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d24100e20 0x7f1d24198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d29aef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d24100e20 0x7f1d24198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40990/0 (socket says 172.21.15.16:40990) 2024-09-19T09:30:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d29aef640 1 -- 172.21.15.16:0/3282284592 learned_addr learned my addr 172.21.15.16:0/3282284592 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:30:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d292ee640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d24102570 0x7f1d24198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d2a2f0640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24102ef0 0x7f1d2419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d29aef640 1 -- 172.21.15.16:0/3282284592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d24102570 msgr2=0x7f1d24198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d29aef640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d24102570 0x7f1d24198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d29aef640 1 -- 172.21.15.16:0/3282284592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24102ef0 msgr2=0x7f1d2419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d29aef640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24102ef0 0x7f1d2419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d29aef640 1 -- 172.21.15.16:0/3282284592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2419d4b0 con 0x7f1d24100e20 2024-09-19T09:30:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.312+0000 7f1d29aef640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d24100e20 0x7f1d24198500 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1d20006010 tx=0x7f1d20003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.313+0000 7f1d1affd640 1 -- 172.21.15.16:0/3282284592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2003f070 con 0x7f1d24100e20 2024-09-19T09:30:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.313+0000 7f1d1affd640 1 -- 172.21.15.16:0/3282284592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d20031c30 con 0x7f1d24100e20 2024-09-19T09:30:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.313+0000 7f1d2bd53640 1 -- 172.21.15.16:0/3282284592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2419d740 con 0x7f1d24100e20 2024-09-19T09:30:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.313+0000 7f1d1affd640 1 -- 172.21.15.16:0/3282284592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20035030 con 0x7f1d24100e20 2024-09-19T09:30:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.313+0000 7f1d2bd53640 1 -- 172.21.15.16:0/3282284592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d241a5080 con 0x7f1d24100e20 2024-09-19T09:30:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.314+0000 7f1d2bd53640 1 -- 172.21.15.16:0/3282284592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d24101200 con 0x7f1d24100e20 2024-09-19T09:30:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.315+0000 7f1d1affd640 1 -- 172.21.15.16:0/3282284592 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f1d20042070 con 0x7f1d24100e20 2024-09-19T09:30:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.315+0000 7f1d1affd640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d100a3240 0x7f1d100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.316+0000 7f1d292ee640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d100a3240 0x7f1d100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.316+0000 7f1d1affd640 1 -- 172.21.15.16:0/3282284592 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7f1d200f0020 con 0x7f1d24100e20 2024-09-19T09:30:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.316+0000 7f1d292ee640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d100a3240 0x7f1d100a5700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1d240fffa0 tx=0x7f1d14009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.320+0000 7f1d1affd640 1 -- 172.21.15.16:0/3282284592 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1d200bcd40 con 0x7f1d24100e20 2024-09-19T09:30:22.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.500+0000 7f1d2bd53640 1 -- 172.21.15.16:0/3282284592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d2410a0a0 con 0x7f1d24100e20 2024-09-19T09:30:22.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.501+0000 7f1d1affd640 1 -- 172.21.15.16:0/3282284592 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22665 (secure 0 0 0) 0x7f1d200b8070 con 0x7f1d24100e20 2024-09-19T09:30:22.503 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-19T09:30:22.503 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"ee1ab676-7667-11ef-bceb-c7b262605968","created":"2024-09-19T09:17:48.242238+0000","modified":"2024-09-19T09:25:42.982671+0000","last_up_change":"2024-09-19T09:24:40.190289+0000","last_in_change":"2024-09-19T09:24:17.190380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T09:20:57.748686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T09:25:25.694451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T09:25:28.262997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dbf3b28b-2282-4057-9e65-19ceb78ff19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6803","nonce":1578957623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6805","nonce":1578957623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6809","nonce":1578957623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1578957623},{"type":"v1","addr":"172.21.15.16:6807","nonce":1578957623}]},"public_addr":"172.21.15.16:6803/1578957623","cluster_addr":"172.21.15.16:6805/1578957623","heartbeat_back_addr":"172.21.15.16:6809/1578957623","heartbeat_front_addr":"172.21.15.16:6807/1578957623","state":["exists","up"]},{"osd":1,"uuid":"c2517aeb-1d79-41ad-a180-369eecb9732d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6801","nonce":3804152349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6803","nonce":3804152349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6807","nonce":3804152349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3804152349},{"type":"v1","addr":"172.21.15.97:6805","nonce":3804152349}]},"public_addr":"172.21.15.97:6801/3804152349","cluster_addr":"172.21.15.97:6803/3804152349","heartbeat_back_addr":"172.21.15.97:6807/3804152349","heartbeat_front_addr":"172.21.15.97:6805/3804152349","state":["exists","up"]},{"osd":2,"uuid":"3c367518-5085-471a-98cb-ec5bf5facdf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6801","nonce":2692613951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6803","nonce":2692613951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6807","nonce":2692613951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":2692613951},{"type":"v1","addr":"172.21.15.142:6805","nonce":2692613951}]},"public_addr":"172.21.15.142:6801/2692613951","cluster_addr":"172.21.15.142:6803/2692613951","heartbeat_back_addr":"172.21.15.142:6807/2692613951","heartbeat_front_addr":"172.21.15.142:6805/2692613951","state":["exists","up"]},{"osd":3,"uuid":"e7118925-f0fa-46b3-8552-3e3bb76a6489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6811","nonce":3169290004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6813","nonce":3169290004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6817","nonce":3169290004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3169290004},{"type":"v1","addr":"172.21.15.16:6815","nonce":3169290004}]},"public_addr":"172.21.15.16:6811/3169290004","cluster_addr":"172.21.15.16:6813/3169290004","heartbeat_back_addr":"172.21.15.16:6817/3169290004","heartbeat_front_addr":"172.21.15.16:6815/3169290004","state":["exists","up"]},{"osd":4,"uuid":"dc1964f7-af9c-4853-843b-9818f87c7768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6809","nonce":3491353825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6811","nonce":3491353825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6815","nonce":3491353825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3491353825},{"type":"v1","addr":"172.21.15.97:6813","nonce":3491353825}]},"public_addr":"172.21.15.97:6809/3491353825","cluster_addr":"172.21.15.97:6811/3491353825","heartbeat_back_addr":"172.21.15.97:6815/3491353825","heartbeat_front_addr":"172.21.15.97:6813/3491353825","state":["exists","up"]},{"osd":5,"uuid":"4d7fbafc-d9ba-4036-a89d-aa295b8a23a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6809","nonce":577099986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6811","nonce":577099986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6815","nonce":577099986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":577099986},{"type":"v1","addr":"172.21.15.142:6813","nonce":577099986}]},"public_addr":"172.21.15.142:6809/577099986","cluster_addr":"172.21.15.142:6811/577099986","heartbeat_back_addr":"172.21.15.142:6815/577099986","heartbeat_front_addr":"172.21.15.142:6813/577099986","state":["exists","up"]},{"osd":6,"uuid":"e3f346fc-c027-44f8-bd37-bccf6ec09e3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6819","nonce":4053539536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6821","nonce":4053539536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6825","nonce":4053539536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4053539536},{"type":"v1","addr":"172.21.15.16:6823","nonce":4053539536}]},"public_addr":"172.21.15.16:6819/4053539536","cluster_addr":"172.21.15.16:6821/4053539536","heartbeat_back_addr":"172.21.15.16:6825/4053539536","heartbeat_front_addr":"172.21.15.16:6823/4053539536","state":["exists","up"]},{"osd":7,"uuid":"d390b547-8949-4d4b-8481-c49d6ae7e424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6817","nonce":1157122670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6819","nonce":1157122670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6823","nonce":1157122670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1157122670},{"type":"v1","addr":"172.21.15.97:6821","nonce":1157122670}]},"public_addr":"172.21.15.97:6817/1157122670","cluster_addr":"172.21.15.97:6819/1157122670","heartbeat_back_addr":"172.21.15.97:6823/1157122670","heartbeat_front_addr":"172.21.15.97:6821/1157122670","state":["exists","up"]},{"osd":8,"uuid":"78a476c4-5deb-4f9e-a266-db4493850732","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6817","nonce":643795004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6819","nonce":643795004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6823","nonce":643795004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":643795004},{"type":"v1","addr":"172.21.15.142:6821","nonce":643795004}]},"public_addr":"172.21.15.142:6817/643795004","cluster_addr":"172.21.15.142:6819/643795004","heartbeat_back_addr":"172.21.15.142:6823/643795004","heartbeat_front_addr":"172.21.15.142:6821/643795004","state":["exists","up"]},{"osd":9,"uuid":"0415d187-4a77-42f7-b8bb-2d0b52e9cb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6827","nonce":2651132360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6829","nonce":2651132360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2651132360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2651132360},{"type":"v1","addr":"172.21.15.16:6831","nonce":2651132360}]},"public_addr":"172.21.15.16:6827/2651132360","cluster_addr":"172.21.15.16:6829/2651132360","heartbeat_back_addr":"172.21.15.16:6833/2651132360","heartbeat_front_addr":"172.21.15.16:6831/2651132360","state":["exists","up"]},{"osd":10,"uuid":"6fa78f96-cdfe-42a0-8d27-0705a0c7df8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6825","nonce":1848371486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6827","nonce":1848371486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6831","nonce":1848371486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1848371486},{"type":"v1","addr":"172.21.15.97:6829","nonce":1848371486}]},"public_addr":"172.21.15.97:6825/1848371486","cluster_addr":"172.21.15.97:6827/1848371486","heartbeat_back_addr":"172.21.15.97:6831/1848371486","heartbeat_front_addr":"172.21.15.97:6829/1848371486","state":["exists","up"]},{"osd":11,"uuid":"d27f20e3-84ce-470c-9889-cd86906b158d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6825","nonce":155083878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6827","nonce":155083878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6831","nonce":155083878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":155083878},{"type":"v1","addr":"172.21.15.142:6829","nonce":155083878}]},"public_addr":"172.21.15.142:6825/155083878","cluster_addr":"172.21.15.142:6827/155083878","heartbeat_back_addr":"172.21.15.142:6831/155083878","heartbeat_front_addr":"172.21.15.142:6829/155083878","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:05.088764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:29.928529+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:20:55.236930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:20.379471+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:21:44.984151+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:10.410074+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:35.233553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:22:59.928700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:24.503069+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:23:49.853073+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:14.006841+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T09:24:38.546467+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3710267992":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/1680178094":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/3740275561":"2024-09-20T09:18:41.244076+0000","172.21.15.16:0/2755421444":"2024-09-20T09:18:41.244076+0000","172.21.15.16:6801/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/357390846":"2024-09-20T09:18:17.070034+0000","172.21.15.16:6800/183909517":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3104087402":"2024-09-20T09:18:17.070034+0000","172.21.15.16:0/3546947161":"2024-09-20T09:18:17.070034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T09:30:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.503+0000 7f1d2bd53640 1 -- 172.21.15.16:0/3282284592 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d100a3240 msgr2=0x7f1d100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.503+0000 7f1d2bd53640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d100a3240 0x7f1d100a5700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1d240fffa0 tx=0x7f1d14009110 comp rx=0 tx=0).stop 2024-09-19T09:30:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.503+0000 7f1d2bd53640 1 -- 172.21.15.16:0/3282284592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d24100e20 msgr2=0x7f1d24198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.503+0000 7f1d2bd53640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d24100e20 0x7f1d24198500 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1d20006010 tx=0x7f1d20003c10 comp rx=0 tx=0).stop 2024-09-19T09:30:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.503+0000 7f1d2bd53640 1 -- 172.21.15.16:0/3282284592 shutdown_connections 2024-09-19T09:30:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.503+0000 7f1d2bd53640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f1d100a3240 0x7f1d100a5700 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.503+0000 7f1d2bd53640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d24102570 0x7f1d24198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.503+0000 7f1d2bd53640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24102ef0 0x7f1d2419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.503+0000 7f1d2bd53640 1 --2- 172.21.15.16:0/3282284592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d24100e20 0x7f1d24198500 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.503+0000 7f1d2bd53640 1 -- 172.21.15.16:0/3282284592 >> 172.21.15.16:0/3282284592 conn(0x7f1d240fc7d0 msgr2=0x7f1d2406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.504+0000 7f1d2bd53640 1 -- 172.21.15.16:0/3282284592 shutdown_connections 2024-09-19T09:30:22.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:22.504+0000 7f1d2bd53640 1 -- 172.21.15.16:0/3282284592 wait complete. 2024-09-19T09:30:23.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T09:30:23.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:23 smithi016 ceph-mon[30979]: pgmap v421: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T09:30:23.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:23 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3282284592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:30:23.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:23 smithi142 ceph-mon[35032]: pgmap v421: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T09:30:23.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:23 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3282284592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:30:23.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:23 smithi097 ceph-mon[34864]: pgmap v421: 97 pgs: 97 active+clean; 8.7 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-19T09:30:23.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:23 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3282284592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T09:30:23.389 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:23.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.879+0000 7fa4dc15a640 1 -- 172.21.15.16:0/826063806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d4112b90 msgr2=0x7fa4d4114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:23.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.879+0000 7fa4dc15a640 1 --2- 172.21.15.16:0/826063806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d4112b90 0x7fa4d4114f80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009ca0 tx=0x7fa4cc02a950 comp rx=0 tx=0).stop 2024-09-19T09:30:23.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.879+0000 7fa4dc15a640 1 -- 172.21.15.16:0/826063806 shutdown_connections 2024-09-19T09:30:23.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.879+0000 7fa4dc15a640 1 --2- 172.21.15.16:0/826063806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d4112b90 0x7fa4d4114f80 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:23.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.879+0000 7fa4dc15a640 1 --2- 172.21.15.16:0/826063806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d4074eb0 0x7fa4d4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:23.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.879+0000 7fa4dc15a640 1 --2- 172.21.15.16:0/826063806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4d4074590 0x7fa4d4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:23.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.879+0000 7fa4dc15a640 1 -- 172.21.15.16:0/826063806 >> 172.21.15.16:0/826063806 conn(0x7fa4d40fd8c0 msgr2=0x7fa4d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:23.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.880+0000 7fa4dc15a640 1 -- 172.21.15.16:0/826063806 shutdown_connections 2024-09-19T09:30:23.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.880+0000 7fa4dc15a640 1 -- 172.21.15.16:0/826063806 wait complete. 2024-09-19T09:30:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.880+0000 7fa4dc15a640 1 Processor -- start 2024-09-19T09:30:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.880+0000 7fa4dc15a640 1 -- start start 2024-09-19T09:30:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.880+0000 7fa4dc15a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4d4074590 0x7fa4d41a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.880+0000 7fa4dc15a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d4074eb0 0x7fa4d41a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.880+0000 7fa4dc15a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d4112b90 0x7fa4d419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.880+0000 7fa4dc15a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4118560 con 0x7fa4d4112b90 2024-09-19T09:30:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.880+0000 7fa4dc15a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d41183e0 con 0x7fa4d4074eb0 2024-09-19T09:30:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.880+0000 7fa4dc15a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d41186e0 con 0x7fa4d4074590 2024-09-19T09:30:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4da6f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d4112b90 0x7fa4d419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4da6f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d4112b90 0x7fa4d419e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41004/0 (socket says 172.21.15.16:41004) 2024-09-19T09:30:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4da6f7640 1 -- 172.21.15.16:0/3829538071 learned_addr learned my addr 172.21.15.16:0/3829538071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:30:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4d9ef6640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4d4074590 0x7fa4d41a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:23.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4d96f5640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d4074eb0 0x7fa4d41a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:23.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4da6f7640 1 -- 172.21.15.16:0/3829538071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4d4074590 msgr2=0x7fa4d41a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:23.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4da6f7640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4d4074590 0x7fa4d41a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:23.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4da6f7640 1 -- 172.21.15.16:0/3829538071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d4074eb0 msgr2=0x7fa4d41a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:23.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4da6f7640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d4074eb0 0x7fa4d41a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:23.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4da6f7640 1 -- 172.21.15.16:0/3829538071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d419efb0 con 0x7fa4d4112b90 2024-09-19T09:30:23.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4d9ef6640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4d4074590 0x7fa4d41a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:30:23.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4d96f5640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d4074eb0 0x7fa4d41a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:30:23.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4da6f7640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d4112b90 0x7fa4d419e810 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc02ae70 tx=0x7fa4cc02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:23.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4caffd640 1 -- 172.21.15.16:0/3829538071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc034040 con 0x7fa4d4112b90 2024-09-19T09:30:23.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4dc15a640 1 -- 172.21.15.16:0/3829538071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d419f240 con 0x7fa4d4112b90 2024-09-19T09:30:23.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4caffd640 1 -- 172.21.15.16:0/3829538071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4cc030920 con 0x7fa4d4112b90 2024-09-19T09:30:23.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4caffd640 1 -- 172.21.15.16:0/3829538071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc045480 con 0x7fa4d4112b90 2024-09-19T09:30:23.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.881+0000 7fa4dc15a640 1 -- 172.21.15.16:0/3829538071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d41ab4f0 con 0x7fa4d4112b90 2024-09-19T09:30:23.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.882+0000 7fa4dc15a640 1 -- 172.21.15.16:0/3829538071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d4075cd0 con 0x7fa4d4112b90 2024-09-19T09:30:23.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.884+0000 7fa4caffd640 1 -- 172.21.15.16:0/3829538071 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa4cc037020 con 0x7fa4d4112b90 2024-09-19T09:30:23.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.884+0000 7fa4caffd640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa4c00a3240 0x7fa4c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:23.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.884+0000 7fa4d9ef6640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa4c00a3240 0x7fa4c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:23.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.884+0000 7fa4caffd640 1 -- 172.21.15.16:0/3829538071 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8240+0+0 (secure 0 0 0) 0x7fa4cc0f08b0 con 0x7fa4d4112b90 2024-09-19T09:30:23.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.885+0000 7fa4d9ef6640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa4c00a3240 0x7fa4c00a5700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00058c0 tx=0x7fa4d0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:23.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:23.889+0000 7fa4caffd640 1 -- 172.21.15.16:0/3829538071 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa4cc0bd5d0 con 0x7fa4d4112b90 2024-09-19T09:30:24.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.107+0000 7fa4dc15a640 1 -- 172.21.15.16:0/3829538071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa4d41ab6c0 con 0x7fa4d4112b90 2024-09-19T09:30:24.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.109+0000 7fa4caffd640 1 -- 172.21.15.16:0/3829538071 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7fa4cc043e40 con 0x7fa4d4112b90 2024-09-19T09:30:24.110 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T09:30:24.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.113+0000 7fa4dc15a640 1 -- 172.21.15.16:0/3829538071 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa4c00a3240 msgr2=0x7fa4c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.113+0000 7fa4dc15a640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa4c00a3240 0x7fa4c00a5700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00058c0 tx=0x7fa4d0005850 comp rx=0 tx=0).stop 2024-09-19T09:30:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.113+0000 7fa4dc15a640 1 -- 172.21.15.16:0/3829538071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d4112b90 msgr2=0x7fa4d419e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.113+0000 7fa4dc15a640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d4112b90 0x7fa4d419e810 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc02ae70 tx=0x7fa4cc02f5b0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.113+0000 7fa4dc15a640 1 -- 172.21.15.16:0/3829538071 shutdown_connections 2024-09-19T09:30:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.113+0000 7fa4dc15a640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4d4112b90 0x7fa4d419e810 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.113+0000 7fa4dc15a640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d4074eb0 0x7fa4d41a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.113+0000 7fa4dc15a640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa4c00a3240 0x7fa4c00a5700 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.113+0000 7fa4dc15a640 1 --2- 172.21.15.16:0/3829538071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4d4074590 0x7fa4d41a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.113+0000 7fa4dc15a640 1 -- 172.21.15.16:0/3829538071 >> 172.21.15.16:0/3829538071 conn(0x7fa4d40fd8c0 msgr2=0x7fa4d40fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:24.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.114+0000 7fa4dc15a640 1 -- 172.21.15.16:0/3829538071 shutdown_connections 2024-09-19T09:30:24.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:24.114+0000 7fa4dc15a640 1 -- 172.21.15.16:0/3829538071 wait complete. 2024-09-19T09:30:24.476 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:24 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a[30975]: 2024-09-19T09:30:24.107+0000 7f37e18e4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T09:30:24.476 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:30:24 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-g[69351]: ignoring --setuser ceph since I am not root 2024-09-19T09:30:24.476 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:30:24 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-g[69351]: ignoring --setgroup ceph since I am not root 2024-09-19T09:30:24.477 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:30:24 smithi016 ceph-mds[69373]: -- 172.21.15.16:0/3927872003 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed4ace9860 con 0x55ed4acc6800 2024-09-19T09:30:24.477 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:30:24 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-g[69351]: starting mds.g at 2024-09-19T09:30:24.477 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:30:24 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-a[69766]: ignoring --setuser ceph since I am not root 2024-09-19T09:30:24.477 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:30:24 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-a[69766]: ignoring --setgroup ceph since I am not root 2024-09-19T09:30:24.477 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:30:24 smithi016 ceph-mds[69770]: -- 172.21.15.16:0/986623229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d7bc317860 con 0x55d7bc2f5000 2024-09-19T09:30:24.477 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:30:24 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-a[69766]: starting mds.a at 2024-09-19T09:30:24.629 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-c[65010]: ignoring --setuser ceph since I am not root 2024-09-19T09:30:24.629 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-c[65010]: ignoring --setgroup ceph since I am not root 2024-09-19T09:30:24.629 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-mds[65014]: -- 172.21.15.142:0/15116546 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e87e5fc20 con 0x561e87e3d000 2024-09-19T09:30:24.629 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-c[65010]: starting mds.c at 2024-09-19T09:30:24.629 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-f[64706]: ignoring --setuser ceph since I am not root 2024-09-19T09:30:24.630 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-f[64706]: ignoring --setgroup ceph since I am not root 2024-09-19T09:30:24.630 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-mds[64710]: -- 172.21.15.142:0/4139056878 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb16da7c20 con 0x55cb16d85000 2024-09-19T09:30:24.630 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-f[64706]: starting mds.f at 2024-09-19T09:30:24.630 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-i[65309]: ignoring --setuser ceph since I am not root 2024-09-19T09:30:24.630 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-i[65309]: ignoring --setgroup ceph since I am not root 2024-09-19T09:30:24.630 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-mds[65313]: -- 172.21.15.142:0/3349774413 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556cbe673c20 con 0x556cbe650800 2024-09-19T09:30:24.630 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-i[65309]: starting mds.i at 2024-09-19T09:30:24.631 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-l[64407]: ignoring --setuser ceph since I am not root 2024-09-19T09:30:24.631 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-l[64407]: ignoring --setgroup ceph since I am not root 2024-09-19T09:30:24.631 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-mds[64411]: -- 172.21.15.142:0/3640018376 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558cc826d860 con 0x558cc824b000 2024-09-19T09:30:24.631 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:30:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-l[64407]: starting mds.l at 2024-09-19T09:30:24.636 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b[64788]: ignoring --setuser ceph since I am not root 2024-09-19T09:30:24.636 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b[64788]: ignoring --setgroup ceph since I am not root 2024-09-19T09:30:24.636 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-mds[64792]: -- 172.21.15.97:0/1846818676 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc48d57860 con 0x55fc48d35000 2024-09-19T09:30:24.637 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b[64788]: starting mds.b at 2024-09-19T09:30:24.637 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-e[64185]: ignoring --setuser ceph since I am not root 2024-09-19T09:30:24.637 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-e[64185]: ignoring --setgroup ceph since I am not root 2024-09-19T09:30:24.637 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-mds[64189]: -- 172.21.15.97:0/890457768 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559ac35b1860 con 0x559ac358e800 2024-09-19T09:30:24.637 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-e[64185]: starting mds.e at 2024-09-19T09:30:24.637 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h[64488]: ignoring --setuser ceph since I am not root 2024-09-19T09:30:24.637 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h[64488]: ignoring --setgroup ceph since I am not root 2024-09-19T09:30:24.638 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-mds[64492]: -- 172.21.15.97:0/3375844656 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556acd32f860 con 0x556acd30d400 2024-09-19T09:30:24.638 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h[64488]: starting mds.h at 2024-09-19T09:30:24.638 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h[64488]: 2024-09-19T09:30:24.155+0000 7fe9706da640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-19T09:30:24.638 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h[64488]: 2024-09-19T09:30:24.156+0000 7fe9706da640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6834/0: (98) Address already in use 2024-09-19T09:30:24.638 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-k[63870]: ignoring --setuser ceph since I am not root 2024-09-19T09:30:24.638 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-k[63870]: ignoring --setgroup ceph since I am not root 2024-09-19T09:30:24.638 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-mds[63874]: -- 172.21.15.97:0/1291873916 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560853add860 con 0x560853abb000 2024-09-19T09:30:24.638 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:30:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-k[63870]: starting mds.k at 2024-09-19T09:30:24.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T09:30:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:25.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:25 smithi016 ceph-mon[30979]: pgmap v422: 97 pgs: 97 active+clean; 8.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-19T09:30:25.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:25 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3829538071' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T09:30:25.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:25 smithi016 ceph-mon[30979]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T09:30:25.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:25 smithi016 ceph-mon[30979]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T09:30:25.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:25 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/3829538071' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T09:30:25.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:25 smithi016 ceph-mon[30979]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T09:30:25.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:25 smithi016 ceph-mon[30979]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T09:30:25.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:25 smithi142 ceph-mon[35032]: pgmap v422: 97 pgs: 97 active+clean; 8.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-19T09:30:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:25 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3829538071' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T09:30:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:25 smithi142 ceph-mon[35032]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T09:30:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:25 smithi142 ceph-mon[35032]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T09:30:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:25 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/3829538071' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T09:30:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:25 smithi142 ceph-mon[35032]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T09:30:25.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:25 smithi142 ceph-mon[35032]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T09:30:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:25 smithi097 ceph-mon[34864]: pgmap v422: 97 pgs: 97 active+clean; 8.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-19T09:30:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:25 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3829538071' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T09:30:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:25 smithi097 ceph-mon[34864]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T09:30:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:25 smithi097 ceph-mon[34864]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T09:30:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:25 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/3829538071' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T09:30:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:25 smithi097 ceph-mon[34864]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T09:30:25.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:25 smithi097 ceph-mon[34864]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T09:30:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.656+0000 7fcedba5b640 1 -- 172.21.15.16:0/214335694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced4074510 msgr2=0x7fced40748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.656+0000 7fcedba5b640 1 --2- 172.21.15.16:0/214335694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced4074510 0x7fced40748f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009be0 tx=0x7fcec402a750 comp rx=0 tx=0).stop 2024-09-19T09:30:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.656+0000 7fcedba5b640 1 -- 172.21.15.16:0/214335694 shutdown_connections 2024-09-19T09:30:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.656+0000 7fcedba5b640 1 --2- 172.21.15.16:0/214335694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fced4074e30 0x7fced410df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.656+0000 7fcedba5b640 1 --2- 172.21.15.16:0/214335694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fced410e440 0x7fced4110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.656+0000 7fcedba5b640 1 --2- 172.21.15.16:0/214335694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced4074510 0x7fced40748f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.656+0000 7fcedba5b640 1 -- 172.21.15.16:0/214335694 >> 172.21.15.16:0/214335694 conn(0x7fced40fd880 msgr2=0x7fced40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.656+0000 7fcedba5b640 1 -- 172.21.15.16:0/214335694 shutdown_connections 2024-09-19T09:30:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.656+0000 7fcedba5b640 1 -- 172.21.15.16:0/214335694 wait complete. 2024-09-19T09:30:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fcedba5b640 1 Processor -- start 2024-09-19T09:30:25.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fcedba5b640 1 -- start start 2024-09-19T09:30:25.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fcedba5b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fced4074510 0x7fced419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:25.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fcedba5b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fced4074e30 0x7fced419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:25.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fcedba5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced410e440 0x7fced419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:25.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fcedba5b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fced41140f0 con 0x7fced410e440 2024-09-19T09:30:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fcedba5b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fced4113f70 con 0x7fced4074e30 2024-09-19T09:30:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fcedba5b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fced4114270 con 0x7fced4074510 2024-09-19T09:30:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fced9ff8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced410e440 0x7fced419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fced9ff8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced410e440 0x7fced419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51082/0 (socket says 172.21.15.16:51082) 2024-09-19T09:30:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fced9ff8640 1 -- 172.21.15.16:0/788471669 learned_addr learned my addr 172.21.15.16:0/788471669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:30:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fced97f7640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fced4074510 0x7fced419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fced8ff6640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fced4074e30 0x7fced419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fced9ff8640 1 -- 172.21.15.16:0/788471669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fced4074510 msgr2=0x7fced419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fced9ff8640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fced4074510 0x7fced419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fced9ff8640 1 -- 172.21.15.16:0/788471669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fced4074e30 msgr2=0x7fced419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fced9ff8640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fced4074e30 0x7fced419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fced9ff8640 1 -- 172.21.15.16:0/788471669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced419f5e0 con 0x7fced410e440 2024-09-19T09:30:25.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.657+0000 7fced8ff6640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fced4074e30 0x7fced419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:30:25.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.658+0000 7fced9ff8640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced410e440 0x7fced419ef00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00d720 tx=0x7fcecc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:25.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.658+0000 7fced27fc640 1 -- 172.21.15.16:0/788471669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc017070 con 0x7fced410e440 2024-09-19T09:30:25.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.658+0000 7fced27fc640 1 -- 172.21.15.16:0/788471669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcecc00de30 con 0x7fced410e440 2024-09-19T09:30:25.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.658+0000 7fcedba5b640 1 -- 172.21.15.16:0/788471669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced419f8d0 con 0x7fced410e440 2024-09-19T09:30:25.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.658+0000 7fced27fc640 1 -- 172.21.15.16:0/788471669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc004d90 con 0x7fced410e440 2024-09-19T09:30:25.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.658+0000 7fcedba5b640 1 -- 172.21.15.16:0/788471669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced41a71b0 con 0x7fced410e440 2024-09-19T09:30:25.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.659+0000 7fcedba5b640 1 -- 172.21.15.16:0/788471669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced40748f0 con 0x7fced410e440 2024-09-19T09:30:25.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.660+0000 7fced27fc640 1 -- 172.21.15.16:0/788471669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fcecc004f30 con 0x7fced410e440 2024-09-19T09:30:25.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.661+0000 7fced27fc640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fcebc0a32c0 0x7fcebc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:25.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.661+0000 7fced97f7640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fcebc0a32c0 0x7fcebc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:25.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.661+0000 7fced27fc640 1 -- 172.21.15.16:0/788471669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8670+0+0 (secure 0 0 0) 0x7fcecc0cf360 con 0x7fced410e440 2024-09-19T09:30:25.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.661+0000 7fced97f7640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fcebc0a32c0 0x7fcebc0a5780 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009330 tx=0x7fcec40092c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:25.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.665+0000 7fced27fc640 1 -- 172.21.15.16:0/788471669 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcecc09be70 con 0x7fced410e440 2024-09-19T09:30:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:25.876+0000 7fcedba5b640 1 -- 172.21.15.16:0/788471669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fced402d7c0 con 0x7fced410e440 2024-09-19T09:30:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:26.118+0000 7fced27fc640 1 -- 172.21.15.16:0/788471669 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7fcecc09b770 con 0x7fced410e440 2024-09-19T09:30:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:26.120+0000 7fcedba5b640 1 -- 172.21.15.16:0/788471669 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fcebc0a32c0 msgr2=0x7fcebc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:26.120+0000 7fcedba5b640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fcebc0a32c0 0x7fcebc0a5780 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009330 tx=0x7fcec40092c0 comp rx=0 tx=0).stop 2024-09-19T09:30:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:26.120+0000 7fcedba5b640 1 -- 172.21.15.16:0/788471669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced410e440 msgr2=0x7fced419ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:26.120+0000 7fcedba5b640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced410e440 0x7fced419ef00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00d720 tx=0x7fcecc00dbe0 comp rx=0 tx=0).stop 2024-09-19T09:30:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:26.120+0000 7fcedba5b640 1 -- 172.21.15.16:0/788471669 shutdown_connections 2024-09-19T09:30:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:26.120+0000 7fcedba5b640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced410e440 0x7fced419ef00 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:26.120+0000 7fcedba5b640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fced4074e30 0x7fced419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:26.120+0000 7fcedba5b640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fcebc0a32c0 0x7fcebc0a5780 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:26.120+0000 7fcedba5b640 1 --2- 172.21.15.16:0/788471669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fced4074510 0x7fced419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:26.120+0000 7fcedba5b640 1 -- 172.21.15.16:0/788471669 >> 172.21.15.16:0/788471669 conn(0x7fced40fd880 msgr2=0x7fced40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:26.120+0000 7fcedba5b640 1 -- 172.21.15.16:0/788471669 shutdown_connections 2024-09-19T09:30:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:26.120+0000 7fcedba5b640 1 -- 172.21.15.16:0/788471669 wait complete. 2024-09-19T09:30:26.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.16:6838/692431213,v1:172.21.15.16:6839/692431213] up:boot 2024-09-19T09:30:26.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.97:6833/1954204202,v1:172.21.15.97:6835/1954204202] up:boot 2024-09-19T09:30:26.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.97:6832/2498795265,v1:172.21.15.97:6834/2498795265] up:boot 2024-09-19T09:30:26.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.142:6832/213879490,v1:172.21.15.142:6834/213879490] up:boot 2024-09-19T09:30:26.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.16:6840/2219608368,v1:172.21.15.16:6841/2219608368] up:boot 2024-09-19T09:30:26.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.142:6833/3133697852,v1:172.21.15.142:6835/3133697852] up:boot 2024-09-19T09:30:26.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.97:6836/2674265773,v1:172.21.15.97:6837/2674265773] up:boot 2024-09-19T09:30:26.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.97:6838/3061339781,v1:172.21.15.97:6839/3061339781] up:boot 2024-09-19T09:30:26.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.142:6837/3903039656,v1:172.21.15.142:6839/3903039656] up:boot 2024-09-19T09:30:26.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: mds.? [v2:172.21.15.142:6836/3276755416,v1:172.21.15.142:6838/3276755416] up:boot 2024-09-19T09:30:26.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T09:30:26.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:30:26.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:30:26.133 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:30:26.133 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:30:26.133 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:30:26.133 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:30:26.133 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:30:26.133 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:30:26.133 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:30:26.133 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:30:26.133 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:26 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/788471669' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:30:26.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.16:6838/692431213,v1:172.21.15.16:6839/692431213] up:boot 2024-09-19T09:30:26.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.97:6833/1954204202,v1:172.21.15.97:6835/1954204202] up:boot 2024-09-19T09:30:26.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.97:6832/2498795265,v1:172.21.15.97:6834/2498795265] up:boot 2024-09-19T09:30:26.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.142:6832/213879490,v1:172.21.15.142:6834/213879490] up:boot 2024-09-19T09:30:26.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.16:6840/2219608368,v1:172.21.15.16:6841/2219608368] up:boot 2024-09-19T09:30:26.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.142:6833/3133697852,v1:172.21.15.142:6835/3133697852] up:boot 2024-09-19T09:30:26.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.97:6836/2674265773,v1:172.21.15.97:6837/2674265773] up:boot 2024-09-19T09:30:26.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.97:6838/3061339781,v1:172.21.15.97:6839/3061339781] up:boot 2024-09-19T09:30:26.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.142:6837/3903039656,v1:172.21.15.142:6839/3903039656] up:boot 2024-09-19T09:30:26.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: mds.? [v2:172.21.15.142:6836/3276755416,v1:172.21.15.142:6838/3276755416] up:boot 2024-09-19T09:30:26.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T09:30:26.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:30:26.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:30:26.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:30:26.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:30:26.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:30:26.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:30:26.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:30:26.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:30:26.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:30:26.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:30:26.381 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:26 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/788471669' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:30:26.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.16:6838/692431213,v1:172.21.15.16:6839/692431213] up:boot 2024-09-19T09:30:26.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.97:6833/1954204202,v1:172.21.15.97:6835/1954204202] up:boot 2024-09-19T09:30:26.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.97:6832/2498795265,v1:172.21.15.97:6834/2498795265] up:boot 2024-09-19T09:30:26.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.142:6832/213879490,v1:172.21.15.142:6834/213879490] up:boot 2024-09-19T09:30:26.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.16:6840/2219608368,v1:172.21.15.16:6841/2219608368] up:boot 2024-09-19T09:30:26.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.142:6833/3133697852,v1:172.21.15.142:6835/3133697852] up:boot 2024-09-19T09:30:26.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.97:6836/2674265773,v1:172.21.15.97:6837/2674265773] up:boot 2024-09-19T09:30:26.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.97:6838/3061339781,v1:172.21.15.97:6839/3061339781] up:boot 2024-09-19T09:30:26.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.142:6837/3903039656,v1:172.21.15.142:6839/3903039656] up:boot 2024-09-19T09:30:26.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: mds.? [v2:172.21.15.142:6836/3276755416,v1:172.21.15.142:6838/3276755416] up:boot 2024-09-19T09:30:26.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T09:30:26.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T09:30:26.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T09:30:26.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T09:30:26.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T09:30:26.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T09:30:26.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T09:30:26.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T09:30:26.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T09:30:26.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T09:30:26.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: from='mgr.14150 172.21.15.16:0/1921668581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T09:30:26.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:26 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/788471669' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T09:30:26.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T09:30:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:27.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:27 smithi016 ceph-mon[30979]: pgmap v424: 97 pgs: 97 active+clean; 8.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-19T09:30:27.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:27 smithi016 ceph-mon[30979]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T09:30:27.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:27 smithi016 ceph-mon[30979]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T09:30:27.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:27 smithi016 ceph-mon[30979]: Cluster is now healthy 2024-09-19T09:30:27.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:27 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/788471669' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:30:27.373 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:27 smithi016 ceph-mon[30979]: fsmap 12 up:standby 2024-09-19T09:30:27.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:27 smithi142 ceph-mon[35032]: pgmap v424: 97 pgs: 97 active+clean; 8.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-19T09:30:27.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:27 smithi142 ceph-mon[35032]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T09:30:27.378 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:27 smithi142 ceph-mon[35032]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T09:30:27.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:27 smithi142 ceph-mon[35032]: Cluster is now healthy 2024-09-19T09:30:27.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:27 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/788471669' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:30:27.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:27 smithi142 ceph-mon[35032]: fsmap 12 up:standby 2024-09-19T09:30:27.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:27 smithi097 ceph-mon[34864]: pgmap v424: 97 pgs: 97 active+clean; 8.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-19T09:30:27.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:27 smithi097 ceph-mon[34864]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T09:30:27.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:27 smithi097 ceph-mon[34864]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T09:30:27.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:27 smithi097 ceph-mon[34864]: Cluster is now healthy 2024-09-19T09:30:27.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:27 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/788471669' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T09:30:27.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:27 smithi097 ceph-mon[34864]: fsmap 12 up:standby 2024-09-19T09:30:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.582+0000 7f3d4edad640 1 -- 172.21.15.16:0/758774878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d48100310 msgr2=0x7f3d481006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.582+0000 7f3d4edad640 1 --2- 172.21.15.16:0/758774878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d48100310 0x7f3d481006f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009900 tx=0x7f3d3c02a410 comp rx=0 tx=0).stop 2024-09-19T09:30:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.582+0000 7f3d4edad640 1 -- 172.21.15.16:0/758774878 shutdown_connections 2024-09-19T09:30:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.582+0000 7f3d4edad640 1 --2- 172.21.15.16:0/758774878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d48100cc0 0x7f3d48106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.582+0000 7f3d4edad640 1 --2- 172.21.15.16:0/758774878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d48106f30 0x7f3d4810eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.582+0000 7f3d4edad640 1 --2- 172.21.15.16:0/758774878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d48100310 0x7f3d481006f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.582+0000 7f3d4edad640 1 -- 172.21.15.16:0/758774878 >> 172.21.15.16:0/758774878 conn(0x7f3d48077210 msgr2=0x7f3d480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.582+0000 7f3d4edad640 1 -- 172.21.15.16:0/758774878 shutdown_connections 2024-09-19T09:30:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.582+0000 7f3d4edad640 1 -- 172.21.15.16:0/758774878 wait complete. 2024-09-19T09:30:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.583+0000 7f3d4edad640 1 Processor -- start 2024-09-19T09:30:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.583+0000 7f3d4edad640 1 -- start start 2024-09-19T09:30:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.583+0000 7f3d4edad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d48100310 0x7f3d48101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.583+0000 7f3d4edad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d48100cc0 0x7f3d481017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.583+0000 7f3d4edad640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d48106f30 0x7f3d48101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.583+0000 7f3d4edad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48111f90 con 0x7f3d48100310 2024-09-19T09:30:27.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.583+0000 7f3d4edad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48111e10 con 0x7f3d48100cc0 2024-09-19T09:30:27.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.583+0000 7f3d4edad640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48112110 con 0x7f3d48106f30 2024-09-19T09:30:27.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.583+0000 7f3d4cb49640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d48100310 0x7f3d48101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:27.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.583+0000 7f3d4cb49640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d48100310 0x7f3d48101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51092/0 (socket says 172.21.15.16:51092) 2024-09-19T09:30:27.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.583+0000 7f3d4cb49640 1 -- 172.21.15.16:0/2227044699 learned_addr learned my addr 172.21.15.16:0/2227044699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:30:27.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.583+0000 7f3d47fff640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d48100cc0 0x7f3d481017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:27.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.584+0000 7f3d4d34a640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d48106f30 0x7f3d48101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:27.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.584+0000 7f3d4cb49640 1 -- 172.21.15.16:0/2227044699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d48106f30 msgr2=0x7f3d48101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.584+0000 7f3d4cb49640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d48106f30 0x7f3d48101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.584+0000 7f3d4cb49640 1 -- 172.21.15.16:0/2227044699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d48100cc0 msgr2=0x7f3d481017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.584+0000 7f3d4cb49640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d48100cc0 0x7f3d481017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.584+0000 7f3d4cb49640 1 -- 172.21.15.16:0/2227044699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d481a68d0 con 0x7f3d48100310 2024-09-19T09:30:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.584+0000 7f3d4d34a640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d48106f30 0x7f3d48101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T09:30:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.584+0000 7f3d4cb49640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d48100310 0x7f3d48101290 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c006010 tx=0x7f3d3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.584+0000 7f3d45ffb640 1 -- 172.21.15.16:0/2227044699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c03f030 con 0x7f3d48100310 2024-09-19T09:30:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.584+0000 7f3d45ffb640 1 -- 172.21.15.16:0/2227044699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d3c003ce0 con 0x7f3d48100310 2024-09-19T09:30:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.584+0000 7f3d4edad640 1 -- 172.21.15.16:0/2227044699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d481a6b60 con 0x7f3d48100310 2024-09-19T09:30:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.584+0000 7f3d45ffb640 1 -- 172.21.15.16:0/2227044699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c035070 con 0x7f3d48100310 2024-09-19T09:30:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.584+0000 7f3d4edad640 1 -- 172.21.15.16:0/2227044699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d481a7040 con 0x7f3d48100310 2024-09-19T09:30:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.585+0000 7f3d4edad640 1 -- 172.21.15.16:0/2227044699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d481006f0 con 0x7f3d48100310 2024-09-19T09:30:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.586+0000 7f3d45ffb640 1 -- 172.21.15.16:0/2227044699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3d3c042030 con 0x7f3d48100310 2024-09-19T09:30:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.587+0000 7f3d45ffb640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3d380a3240 0x7f3d380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.587+0000 7f3d47fff640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3d380a3240 0x7f3d380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.587+0000 7f3d45ffb640 1 -- 172.21.15.16:0/2227044699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8670+0+0 (secure 0 0 0) 0x7f3d3c0f02e0 con 0x7f3d48100310 2024-09-19T09:30:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.588+0000 7f3d47fff640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3d380a3240 0x7f3d380a5700 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3d48102d20 tx=0x7f3d3400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.590+0000 7f3d45ffb640 1 -- 172.21.15.16:0/2227044699 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3d3c0bceb0 con 0x7f3d48100310 2024-09-19T09:30:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:27.765+0000 7f3d4edad640 1 -- 172.21.15.16:0/2227044699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3d481027c0 con 0x7f3d48100310 2024-09-19T09:30:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:28.125+0000 7f3d45ffb640 1 -- 172.21.15.16:0/2227044699 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) ==== 181+0+0 (secure 0 0 0) 0x7f3d481027c0 con 0x7f3d48100310 2024-09-19T09:30:28.126 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-19T09:30:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:28.128+0000 7f3d4edad640 1 -- 172.21.15.16:0/2227044699 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3d380a3240 msgr2=0x7f3d380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:28.128+0000 7f3d4edad640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3d380a3240 0x7f3d380a5700 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3d48102d20 tx=0x7f3d3400b390 comp rx=0 tx=0).stop 2024-09-19T09:30:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:28.128+0000 7f3d4edad640 1 -- 172.21.15.16:0/2227044699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d48100310 msgr2=0x7f3d48101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:28.128+0000 7f3d4edad640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d48100310 0x7f3d48101290 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c006010 tx=0x7f3d3c0042f0 comp rx=0 tx=0).stop 2024-09-19T09:30:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:28.129+0000 7f3d4edad640 1 -- 172.21.15.16:0/2227044699 shutdown_connections 2024-09-19T09:30:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:28.129+0000 7f3d4edad640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7f3d380a3240 0x7f3d380a5700 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:28.129+0000 7f3d4edad640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d48106f30 0x7f3d48101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:28.129+0000 7f3d4edad640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d48100cc0 0x7f3d481017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:28.129+0000 7f3d4edad640 1 --2- 172.21.15.16:0/2227044699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d48100310 0x7f3d48101290 secure :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c006010 tx=0x7f3d3c0042f0 comp rx=0 tx=0).stop 2024-09-19T09:30:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:28.129+0000 7f3d4edad640 1 -- 172.21.15.16:0/2227044699 >> 172.21.15.16:0/2227044699 conn(0x7f3d48077210 msgr2=0x7f3d481057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:28.129+0000 7f3d4edad640 1 -- 172.21.15.16:0/2227044699 shutdown_connections 2024-09-19T09:30:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:28.129+0000 7f3d4edad640 1 -- 172.21.15.16:0/2227044699 wait complete. 2024-09-19T09:30:28.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:28 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2227044699' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:30:28.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:28 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2227044699' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:30:28.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:28 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2227044699' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:30:28.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid ee1ab676-7667-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T09:30:29.085 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T09:30:29.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:29 smithi142 ceph-mon[35032]: pgmap v425: 97 pgs: 97 active+clean; 8.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 716 B/s wr, 9 op/s 2024-09-19T09:30:29.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:29 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/2227044699' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:30:29.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:29 smithi142 ceph-mon[35032]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T09:30:29.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:29 smithi097 ceph-mon[34864]: pgmap v425: 97 pgs: 97 active+clean; 8.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 716 B/s wr, 9 op/s 2024-09-19T09:30:29.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:29 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/2227044699' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:30:29.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:29 smithi097 ceph-mon[34864]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T09:30:29.456 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:29 smithi016 ceph-mon[30979]: pgmap v425: 97 pgs: 97 active+clean; 8.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 716 B/s wr, 9 op/s 2024-09-19T09:30:29.456 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:29 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/2227044699' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:30:29.456 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:29 smithi016 ceph-mon[30979]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T09:30:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.540+0000 7fa5d362c640 1 -- 172.21.15.16:0/382656318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5cc106a80 msgr2=0x7fa5cc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.540+0000 7fa5d362c640 1 --2- 172.21.15.16:0/382656318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5cc106a80 0x7fa5cc106e60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009930 tx=0x7fa5c802a420 comp rx=0 tx=0).stop 2024-09-19T09:30:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.540+0000 7fa5d362c640 1 -- 172.21.15.16:0/382656318 shutdown_connections 2024-09-19T09:30:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.540+0000 7fa5d362c640 1 --2- 172.21.15.16:0/382656318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5cc100a80 0x7fa5cc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.540+0000 7fa5d362c640 1 --2- 172.21.15.16:0/382656318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5cc101420 0x7fa5cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.540+0000 7fa5d362c640 1 --2- 172.21.15.16:0/382656318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5cc106a80 0x7fa5cc106e60 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.540+0000 7fa5d362c640 1 -- 172.21.15.16:0/382656318 >> 172.21.15.16:0/382656318 conn(0x7fa5cc0fc7a0 msgr2=0x7fa5cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.540+0000 7fa5d362c640 1 -- 172.21.15.16:0/382656318 shutdown_connections 2024-09-19T09:30:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.540+0000 7fa5d362c640 1 -- 172.21.15.16:0/382656318 wait complete. 2024-09-19T09:30:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.541+0000 7fa5d362c640 1 Processor -- start 2024-09-19T09:30:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.541+0000 7fa5d362c640 1 -- start start 2024-09-19T09:30:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.541+0000 7fa5d362c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5cc100a80 0x7fa5cc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.541+0000 7fa5d362c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5cc101420 0x7fa5cc198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.541+0000 7fa5d362c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5cc106a80 0x7fa5cc19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.541+0000 7fa5d362c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc111fa0 con 0x7fa5cc106a80 2024-09-19T09:30:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.541+0000 7fa5d362c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc111e20 con 0x7fa5cc101420 2024-09-19T09:30:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.541+0000 7fa5d362c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc112120 con 0x7fa5cc100a80 2024-09-19T09:30:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.541+0000 7fa5d1bc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5cc106a80 0x7fa5cc19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.541+0000 7fa5d1bc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5cc106a80 0x7fa5cc19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51118/0 (socket says 172.21.15.16:51118) 2024-09-19T09:30:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.541+0000 7fa5d1bc9640 1 -- 172.21.15.16:0/1876644262 learned_addr learned my addr 172.21.15.16:0/1876644262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-19T09:30:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.541+0000 7fa5d13c8640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5cc100a80 0x7fa5cc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.541+0000 7fa5d0bc7640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5cc101420 0x7fa5cc198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.542+0000 7fa5d1bc9640 1 -- 172.21.15.16:0/1876644262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5cc100a80 msgr2=0x7fa5cc198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.542+0000 7fa5d1bc9640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5cc100a80 0x7fa5cc198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.542+0000 7fa5d1bc9640 1 -- 172.21.15.16:0/1876644262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5cc101420 msgr2=0x7fa5cc198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.542+0000 7fa5d1bc9640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5cc101420 0x7fa5cc198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.542+0000 7fa5d1bc9640 1 -- 172.21.15.16:0/1876644262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc19d4b0 con 0x7fa5cc106a80 2024-09-19T09:30:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.542+0000 7fa5d13c8640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5cc100a80 0x7fa5cc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T09:30:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.542+0000 7fa5d1bc9640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5cc106a80 0x7fa5cc19cdd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400eaa0 tx=0x7fa5c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.542+0000 7fa5c27fc640 1 -- 172.21.15.16:0/1876644262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c400cdb0 con 0x7fa5cc106a80 2024-09-19T09:30:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.542+0000 7fa5d362c640 1 -- 172.21.15.16:0/1876644262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc19d7a0 con 0x7fa5cc106a80 2024-09-19T09:30:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.542+0000 7fa5c27fc640 1 -- 172.21.15.16:0/1876644262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c4005020 con 0x7fa5cc106a80 2024-09-19T09:30:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.542+0000 7fa5c27fc640 1 -- 172.21.15.16:0/1876644262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40116b0 con 0x7fa5cc106a80 2024-09-19T09:30:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.542+0000 7fa5d362c640 1 -- 172.21.15.16:0/1876644262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc1a5080 con 0x7fa5cc106a80 2024-09-19T09:30:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.543+0000 7fa5d362c640 1 -- 172.21.15.16:0/1876644262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5cc100ee0 con 0x7fa5cc106a80 2024-09-19T09:30:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.544+0000 7fa5c27fc640 1 -- 172.21.15.16:0/1876644262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa5c4005560 con 0x7fa5cc106a80 2024-09-19T09:30:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.545+0000 7fa5c27fc640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa5b80a3240 0x7fa5b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T09:30:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.545+0000 7fa5d13c8640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa5b80a3240 0x7fa5b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T09:30:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.545+0000 7fa5c27fc640 1 -- 172.21.15.16:0/1876644262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8116+0+0 (secure 0 0 0) 0x7fa5c40cffe0 con 0x7fa5cc106a80 2024-09-19T09:30:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.546+0000 7fa5d13c8640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa5b80a3240 0x7fa5b80a5700 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8000c00 tx=0x7fa5c8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T09:30:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.548+0000 7fa5c27fc640 1 -- 172.21.15.16:0/1876644262 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa5c4016030 con 0x7fa5cc106a80 2024-09-19T09:30:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:29.726+0000 7fa5d362c640 1 -- 172.21.15.16:0/1876644262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa5cc02d7c0 con 0x7fa5cc106a80 2024-09-19T09:30:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:30.134+0000 7fa5c27fc640 1 -- 172.21.15.16:0/1876644262 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) ==== 169+0+0 (secure 0 0 0) 0x7fa5c409cd20 con 0x7fa5cc106a80 2024-09-19T09:30:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-19T09:30:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:30.138+0000 7fa5d362c640 1 -- 172.21.15.16:0/1876644262 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa5b80a3240 msgr2=0x7fa5b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:30.138+0000 7fa5d362c640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa5b80a3240 0x7fa5b80a5700 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8000c00 tx=0x7fa5c8005a20 comp rx=0 tx=0).stop 2024-09-19T09:30:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:30.138+0000 7fa5d362c640 1 -- 172.21.15.16:0/1876644262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5cc106a80 msgr2=0x7fa5cc19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T09:30:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:30.138+0000 7fa5d362c640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5cc106a80 0x7fa5cc19cdd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400eaa0 tx=0x7fa5c400ef60 comp rx=0 tx=0).stop 2024-09-19T09:30:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:30.138+0000 7fa5d362c640 1 -- 172.21.15.16:0/1876644262 shutdown_connections 2024-09-19T09:30:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:30.138+0000 7fa5d362c640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5cc106a80 0x7fa5cc19cdd0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:30.138+0000 7fa5d362c640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5cc101420 0x7fa5cc198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:30.138+0000 7fa5d362c640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.16:6800/4287557979,v1:172.21.15.16:6801/4287557979] conn(0x7fa5b80a3240 0x7fa5b80a5700 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:30.138+0000 7fa5d362c640 1 --2- 172.21.15.16:0/1876644262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5cc100a80 0x7fa5cc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T09:30:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:30.138+0000 7fa5d362c640 1 -- 172.21.15.16:0/1876644262 >> 172.21.15.16:0/1876644262 conn(0x7fa5cc0fc7a0 msgr2=0x7fa5cc0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T09:30:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:30.138+0000 7fa5d362c640 1 -- 172.21.15.16:0/1876644262 shutdown_connections 2024-09-19T09:30:30.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-19T09:30:30.138+0000 7fa5d362c640 1 -- 172.21.15.16:0/1876644262 wait complete. 2024-09-19T09:30:30.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:30 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1876644262' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:30:30.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:30 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1876644262' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:30:30.628 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:30 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1876644262' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T09:30:30.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:30:30.859 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:30:30.894 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:30:30.928 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T09:30:30.928 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T09:30:30.945 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T09:30:30.963 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T09:30:30.986 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T09:30:30.987 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T09:30:30.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.a 2024-09-19T09:30:31.137 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:31 smithi016 systemd[1]: Stopping Ceph mon.a for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:30:31.379 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:31 smithi142 ceph-mon[35032]: pgmap v427: 65 pgs: 65 active+clean; 577 KiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:31.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:31 smithi142 ceph-mon[35032]: from='client.? 172.21.15.16:0/1876644262' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:30:31.380 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:31 smithi142 ceph-mon[35032]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T09:30:31.386 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:31 smithi097 ceph-mon[34864]: pgmap v427: 65 pgs: 65 active+clean; 577 KiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:31.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:31 smithi097 ceph-mon[34864]: from='client.? 172.21.15.16:0/1876644262' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:30:31.387 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:31 smithi097 ceph-mon[34864]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T09:30:31.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:31 smithi016 ceph-mon[30979]: pgmap v427: 65 pgs: 65 active+clean; 577 KiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T09:30:31.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:31 smithi016 ceph-mon[30979]: from='client.? 172.21.15.16:0/1876644262' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T09:30:31.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:31 smithi016 ceph-mon[30979]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T09:30:31.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:31 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a[30975]: 2024-09-19T09:30:31.316+0000 7f37e70ef640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:31.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:31 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a[30975]: 2024-09-19T09:30:31.316+0000 7f37e70ef640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T09:30:31.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:31 smithi016 podman[91624]: 2024-09-19 09:30:31.36869634 +0000 UTC m=+0.174573032 container died 844da19cfdd1689faec127b82d749a038d28ebc637edc44abbacd02751e9942b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T09:30:32.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:31 smithi016 podman[91624]: 2024-09-19 09:30:31.842112981 +0000 UTC m=+0.647989713 container cleanup 844da19cfdd1689faec127b82d749a038d28ebc637edc44abbacd02751e9942b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:32.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 19 09:30:31 smithi016 bash[91624]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-a 2024-09-19T09:30:32.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T09:30:32.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:32.353 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T09:30:32.353 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T09:30:32.353 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.b 2024-09-19T09:30:32.636 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:32 smithi097 systemd[1]: Stopping Ceph mon.b for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:30:33.136 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:32 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-b[34860]: 2024-09-19T09:30:32.696+0000 7f64c5568640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:33.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:32 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-b[34860]: 2024-09-19T09:30:32.696+0000 7f64c5568640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T09:30:33.137 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:32 smithi097 podman[67492]: 2024-09-19 09:30:32.891624408 +0000 UTC m=+0.340196963 container died 09082255ca7d748361d2857e8bcc374d67fea16f5cd6e072f2869dfea2b43483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T09:30:33.632 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:33 smithi097 podman[67492]: 2024-09-19 09:30:33.405025571 +0000 UTC m=+0.853598079 container cleanup 09082255ca7d748361d2857e8bcc374d67fea16f5cd6e072f2869dfea2b43483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T09:30:33.632 INFO:journalctl@ceph.mon.b.smithi097.stdout:Sep 19 09:30:33 smithi097 bash[67492]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-b 2024-09-19T09:30:33.790 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T09:30:33.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:33.837 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T09:30:33.837 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T09:30:33.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.c 2024-09-19T09:30:34.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:33 smithi142 systemd[1]: Stopping Ceph mon.c for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:30:34.496 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:34 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-c[35028]: 2024-09-19T09:30:34.169+0000 7f119c3c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:34.496 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:34 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-c[35028]: 2024-09-19T09:30:34.170+0000 7f119c3c1640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T09:30:34.496 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:34 smithi142 podman[68162]: 2024-09-19 09:30:34.231742802 +0000 UTC m=+0.194551216 container died e7d1338d9de765e9e91ea5ee65c6d0fc5144aa05f88679bfe1d42c4820156c1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T09:30:34.863 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:34 smithi142 podman[68162]: 2024-09-19 09:30:34.59513898 +0000 UTC m=+0.557947377 container cleanup e7d1338d9de765e9e91ea5ee65c6d0fc5144aa05f88679bfe1d42c4820156c1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:34.863 INFO:journalctl@ceph.mon.c.smithi142.stdout:Sep 19 09:30:34 smithi142 bash[68162]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mon-c 2024-09-19T09:30:35.035 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T09:30:35.084 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:35.084 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T09:30:35.084 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T09:30:35.084 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.x 2024-09-19T09:30:35.390 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:30:35 smithi016 systemd[1]: Stopping Ceph mgr.x for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:30:35.706 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:30:35 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x[31208]: 2024-09-19T09:30:35.388+0000 7f45061f6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:35.707 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:30:35 smithi016 podman[91743]: 2024-09-19 09:30:35.419756481 +0000 UTC m=+0.138240695 container died ef968a83280163064e30438e5e73405fcd1250f9c82c24a4d89c8ff7e681217c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-2f599a4) 2024-09-19T09:30:36.003 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:30:35 smithi016 podman[91743]: 2024-09-19 09:30:35.897766656 +0000 UTC m=+0.616250906 container cleanup ef968a83280163064e30438e5e73405fcd1250f9c82c24a4d89c8ff7e681217c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T09:30:36.003 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:30:35 smithi016 bash[91743]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x 2024-09-19T09:30:36.325 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 19 09:30:36 smithi016 podman[91754]: 2024-09-19 09:30:36.074554356 +0000 UTC m=+0.652841862 container remove ef968a83280163064e30438e5e73405fcd1250f9c82c24a4d89c8ff7e681217c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T09:30:36.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T09:30:36.367 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:36.367 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T09:30:36.368 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T09:30:36.368 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.y 2024-09-19T09:30:36.690 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:30:36 smithi097 systemd[1]: Stopping Ceph mgr.y for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:30:36.969 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:30:36 smithi097 podman[67616]: 2024-09-19 09:30:36.688991618 +0000 UTC m=+0.123932283 container died ec045325526f007246863185e988d6f009c6d770a6132c11d03b3f0dacc08769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T09:30:37.320 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:30:37 smithi097 podman[67616]: 2024-09-19 09:30:37.051408416 +0000 UTC m=+0.486349080 container cleanup ec045325526f007246863185e988d6f009c6d770a6132c11d03b3f0dacc08769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T09:30:37.320 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Sep 19 09:30:37 smithi097 bash[67616]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-y 2024-09-19T09:30:37.445 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T09:30:37.480 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:37.481 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T09:30:37.481 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T09:30:37.481 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.z 2024-09-19T09:30:37.795 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:30:37 smithi142 systemd[1]: Stopping Ceph mgr.z for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:30:38.129 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:30:37 smithi142 podman[68286]: 2024-09-19 09:30:37.795309769 +0000 UTC m=+0.119203455 container died 65283b2503fb1673002141afd2c695bb896badf8d4574630ce0f8ab74c61571e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:38.556 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:30:38 smithi142 podman[68286]: 2024-09-19 09:30:38.149286811 +0000 UTC m=+0.473180541 container cleanup 65283b2503fb1673002141afd2c695bb896badf8d4574630ce0f8ab74c61571e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:38.557 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:30:38 smithi142 bash[68286]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z 2024-09-19T09:30:38.557 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:30:38 smithi142 podman[68300]: 2024-09-19 09:30:38.342144068 +0000 UTC m=+0.542351720 container remove 65283b2503fb1673002141afd2c695bb896badf8d4574630ce0f8ab74c61571e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:38.557 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Sep 19 09:30:38 smithi142 systemd[1]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-19T09:30:38.573 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T09:30:38.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:38.609 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T09:30:38.609 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T09:30:38.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.0 2024-09-19T09:30:38.911 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:38 smithi016 systemd[1]: Stopping Ceph osd.0 for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:30:39.253 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:38 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0[42219]: 2024-09-19T09:30:38.909+0000 7f05e16b4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:39.253 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:38 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0[42219]: 2024-09-19T09:30:38.909+0000 7f05e16b4640 -1 osd.0 84 *** Got signal Terminated *** 2024-09-19T09:30:39.253 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:38 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0[42219]: 2024-09-19T09:30:38.909+0000 7f05e16b4640 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:30:44.572 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:44 smithi016 podman[91862]: 2024-09-19 09:30:44.286447977 +0000 UTC m=+5.478310234 container died 4222acc5111513c8f181d4e0dfa53a75361144c2b1ecb69de6d8336a127e5ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T09:30:45.003 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:44 smithi016 podman[91862]: 2024-09-19 09:30:44.786474201 +0000 UTC m=+5.978336458 container cleanup 4222acc5111513c8f181d4e0dfa53a75361144c2b1ecb69de6d8336a127e5ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:30:45.003 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:44 smithi016 bash[91862]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0 2024-09-19T09:30:45.338 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:45 smithi016 podman[91877]: 2024-09-19 09:30:45.013080156 +0000 UTC m=+0.724756917 container remove 4222acc5111513c8f181d4e0dfa53a75361144c2b1ecb69de6d8336a127e5ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T09:30:45.659 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:45 smithi016 podman[91968]: 2024-09-19 09:30:45.240453368 +0000 UTC m=+0.016578968 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:45.659 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:45 smithi016 podman[91968]: 2024-09-19 09:30:45.397423868 +0000 UTC m=+0.173549448 container create 4e117cd8f6917b4982b8890049c45d01420bd1d1f57983c0c9f58982bc244f1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T09:30:45.660 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:45 smithi016 podman[91968]: 2024-09-19 09:30:45.658831733 +0000 UTC m=+0.434957323 container init 4e117cd8f6917b4982b8890049c45d01420bd1d1f57983c0c9f58982bc244f1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T09:30:46.003 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:45 smithi016 podman[91968]: 2024-09-19 09:30:45.662702751 +0000 UTC m=+0.438828332 container start 4e117cd8f6917b4982b8890049c45d01420bd1d1f57983c0c9f58982bc244f1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:30:46.003 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:45 smithi016 podman[91968]: 2024-09-19 09:30:45.767197515 +0000 UTC m=+0.543323113 container attach 4e117cd8f6917b4982b8890049c45d01420bd1d1f57983c0c9f58982bc244f1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T09:30:46.003 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 19 09:30:45 smithi016 podman[91968]: 2024-09-19 09:30:45.856111521 +0000 UTC m=+0.632237110 container died 4e117cd8f6917b4982b8890049c45d01420bd1d1f57983c0c9f58982bc244f1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T09:30:46.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T09:30:46.590 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:46.591 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T09:30:46.591 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T09:30:46.591 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.1 2024-09-19T09:30:46.886 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:46 smithi097 systemd[1]: Stopping Ceph osd.1 for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:30:47.386 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:46 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1[39353]: 2024-09-19T09:30:46.888+0000 7f7499055640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:47.386 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:46 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1[39353]: 2024-09-19T09:30:46.888+0000 7f7499055640 -1 osd.1 84 *** Got signal Terminated *** 2024-09-19T09:30:47.387 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:46 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1[39353]: 2024-09-19T09:30:46.888+0000 7f7499055640 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:30:52.632 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:52 smithi097 podman[67738]: 2024-09-19 09:30:52.358406651 +0000 UTC m=+5.579500548 container died d0dd91099bbfed7af00be15d7875ab8ac4799e8a222676671ea7cfb20a22be93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:30:52.886 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:52 smithi097 podman[67738]: 2024-09-19 09:30:52.730535982 +0000 UTC m=+5.951629881 container cleanup d0dd91099bbfed7af00be15d7875ab8ac4799e8a222676671ea7cfb20a22be93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T09:30:52.886 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:52 smithi097 bash[67738]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1 2024-09-19T09:30:53.340 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:52 smithi097 podman[67754]: 2024-09-19 09:30:52.989877765 +0000 UTC m=+0.633162309 container remove d0dd91099bbfed7af00be15d7875ab8ac4799e8a222676671ea7cfb20a22be93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T09:30:53.637 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:53 smithi097 podman[67849]: 2024-09-19 09:30:53.242421255 +0000 UTC m=+0.015792557 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:30:53.637 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:53 smithi097 podman[67849]: 2024-09-19 09:30:53.34045409 +0000 UTC m=+0.113825391 container create 549a5598cad7b907dd1c612a37e449dec1412d9c670485d4c7cd69aa2f66a12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:30:53.637 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:53 smithi097 podman[67849]: 2024-09-19 09:30:53.576446132 +0000 UTC m=+0.349817425 container init 549a5598cad7b907dd1c612a37e449dec1412d9c670485d4c7cd69aa2f66a12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-19T09:30:53.637 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:53 smithi097 podman[67849]: 2024-09-19 09:30:53.579974694 +0000 UTC m=+0.353345996 container start 549a5598cad7b907dd1c612a37e449dec1412d9c670485d4c7cd69aa2f66a12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T09:30:54.047 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:53 smithi097 podman[67849]: 2024-09-19 09:30:53.695611381 +0000 UTC m=+0.468982678 container attach 549a5598cad7b907dd1c612a37e449dec1412d9c670485d4c7cd69aa2f66a12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T09:30:54.047 INFO:journalctl@ceph.osd.1.smithi097.stdout:Sep 19 09:30:53 smithi097 podman[67849]: 2024-09-19 09:30:53.764549205 +0000 UTC m=+0.537920506 container died 549a5598cad7b907dd1c612a37e449dec1412d9c670485d4c7cd69aa2f66a12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-19T09:30:54.372 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T09:30:54.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:30:54.412 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T09:30:54.412 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T09:30:54.412 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.2 2024-09-19T09:30:54.712 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:30:54 smithi142 systemd[1]: Stopping Ceph osd.2 for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:30:55.128 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:30:54 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2[40014]: 2024-09-19T09:30:54.710+0000 7feacf108640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:30:55.129 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:30:54 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2[40014]: 2024-09-19T09:30:54.710+0000 7feacf108640 -1 osd.2 84 *** Got signal Terminated *** 2024-09-19T09:30:55.129 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:30:54 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2[40014]: 2024-09-19T09:30:54.710+0000 7feacf108640 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:31:00.413 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:31:00 smithi142 podman[68411]: 2024-09-19 09:31:00.141059064 +0000 UTC m=+5.524459700 container died 88b700d7e60ecc51d0ab7ebfd3761fec828e8c78be124dea10a660aa10a001cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T09:31:00.778 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:31:00 smithi142 podman[68411]: 2024-09-19 09:31:00.485449447 +0000 UTC m=+5.868850075 container cleanup 88b700d7e60ecc51d0ab7ebfd3761fec828e8c78be124dea10a660aa10a001cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T09:31:00.778 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:31:00 smithi142 bash[68411]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2 2024-09-19T09:31:01.116 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:31:00 smithi142 podman[68439]: 2024-09-19 09:31:00.778078251 +0000 UTC m=+0.637140950 container remove 88b700d7e60ecc51d0ab7ebfd3761fec828e8c78be124dea10a660aa10a001cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T09:31:01.374 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:31:01 smithi142 podman[68527]: 2024-09-19 09:31:01.0183851 +0000 UTC m=+0.015633647 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:31:01.374 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:31:01 smithi142 podman[68527]: 2024-09-19 09:31:01.162530839 +0000 UTC m=+0.159779418 container create 4d963c6d60caf96d6055a9deeab05dd725c4c6044633e92998cd0fa93239b90e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T09:31:01.629 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:31:01 smithi142 podman[68527]: 2024-09-19 09:31:01.373301613 +0000 UTC m=+0.370550150 container init 4d963c6d60caf96d6055a9deeab05dd725c4c6044633e92998cd0fa93239b90e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:31:01.629 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:31:01 smithi142 podman[68527]: 2024-09-19 09:31:01.376711941 +0000 UTC m=+0.373960481 container start 4d963c6d60caf96d6055a9deeab05dd725c4c6044633e92998cd0fa93239b90e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T09:31:01.629 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:31:01 smithi142 podman[68527]: 2024-09-19 09:31:01.481461581 +0000 UTC m=+0.478710122 container attach 4d963c6d60caf96d6055a9deeab05dd725c4c6044633e92998cd0fa93239b90e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T09:31:01.629 INFO:journalctl@ceph.osd.2.smithi142.stdout:Sep 19 09:31:01 smithi142 podman[68527]: 2024-09-19 09:31:01.565351776 +0000 UTC m=+0.562600314 container died 4d963c6d60caf96d6055a9deeab05dd725c4c6044633e92998cd0fa93239b90e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T09:31:02.161 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T09:31:02.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:02.201 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T09:31:02.201 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T09:31:02.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.3 2024-09-19T09:31:02.503 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:02 smithi016 systemd[1]: Stopping Ceph osd.3 for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:31:03.003 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:02 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3[49037]: 2024-09-19T09:31:02.543+0000 7f48bcb7e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:03.003 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:02 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3[49037]: 2024-09-19T09:31:02.543+0000 7f48bcb7e640 -1 osd.3 84 *** Got signal Terminated *** 2024-09-19T09:31:03.003 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:02 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3[49037]: 2024-09-19T09:31:02.543+0000 7f48bcb7e640 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:31:05.386 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:05 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:05.294+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:05.503 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:05 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3[49037]: 2024-09-19T09:31:05.093+0000 7f48b517b640 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:38.722289+0000 front 2024-09-19T09:30:38.722395+0000 (oldest deadline 2024-09-19T09:31:04.622521+0000) 2024-09-19T09:31:06.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:06 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:06.052+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:06.503 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:06 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3[49037]: 2024-09-19T09:31:06.103+0000 7f48b517b640 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:38.722289+0000 front 2024-09-19T09:30:38.722395+0000 (oldest deadline 2024-09-19T09:31:04.622521+0000) 2024-09-19T09:31:06.636 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:06 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:06.279+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:07.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:07 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:07.070+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:07.386 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:07 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:07.290+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:07.386 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:07 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:31:07.128+0000 7fc7c3198640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.506951+0000 front 2024-09-19T09:30:42.507061+0000 (oldest deadline 2024-09-19T09:31:07.107399+0000) 2024-09-19T09:31:07.503 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:07 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3[49037]: 2024-09-19T09:31:07.097+0000 7f48b517b640 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:38.722289+0000 front 2024-09-19T09:30:38.722395+0000 (oldest deadline 2024-09-19T09:31:04.622521+0000) 2024-09-19T09:31:08.316 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:08 smithi016 podman[92073]: 2024-09-19 09:31:08.015593514 +0000 UTC m=+5.605727832 container died 2936d667e9216a3bb1db404c7d2c4d2e08174494eb5a289da96ab71aa68c2a55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T09:31:08.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:08 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:08.120+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:08.386 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:08 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:08.306+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:08.386 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:08 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:31:08.079+0000 7fc7c3198640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.506951+0000 front 2024-09-19T09:30:42.507061+0000 (oldest deadline 2024-09-19T09:31:07.107399+0000) 2024-09-19T09:31:08.753 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:08 smithi016 podman[92073]: 2024-09-19 09:31:08.506894763 +0000 UTC m=+6.097029086 container cleanup 2936d667e9216a3bb1db404c7d2c4d2e08174494eb5a289da96ab71aa68c2a55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T09:31:08.753 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:08 smithi016 bash[92073]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3 2024-09-19T09:31:08.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:08 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:08.460+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:09.141 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:08 smithi016 podman[92091]: 2024-09-19 09:31:08.775401485 +0000 UTC m=+0.748660440 container remove 2936d667e9216a3bb1db404c7d2c4d2e08174494eb5a289da96ab71aa68c2a55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T09:31:09.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:09 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:09.125+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:09.386 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:09 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:31:09.083+0000 7fc7c3198640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.506951+0000 front 2024-09-19T09:30:42.507061+0000 (oldest deadline 2024-09-19T09:31:07.107399+0000) 2024-09-19T09:31:09.386 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:09 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:09.322+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:09.462 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:09 smithi016 podman[92177]: 2024-09-19 09:31:09.043863003 +0000 UTC m=+0.016779577 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:31:09.463 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:09 smithi016 podman[92177]: 2024-09-19 09:31:09.142993924 +0000 UTC m=+0.115910483 container create fd2420c0dd2d482b75d7c284de917264210888415b6aad447ddc3d0f7b2198a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T09:31:09.463 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:09 smithi016 podman[92177]: 2024-09-19 09:31:09.362176286 +0000 UTC m=+0.335092847 container init fd2420c0dd2d482b75d7c284de917264210888415b6aad447ddc3d0f7b2198a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T09:31:09.463 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:09 smithi016 podman[92177]: 2024-09-19 09:31:09.36575442 +0000 UTC m=+0.338670979 container start fd2420c0dd2d482b75d7c284de917264210888415b6aad447ddc3d0f7b2198a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T09:31:09.463 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:09 smithi016 podman[92177]: 2024-09-19 09:31:09.462167528 +0000 UTC m=+0.435084097 container attach fd2420c0dd2d482b75d7c284de917264210888415b6aad447ddc3d0f7b2198a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T09:31:09.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:09 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:09.474+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:09.753 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 19 09:31:09 smithi016 podman[92177]: 2024-09-19 09:31:09.557766152 +0000 UTC m=+0.530682721 container died fd2420c0dd2d482b75d7c284de917264210888415b6aad447ddc3d0f7b2198a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T09:31:09.754 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:09 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:09.555+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:10.129 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:09 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:09.665+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:10.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T09:31:10.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:10.260 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T09:31:10.260 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T09:31:10.260 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.4 2024-09-19T09:31:10.339 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:10 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:31:10.069+0000 7fc7c3198640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.506951+0000 front 2024-09-19T09:30:42.507061+0000 (oldest deadline 2024-09-19T09:31:07.107399+0000) 2024-09-19T09:31:10.339 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:10 smithi097 systemd[1]: Stopping Ceph osd.4 for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:31:10.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:10 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:10.133+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:10.636 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:10 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:10.337+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:10.636 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:10 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:31:10.557+0000 7fc7cab9b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:10.636 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:10 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:31:10.557+0000 7fc7cab9b640 -1 osd.4 84 *** Got signal Terminated *** 2024-09-19T09:31:10.636 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:10 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:31:10.557+0000 7fc7cab9b640 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:31:10.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:10 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:10.468+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:10.753 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:10 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:10.584+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:11.110 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:10 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:10.646+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:11.370 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:11 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:31:11.073+0000 7fc7c3198640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.506951+0000 front 2024-09-19T09:30:42.507061+0000 (oldest deadline 2024-09-19T09:31:07.107399+0000) 2024-09-19T09:31:11.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:11 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:11.109+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:11.636 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:11 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:11.368+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:11.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:11 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:11.469+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:11.754 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:11 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:11.623+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:11.878 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:11 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:11.615+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:12.377 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:12 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:31:12.051+0000 7fc7c3198640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.506951+0000 front 2024-09-19T09:30:42.507061+0000 (oldest deadline 2024-09-19T09:31:07.107399+0000) 2024-09-19T09:31:12.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:12 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:12.093+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:12.636 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:12 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:12.376+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:13.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:12 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:12.636+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:13.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:12 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:12.507+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:13.060 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:12 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:12.633+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:13.356 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:13 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:31:13.011+0000 7fc7c3198640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.506951+0000 front 2024-09-19T09:30:42.507061+0000 (oldest deadline 2024-09-19T09:31:07.107399+0000) 2024-09-19T09:31:13.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:13 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:13.060+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:13.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:13 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:13.402+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:13.636 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:13 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:13.355+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:13.878 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:13 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:13.588+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:14.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:13 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:13.593+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:14.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:13 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:13.511+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:14.359 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:14 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:31:14.001+0000 7fc7c3198640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.506951+0000 front 2024-09-19T09:30:42.507061+0000 (oldest deadline 2024-09-19T09:31:07.107399+0000) 2024-09-19T09:31:14.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:14 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:14.023+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:14.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:14 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:14.379+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:14.636 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:14 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:14.357+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:14.636 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:14 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:14.358+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:14.753 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:14 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:14.603+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:14.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:14 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:14.495+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:14.878 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:14 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:14.595+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:15.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:15 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:15.025+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:15.386 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:15 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4[45860]: 2024-09-19T09:31:14.999+0000 7fc7c3198640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.506951+0000 front 2024-09-19T09:30:42.507061+0000 (oldest deadline 2024-09-19T09:31:07.107399+0000) 2024-09-19T09:31:15.879 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:15 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:15.597+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:15.879 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:15 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:15.597+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:49.320121+0000 front 2024-09-19T09:30:49.320107+0000 (oldest deadline 2024-09-19T09:31:14.619989+0000) 2024-09-19T09:31:15.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:15 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:15.415+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:15.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:15 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:15.398+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:15.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:15 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:15.399+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:16.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:15 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:15.580+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:16.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:15 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:15.531+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:16.276 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:16 smithi097 podman[67964]: 2024-09-19 09:31:16.007755791 +0000 UTC m=+5.550632409 container died 55aa73db119e29114c2b8cc8c07858ee5f53044c859904f54865b57e56c85d0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:31:16.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:16 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:16.001+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:16.617 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:16 smithi097 podman[67964]: 2024-09-19 09:31:16.365911137 +0000 UTC m=+5.908787745 container cleanup 55aa73db119e29114c2b8cc8c07858ee5f53044c859904f54865b57e56c85d0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-19T09:31:16.618 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:16 smithi097 bash[67964]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4 2024-09-19T09:31:16.618 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:16 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:16.411+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:16.618 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:16 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:16.440+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:16.618 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:16 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:16.440+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:16.879 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:16 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:16.571+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:16.879 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:16 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:16.571+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:49.320121+0000 front 2024-09-19T09:30:49.320107+0000 (oldest deadline 2024-09-19T09:31:14.619989+0000) 2024-09-19T09:31:16.888 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:16 smithi097 podman[67980]: 2024-09-19 09:31:16.616808929 +0000 UTC m=+0.602250932 container remove 55aa73db119e29114c2b8cc8c07858ee5f53044c859904f54865b57e56c85d0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T09:31:17.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:16 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:16.624+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:17.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:16 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:16.535+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:17.296 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:16 smithi097 podman[68070]: 2024-09-19 09:31:16.881156424 +0000 UTC m=+0.015663333 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:31:17.296 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:16 smithi097 podman[68070]: 2024-09-19 09:31:16.984193832 +0000 UTC m=+0.118700728 container create c0c1619f4f76d345bfb015415931b546fe1f71b70ad06b8a488bff8cfb3945d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:31:17.296 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:17 smithi097 podman[68070]: 2024-09-19 09:31:17.220175759 +0000 UTC m=+0.354682654 container init c0c1619f4f76d345bfb015415931b546fe1f71b70ad06b8a488bff8cfb3945d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T09:31:17.296 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:17 smithi097 podman[68070]: 2024-09-19 09:31:17.223721855 +0000 UTC m=+0.358228753 container start c0c1619f4f76d345bfb015415931b546fe1f71b70ad06b8a488bff8cfb3945d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T09:31:17.296 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:17 smithi097 podman[68070]: 2024-09-19 09:31:17.294978508 +0000 UTC m=+0.429485404 container attach c0c1619f4f76d345bfb015415931b546fe1f71b70ad06b8a488bff8cfb3945d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T09:31:17.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:16 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:16.985+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:17.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:16 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:16.985+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:17.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:17 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:17.385+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:17.636 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 19 09:31:17 smithi097 podman[68070]: 2024-09-19 09:31:17.41049959 +0000 UTC m=+0.545006496 container died c0c1619f4f76d345bfb015415931b546fe1f71b70ad06b8a488bff8cfb3945d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T09:31:17.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:17 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:17.426+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:17.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:17 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:17.426+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:17.879 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:17 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:17.602+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:17.879 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:17 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:17.535+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:17.879 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:17 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:17.535+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:49.320121+0000 front 2024-09-19T09:30:49.320107+0000 (oldest deadline 2024-09-19T09:31:14.619989+0000) 2024-09-19T09:31:17.998 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T09:31:18.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:17 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:17.665+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:18.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:17 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:17.665+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:18.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:17 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:17.547+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:18.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:17 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:17.547+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:18.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:18.039 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T09:31:18.039 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T09:31:18.039 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.5 2024-09-19T09:31:18.354 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:18 smithi142 systemd[1]: Stopping Ceph osd.5 for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:31:18.354 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:17 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:17.977+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:18.354 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:17 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:17.977+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:18.611 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:18 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:18.353+0000 7f5b436cd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:18.611 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:18 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:18.353+0000 7f5b436cd640 -1 osd.5 84 *** Got signal Terminated *** 2024-09-19T09:31:18.611 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:18 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:18.353+0000 7f5b436cd640 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:31:18.611 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:18 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:18.506+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:18.611 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:18 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:18.506+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:49.320121+0000 front 2024-09-19T09:30:49.320107+0000 (oldest deadline 2024-09-19T09:31:14.619989+0000) 2024-09-19T09:31:18.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:18 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:18.384+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:18.636 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:18 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:18.415+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:18.636 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:18 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:18.415+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:18.879 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:18 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:18.610+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:19.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:18 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:18.711+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:19.004 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:18 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:18.711+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:19.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:18 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:18.549+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:19.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:18 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:18.549+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:19.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:18 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:18.935+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:19.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:18 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:18.935+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:19.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:19 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:19.432+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:19.636 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:19 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:19.370+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:19.636 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:19 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:19.370+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:19.879 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:19 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:19.641+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:19.879 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:19 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:19.548+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:19.879 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:19 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:19.548+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:49.320121+0000 front 2024-09-19T09:30:49.320107+0000 (oldest deadline 2024-09-19T09:31:14.619989+0000) 2024-09-19T09:31:20.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:19 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:19.736+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:20.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:19 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:19.736+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:20.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:19 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:19.530+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:20.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:19 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:19.530+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:20.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:19 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:19.977+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:20.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:19 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:19.977+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:20.878 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:20 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:20.511+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:20.879 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:20 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:20.511+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:49.320121+0000 front 2024-09-19T09:30:49.320107+0000 (oldest deadline 2024-09-19T09:31:14.619989+0000) 2024-09-19T09:31:20.879 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:20 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:20.628+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:20.879 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:20 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:20.628+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:20.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:20 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:20.427+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:20.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:20 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:20.388+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:20.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:20 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:20.388+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:21.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:20.772+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:21.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:20.772+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:21.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:20.551+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:21.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:20 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:20.551+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:21.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:20 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:20.967+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:21.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:20 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:20.967+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:21.879 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:21 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:21.597+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:21.879 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:21 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:21.597+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:21.879 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:21 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:21.492+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:21.879 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:21 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:21.492+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:49.320121+0000 front 2024-09-19T09:30:49.320107+0000 (oldest deadline 2024-09-19T09:31:14.619989+0000) 2024-09-19T09:31:21.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:21 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:21.429+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:21.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:21 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:21.436+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:21.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:21 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:21.436+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:21.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:21 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:21.436+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:22.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:21 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:21.739+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:22.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:21 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:21.739+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:22.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:21 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:21.563+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:22.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:21 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:21.563+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:22.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:21 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:21.563+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:22.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:21 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:21.967+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:22.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:21 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:21.967+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:22.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:21 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:21.967+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:22.878 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:22 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:22.622+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:22.879 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:22 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:22.622+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:22.879 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:22 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:22.478+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:42.919158+0000 front 2024-09-19T09:30:42.919231+0000 (oldest deadline 2024-09-19T09:31:09.319606+0000) 2024-09-19T09:31:22.879 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:22 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5[46700]: 2024-09-19T09:31:22.478+0000 7f5b3fcd2640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:49.320121+0000 front 2024-09-19T09:30:49.320107+0000 (oldest deadline 2024-09-19T09:31:14.619989+0000) 2024-09-19T09:31:22.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:22 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:22.432+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:22.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:22 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:22.444+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:22.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:22 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:22.444+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:22.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:22 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:22.444+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:22.904 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:22.775+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:22.904 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:22.775+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:22.905 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:22.775+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:59.065562+0000 front 2024-09-19T09:30:59.065339+0000 (oldest deadline 2024-09-19T09:31:21.965312+0000) 2024-09-19T09:31:22.905 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:22.542+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:22.905 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:22.542+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:22.905 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:22 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:22.542+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:23.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:22 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:22.967+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:23.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:22 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:22.967+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:23.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:22 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:22.967+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:23.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:23 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:23.421+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:23.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:23 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:23.421+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:23.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:23 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:23.469+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:23.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:23 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:23.469+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:23.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:23 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:23.469+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:23.916 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:23 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:23.733+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:23.916 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:23 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:23.733+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:23.916 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:23 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:23.733+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:59.065562+0000 front 2024-09-19T09:30:59.065339+0000 (oldest deadline 2024-09-19T09:31:21.965312+0000) 2024-09-19T09:31:23.917 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:23 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:23.590+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:23.917 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:23 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:23.590+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:23.917 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:23 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:23.590+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:23.984 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:23 smithi142 podman[68631]: 2024-09-19 09:31:23.817960868 +0000 UTC m=+5.586795366 container died 0dbab7611710564b91c961e2782f54503b464dc2dd89c92c51c0ee518dc827c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:31:23.984 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:23 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:23.633+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:23.984 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:23 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:23.633+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:24.379 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:24 smithi142 podman[68631]: 2024-09-19 09:31:24.177585401 +0000 UTC m=+5.946419892 container cleanup 0dbab7611710564b91c961e2782f54503b464dc2dd89c92c51c0ee518dc827c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T09:31:24.379 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:24 smithi142 bash[68631]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5 2024-09-19T09:31:24.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:23 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:23.981+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:24.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:23 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:23.981+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:24.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:23 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:23.981+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:24.750 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:24 smithi142 podman[68646]: 2024-09-19 09:31:24.412089871 +0000 UTC m=+0.592030118 container remove 0dbab7611710564b91c961e2782f54503b464dc2dd89c92c51c0ee518dc827c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T09:31:24.751 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:24.639+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:24.751 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:24 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:24.639+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:24.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:24.428+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:24.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:24.428+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:24.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:24.518+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:24.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:24.518+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:24.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:24 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:24.518+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:25.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:24 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:24.731+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:25.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:24 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:24.731+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:25.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:24 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:24.731+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:59.065562+0000 front 2024-09-19T09:30:59.065339+0000 (oldest deadline 2024-09-19T09:31:21.965312+0000) 2024-09-19T09:31:25.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:24 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:24.545+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:25.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:24 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:24.545+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:25.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:24 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:24.545+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:25.024 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:24 smithi142 podman[68731]: 2024-09-19 09:31:24.652706085 +0000 UTC m=+0.015693690 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:31:25.024 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:24 smithi142 podman[68731]: 2024-09-19 09:31:24.7983723 +0000 UTC m=+0.161359888 container create 461d9de7284402040c30a6d05e26151e3ddeb70abceb625957e79ec2b8b6368d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T09:31:25.024 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:25 smithi142 podman[68731]: 2024-09-19 09:31:25.023521251 +0000 UTC m=+0.386508845 container init 461d9de7284402040c30a6d05e26151e3ddeb70abceb625957e79ec2b8b6368d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:31:25.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:25 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:25.026+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:25.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:25 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:25.026+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:25.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:25 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:25.026+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:25.379 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:25 smithi142 podman[68731]: 2024-09-19 09:31:25.027052245 +0000 UTC m=+0.390039838 container start 461d9de7284402040c30a6d05e26151e3ddeb70abceb625957e79ec2b8b6368d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T09:31:25.380 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:25 smithi142 podman[68731]: 2024-09-19 09:31:25.12357244 +0000 UTC m=+0.486560050 container attach 461d9de7284402040c30a6d05e26151e3ddeb70abceb625957e79ec2b8b6368d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:31:25.380 INFO:journalctl@ceph.osd.5.smithi142.stdout:Sep 19 09:31:25 smithi142 podman[68731]: 2024-09-19 09:31:25.216142479 +0000 UTC m=+0.579130068 container died 461d9de7284402040c30a6d05e26151e3ddeb70abceb625957e79ec2b8b6368d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-19T09:31:25.787 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:25 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:25.404+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:25.788 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:25 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:25.404+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:25.788 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:25 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:25.536+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:25.788 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:25 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:25.536+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:25.788 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:25 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:25.536+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:25.810 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:25 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:25.614+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:25.810 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:25 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:25.614+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:25.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T09:31:25.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:25.866 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T09:31:25.866 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T09:31:25.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.6 2024-09-19T09:31:25.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:25 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:25.719+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:25.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:25 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:25.719+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:25.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:25 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:25.719+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:59.065562+0000 front 2024-09-19T09:30:59.065339+0000 (oldest deadline 2024-09-19T09:31:21.965312+0000) 2024-09-19T09:31:25.895 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:25 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:25.564+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:25.895 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:25 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:25.564+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:25.895 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:25 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:25.564+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:26.178 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:25 smithi016 systemd[1]: Stopping Ceph osd.6 for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:31:26.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:26 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:26.067+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:26.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:26 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:26.067+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:26.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:26 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:26.067+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:26.503 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:26 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:26.177+0000 7f7152fdc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:26.503 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:26 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:26.177+0000 7f7152fdc640 -1 osd.6 84 *** Got signal Terminated *** 2024-09-19T09:31:26.503 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:26 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:26.177+0000 7f7152fdc640 -1 osd.6 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:31:26.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:26 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:26.394+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:26.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:26 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:26.394+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:26.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:26 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:26.584+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:26.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:26 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:26.584+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:26.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:26 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:26.584+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:26.921 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:26 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:26.642+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:26.922 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:26 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:26.642+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:27.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:26 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:26.535+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:27.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:26 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:26.535+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:27.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:26 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:26.535+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:27.004 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:26 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:26.758+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:27.004 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:26 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:26.758+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:27.004 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:26 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:26.758+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:59.065562+0000 front 2024-09-19T09:30:59.065339+0000 (oldest deadline 2024-09-19T09:31:21.965312+0000) 2024-09-19T09:31:27.176 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:27 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:27.090+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:27.177 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:27 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:27.090+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:27.177 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:27 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:27.090+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:27.633 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:27 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:27.360+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:27.634 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:27 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:27.360+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:27.753 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:27 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:27.711+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:27.753 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:27 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:27.712+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:27.753 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:27 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:27.712+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:59.065562+0000 front 2024-09-19T09:30:59.065339+0000 (oldest deadline 2024-09-19T09:31:21.965312+0000) 2024-09-19T09:31:27.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:27 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:27.489+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:27.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:27 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:27.489+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:27.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:27 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:27.489+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:27.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:27 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:27.632+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:27.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:27 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:27.632+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:27.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:27 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:27.632+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:28.085 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:27 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:27.684+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:28.085 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:27 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:27.684+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:28.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:28 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:28.083+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:28.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:28 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:28.083+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:28.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:28 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:28.083+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:28.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:28 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:28.373+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:28.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:28 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:28.373+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:28.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:28 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:28.611+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:28.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:28 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:28.611+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:28.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:28 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:28.611+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:29.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:28 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:28.664+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:29.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:28 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:28.664+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:29.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:28 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:28.664+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:59.065562+0000 front 2024-09-19T09:30:59.065339+0000 (oldest deadline 2024-09-19T09:31:21.965312+0000) 2024-09-19T09:31:29.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:28 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:28.536+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:29.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:28 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:28.536+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:29.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:28 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:28.536+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:29.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:28 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:28.536+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:29.092 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:28 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:28.659+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:29.092 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:28 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:28.659+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:29.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:29 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:29.089+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:29.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:29 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:29.089+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:29.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:29 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:29.089+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:29.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:29 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:29.089+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:29.878 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:29 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:29.618+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:29.879 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:29 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:29.618+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:29.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:29 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:29.408+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:29.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:29 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:29.408+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:29.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:29 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:29.592+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:29.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:29 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:29.592+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:29.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:29 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:29.592+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:29.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:29 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:29.592+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:30.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:29 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:29.674+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:30.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:29 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:29.674+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:30.003 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:29 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:29.674+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:59.065562+0000 front 2024-09-19T09:30:59.065339+0000 (oldest deadline 2024-09-19T09:31:21.965312+0000) 2024-09-19T09:31:30.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:29 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:29.550+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:30.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:29 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:29.550+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:30.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:29 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:29.550+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:30.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:29 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:29.550+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:30.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:30 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:30.112+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:30.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:30 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:30.112+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:30.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:30 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:30.112+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:30.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:30 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:30.112+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:30.753 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:30 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:30.651+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.263484+0000 front 2024-09-19T09:30:43.263551+0000 (oldest deadline 2024-09-19T09:31:08.563624+0000) 2024-09-19T09:31:30.753 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:30 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:30.651+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.464885+0000 front 2024-09-19T09:30:51.464702+0000 (oldest deadline 2024-09-19T09:31:17.364622+0000) 2024-09-19T09:31:30.753 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:30 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6[55551]: 2024-09-19T09:31:30.651+0000 7f714b5d9640 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:59.065562+0000 front 2024-09-19T09:30:59.065339+0000 (oldest deadline 2024-09-19T09:31:21.965312+0000) 2024-09-19T09:31:30.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:30 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:30.501+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:30.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:30 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:30.501+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:30.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:30 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:30.501+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:30.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:30 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:30.501+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:30.878 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:30 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:30.625+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:30.879 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:30 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:30.625+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:30.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:30 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:30.387+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:30.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:30 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:30.387+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:30.887 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:30 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:30.387+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.562842+0000 front 2024-09-19T09:31:06.562942+0000 (oldest deadline 2024-09-19T09:31:29.462633+0000) 2024-09-19T09:31:30.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:30 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:30.599+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:30.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:30 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:30.599+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:30.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:30 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:30.599+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:30.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:30 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:30.599+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:31.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:31 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:31.123+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:31.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:31 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:31.123+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:31.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:31 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:31.123+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:31.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:31 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:31.123+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:31.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:31 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:31.375+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:31.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:31 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:31.375+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:31.637 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:31 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:31.375+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.562842+0000 front 2024-09-19T09:31:06.562942+0000 (oldest deadline 2024-09-19T09:31:29.462633+0000) 2024-09-19T09:31:31.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:31 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:31.556+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:31.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:31 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:31.556+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:31.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:31 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:31.556+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:31.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:31 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:31.556+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:31.753 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:31 smithi016 podman[92290]: 2024-09-19 09:31:31.632843071 +0000 UTC m=+5.566901303 container died 39740579cdc9f99bafbac4619de70b20bb38b7328807c6627b4b31b0dc1ced39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T09:31:31.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:31 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:31.480+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:31.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:31 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:31.480+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:31.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:31 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:31.480+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:31.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:31 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:31.480+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:32.108 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:31 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:31.675+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:32.108 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:31 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:31.675+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:32.253 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:32 smithi016 podman[92290]: 2024-09-19 09:31:32.043928075 +0000 UTC m=+5.977986306 container cleanup 39740579cdc9f99bafbac4619de70b20bb38b7328807c6627b4b31b0dc1ced39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T09:31:32.253 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:32 smithi016 bash[92290]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6 2024-09-19T09:31:32.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:32 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:32.106+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:32.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:32 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:32.106+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:32.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:32 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:32.106+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:32.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:32 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:32.106+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:32.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:32 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:32.372+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:32.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:32 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:32.372+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:32.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:32 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:32.372+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.562842+0000 front 2024-09-19T09:31:06.562942+0000 (oldest deadline 2024-09-19T09:31:29.462633+0000) 2024-09-19T09:31:32.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:32 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:32.532+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:32.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:32 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:32.532+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:32.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:32 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:32.532+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:32.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:32 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:32.532+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:32.753 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:32 smithi016 podman[92306]: 2024-09-19 09:31:32.362399294 +0000 UTC m=+0.723685027 container remove 39740579cdc9f99bafbac4619de70b20bb38b7328807c6627b4b31b0dc1ced39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T09:31:32.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:32 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:32.506+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:32.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:32 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:32.506+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:32.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:32 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:32.506+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:32.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:32 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:32.506+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:33.094 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:32 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:32.692+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:33.094 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:32 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:32.692+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:33.094 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:32 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:32.692+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:33.100 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:32 smithi016 podman[92397]: 2024-09-19 09:31:32.730419108 +0000 UTC m=+0.016700834 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:31:33.100 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:32 smithi016 podman[92397]: 2024-09-19 09:31:32.855323328 +0000 UTC m=+0.141605038 container create c405d4344056c7db2096d31b47af90c09dda097ba21eea1c2ad1235fcfd2deae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:31:33.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:33 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:33.092+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:33.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:33 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:33.092+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:33.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:33 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:33.092+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:33.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:33 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:33.092+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:33.503 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:33 smithi016 podman[92397]: 2024-09-19 09:31:33.099741105 +0000 UTC m=+0.386022825 container init c405d4344056c7db2096d31b47af90c09dda097ba21eea1c2ad1235fcfd2deae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T09:31:33.504 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:33 smithi016 podman[92397]: 2024-09-19 09:31:33.103662923 +0000 UTC m=+0.389944635 container start c405d4344056c7db2096d31b47af90c09dda097ba21eea1c2ad1235fcfd2deae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T09:31:33.504 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:33 smithi016 podman[92397]: 2024-09-19 09:31:33.19126077 +0000 UTC m=+0.477542486 container attach c405d4344056c7db2096d31b47af90c09dda097ba21eea1c2ad1235fcfd2deae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4) 2024-09-19T09:31:33.504 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:33 smithi016 podman[92397]: 2024-09-19 09:31:33.305154403 +0000 UTC m=+0.591436113 container died c405d4344056c7db2096d31b47af90c09dda097ba21eea1c2ad1235fcfd2deae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:31:33.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:33 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:33.383+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:33.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:33 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:33.383+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:33.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:33 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:33.383+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.562842+0000 front 2024-09-19T09:31:06.562942+0000 (oldest deadline 2024-09-19T09:31:29.462633+0000) 2024-09-19T09:31:33.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:33 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:33.576+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:33.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:33 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:33.576+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:33.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:33 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:33.576+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:33.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:33 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:33.576+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:33.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:33 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:33.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:33.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:33 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:33.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:33.858 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:33 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:33.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:33.859 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:33 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:33.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:34.098 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:33 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:33.651+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:34.098 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:33 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:33.651+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:34.098 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:33 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:33.651+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:34.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T09:31:34.167 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:34 smithi016 podman[92420]: 2024-09-19 09:31:34.078855766 +0000 UTC m=+0.768067588 container remove c405d4344056c7db2096d31b47af90c09dda097ba21eea1c2ad1235fcfd2deae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:31:34.167 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:34 smithi016 systemd[1]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.6.service: Deactivated successfully. 2024-09-19T09:31:34.167 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:34 smithi016 systemd[1]: Stopped Ceph osd.6 for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:31:34.167 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 19 09:31:34 smithi016 systemd[1]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.6.service: Consumed 8.084s CPU time. 2024-09-19T09:31:34.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:34.184 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T09:31:34.184 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T09:31:34.184 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.7 2024-09-19T09:31:34.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:34 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:34.095+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:34.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:34 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:34.095+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:34.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:34 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:34.095+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:34.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:34 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:34.095+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:34.485 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:34 smithi097 systemd[1]: Stopping Ceph osd.7 for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:31:34.485 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:34 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:34.380+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:34.485 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:34 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:34.380+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:34.485 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:34 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:34.380+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.562842+0000 front 2024-09-19T09:31:06.562942+0000 (oldest deadline 2024-09-19T09:31:29.462633+0000) 2024-09-19T09:31:34.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:34 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:34.483+0000 7fd5b042c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:34.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:34 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:34.483+0000 7fd5b042c640 -1 osd.7 84 *** Got signal Terminated *** 2024-09-19T09:31:34.886 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:34 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:34.483+0000 7fd5b042c640 -1 osd.7 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:31:34.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:34 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:34.589+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:34.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:34 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:34.589+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:34.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:34 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:34.589+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:34.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:34 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:34.589+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:35.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:34 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:34.503+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:35.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:34 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:34.503+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:35.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:34 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:34.503+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:35.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:34 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:34.503+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:35.053 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:34 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:34.638+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:35.053 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:34 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:34.638+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:35.053 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:34 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:34.638+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:35.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:35 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:35.051+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:35.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:35 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:35.051+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:35.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:35 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:35.051+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:35.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:35 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:35.051+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:35.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:35 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:35.377+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:35.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:35 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:35.377+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:35.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:35 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:35.377+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.562842+0000 front 2024-09-19T09:31:06.562942+0000 (oldest deadline 2024-09-19T09:31:29.462633+0000) 2024-09-19T09:31:35.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:35 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:35.601+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:35.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:35 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:35.601+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:35.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:35 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:35.601+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:35.637 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:35 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:35.601+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:35.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:35 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:35.457+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:35.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:35 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:35.457+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:35.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:35 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:35.457+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:35.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:35 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:35.457+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:36.082 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:35 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:35.665+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:36.082 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:35 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:35.665+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:36.082 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:35 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:35.665+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:36.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:36 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:36.080+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:36.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:36 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:36.080+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:36.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:36 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:36.080+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:36.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:36 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:36.080+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:36.584 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:36 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:36.331+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:36.584 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:36 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:36.331+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:36.584 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:36 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:36.331+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.562842+0000 front 2024-09-19T09:31:06.562942+0000 (oldest deadline 2024-09-19T09:31:29.462633+0000) 2024-09-19T09:31:36.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:36.460+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:36.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:36.460+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:36.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:36.460+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:36.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:36.460+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:36.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:36 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:36.460+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:36.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:36 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:36.583+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:36.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:36 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:36.583+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:36.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:36 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:36.583+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:36.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:36 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:36.583+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:37.114 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:36 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:36.713+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:37.114 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:36 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:36.713+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:37.115 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:36 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:36.713+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:37.378 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:37 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:37.113+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:37.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:37 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:37.113+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:37.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:37 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:37.113+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:37.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:37 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:37.113+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:37.386 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:37 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:37.289+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:37.386 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:37 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:37.289+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:37.386 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:37 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:37.289+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.562842+0000 front 2024-09-19T09:31:06.562942+0000 (oldest deadline 2024-09-19T09:31:29.462633+0000) 2024-09-19T09:31:37.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:37 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:37.475+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:37.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:37 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:37.475+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:37.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:37 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:37.475+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:37.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:37 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:37.475+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:37.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:37 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:37.475+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:37.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:37 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:37.605+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:37.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:37 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:37.605+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:37.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:37 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:37.605+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:37.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:37 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:37.605+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:38.118 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:37 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:37.731+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:38.118 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:37 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:37.731+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:38.118 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:37 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:37.731+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:38.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:38 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:38.116+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:38.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:38 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:38.116+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:38.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:38 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:38.116+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:38.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:38 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:38.116+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:38.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:38 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:38.116+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:38.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:38 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:38.275+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:38.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:38 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:38.275+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:38.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:38 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:38.275+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.562842+0000 front 2024-09-19T09:31:06.562942+0000 (oldest deadline 2024-09-19T09:31:29.462633+0000) 2024-09-19T09:31:38.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:38 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:38.275+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:14.064426+0000 front 2024-09-19T09:31:14.064361+0000 (oldest deadline 2024-09-19T09:31:38.065201+0000) 2024-09-19T09:31:38.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:38 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:38.457+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:38.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:38 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:38.457+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:38.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:38 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:38.457+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:38.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:38 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:38.457+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:38.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:38 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:38.457+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:39.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:38 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:38.744+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:39.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:38 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:38.744+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:39.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:38 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:38.744+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:39.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:38 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:38.744+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:12.499176+0000 front 2024-09-19T09:31:12.499278+0000 (oldest deadline 2024-09-19T09:31:38.299520+0000) 2024-09-19T09:31:39.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:38 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:38.647+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:39.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:38 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:38.647+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:39.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:38 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:38.647+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:39.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:38 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:38.647+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:39.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:39 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:39.166+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:39.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:39 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:39.166+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:39.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:39 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:39.166+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:39.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:39 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:39.166+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:39.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:39 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:39.166+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:39.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:39 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:39.303+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:47.959496+0000 front 2024-09-19T09:30:47.959568+0000 (oldest deadline 2024-09-19T09:31:12.560322+0000) 2024-09-19T09:31:39.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:39 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:39.303+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.861440+0000 front 2024-09-19T09:30:57.861758+0000 (oldest deadline 2024-09-19T09:31:22.561439+0000) 2024-09-19T09:31:39.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:39 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:39.303+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.562842+0000 front 2024-09-19T09:31:06.562942+0000 (oldest deadline 2024-09-19T09:31:29.462633+0000) 2024-09-19T09:31:39.636 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:39 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7[52551]: 2024-09-19T09:31:39.303+0000 7fd5aca31640 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:14.064426+0000 front 2024-09-19T09:31:14.064361+0000 (oldest deadline 2024-09-19T09:31:38.065201+0000) 2024-09-19T09:31:39.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:39 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:39.475+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:39.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:39 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:39.475+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:39.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:39 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:39.475+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:39.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:39 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:39.475+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:39.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:39 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:39.475+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:39.929 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:39 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:39.663+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:39.929 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:39 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:39.663+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:39.929 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:39 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:39.663+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:39.929 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:39 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:39.663+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:39.929 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:39 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:39.663+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:40.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:39 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:39.713+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:40.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:39 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:39.713+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:40.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:39 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:39.713+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:40.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:39 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:39.713+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:12.499176+0000 front 2024-09-19T09:31:12.499278+0000 (oldest deadline 2024-09-19T09:31:38.299520+0000) 2024-09-19T09:31:40.210 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:39 smithi097 podman[68185]: 2024-09-19 09:31:39.927939174 +0000 UTC m=+5.550148720 container died 76b5d73c6e8213be13ff59c3e130b00ece65dc35b399820ee1368ec0a3799492 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T09:31:40.553 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:40 smithi097 podman[68185]: 2024-09-19 09:31:40.30908317 +0000 UTC m=+5.931292707 container cleanup 76b5d73c6e8213be13ff59c3e130b00ece65dc35b399820ee1368ec0a3799492 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-19T09:31:40.553 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:40 smithi097 bash[68185]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7 2024-09-19T09:31:40.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:40.170+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:40.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:40.170+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:40.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:40.170+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:40.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:40.170+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:40.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:40.170+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:40.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:40 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:40.497+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:40.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:40 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:40.497+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:40.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:40 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:40.497+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:40.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:40 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:40.497+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:40.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:40 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:40.497+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:40.870 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:40 smithi097 podman[68201]: 2024-09-19 09:31:40.551910019 +0000 UTC m=+0.626697072 container remove 76b5d73c6e8213be13ff59c3e130b00ece65dc35b399820ee1368ec0a3799492 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-19T09:31:40.870 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:40 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:40.697+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:40.870 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:40 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:40.697+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:40.871 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:40 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:40.697+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:40.871 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:40 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:40.697+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:40.871 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:40 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:40.697+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:41.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:40.759+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:41.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:40.759+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:41.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:40.759+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:41.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:40 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:40.759+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:12.499176+0000 front 2024-09-19T09:31:12.499278+0000 (oldest deadline 2024-09-19T09:31:38.299520+0000) 2024-09-19T09:31:41.136 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:40 smithi097 podman[68286]: 2024-09-19 09:31:40.869207671 +0000 UTC m=+0.110189615 container create 63c434a8db9da0535fa0d03d3cc4c6b998f6c302c681f25f07663d0ed30e4a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:31:41.136 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:40 smithi097 podman[68286]: 2024-09-19 09:31:40.774751883 +0000 UTC m=+0.015733798 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:31:41.136 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:41 smithi097 podman[68286]: 2024-09-19 09:31:41.104880835 +0000 UTC m=+0.345862742 container init 63c434a8db9da0535fa0d03d3cc4c6b998f6c302c681f25f07663d0ed30e4a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T09:31:41.136 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:41 smithi097 podman[68286]: 2024-09-19 09:31:41.108482464 +0000 UTC m=+0.349464363 container start 63c434a8db9da0535fa0d03d3cc4c6b998f6c302c681f25f07663d0ed30e4a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-19T09:31:41.600 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:41 smithi097 podman[68286]: 2024-09-19 09:31:41.188186756 +0000 UTC m=+0.429168667 container attach 63c434a8db9da0535fa0d03d3cc4c6b998f6c302c681f25f07663d0ed30e4a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T09:31:41.600 INFO:journalctl@ceph.osd.7.smithi097.stdout:Sep 19 09:31:41 smithi097 podman[68286]: 2024-09-19 09:31:41.295790617 +0000 UTC m=+0.536772516 container died 63c434a8db9da0535fa0d03d3cc4c6b998f6c302c681f25f07663d0ed30e4a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-09-19T09:31:41.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:41.127+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:41.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:41.127+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:41.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:41.127+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:41.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:41.127+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:41.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:41.127+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:41.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:41 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:41.495+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:41.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:41 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:41.495+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:41.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:41 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:41.495+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:41.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:41 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:41.495+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:41.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:41 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:41.495+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:41.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:41 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:41.719+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:41.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:41 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:41.719+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:41.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:41 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:41.719+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:41.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:41 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:41.719+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:41.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:41 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:41.719+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:41.928 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T09:31:41.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:41.966 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T09:31:41.966 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T09:31:41.966 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.8 2024-09-19T09:31:41.996 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:41.736+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:41.996 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:41.736+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:41.996 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:41.736+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:41.996 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:41 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:41.736+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:12.499176+0000 front 2024-09-19T09:31:12.499278+0000 (oldest deadline 2024-09-19T09:31:38.299520+0000) 2024-09-19T09:31:42.279 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:42 smithi142 systemd[1]: Stopping Ceph osd.8 for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:31:42.279 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:42 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:42.152+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:42.279 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:42 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:42.152+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:42.280 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:42 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:42.152+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:42.280 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:42 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:42.152+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:42.280 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:42 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:42.152+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:42.629 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:42 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:42.276+0000 7fcadeeb3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:42.629 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:42 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:42.276+0000 7fcadeeb3640 -1 osd.8 84 *** Got signal Terminated *** 2024-09-19T09:31:42.629 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:42 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:42.276+0000 7fcadeeb3640 -1 osd.8 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:31:43.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:42 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:42.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:43.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:42 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:42.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:43.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:42 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:42.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:43.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:42 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:42.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:43.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:42 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:42.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:43.128 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:42 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:42.744+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:43.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:42 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:42.744+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:43.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:42 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:42.744+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:43.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:42 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:42.744+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:12.499176+0000 front 2024-09-19T09:31:12.499278+0000 (oldest deadline 2024-09-19T09:31:38.299520+0000) 2024-09-19T09:31:43.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:42 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:42.703+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:43.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:42 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:42.703+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:43.137 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:42 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:42.703+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:43.137 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:42 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:42.703+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:43.137 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:42 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:42.703+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:43.628 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:43 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:43.133+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:43.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:43 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:43.133+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:43.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:43 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:43.133+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:43.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:43 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:43.133+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:43.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:43 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:43.133+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:44.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:43.518+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:44.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:43.518+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:44.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:43.518+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:44.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:43.518+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:44.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:43 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:43.518+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:44.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:43 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:43.751+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:44.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:43 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:43.751+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:44.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:43 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:43.751+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:44.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:43 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:43.751+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:12.499176+0000 front 2024-09-19T09:31:12.499278+0000 (oldest deadline 2024-09-19T09:31:38.299520+0000) 2024-09-19T09:31:44.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:43.687+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:44.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:43.687+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:44.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:43.687+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:44.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:43.687+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:44.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:43 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:43.687+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:44.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:44 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:44.141+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:44.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:44 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:44.141+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:44.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:44 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:44.141+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:44.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:44 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:44.141+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:44.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:44 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:44.141+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:45.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:44 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:44.537+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:45.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:44 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:44.537+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:45.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:44 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:44.537+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:45.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:44 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:44.537+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:45.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:44 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:44.537+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:45.119 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:44 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:44.790+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:45.119 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:44 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:44.790+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:45.119 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:44 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:44.790+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:45.120 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:44 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:44.790+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:12.499176+0000 front 2024-09-19T09:31:12.499278+0000 (oldest deadline 2024-09-19T09:31:38.299520+0000) 2024-09-19T09:31:45.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:44 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:44.641+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:45.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:44 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:44.641+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:45.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:44 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:44.641+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:45.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:44 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:44.641+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:45.137 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:44 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:44.641+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:45.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:45 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:45.117+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:45.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:45 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:45.117+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:45.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:45 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:45.117+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:45.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:45 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:45.117+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:45.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:45 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:45.117+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:45.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:45 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:45.632+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:45.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:45 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:45.632+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:45.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:45 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:45.632+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:45.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:45 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:45.632+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:45.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:45 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:45.632+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:45.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:45 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:45.632+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:46.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:45 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:45.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:46.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:45 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:45.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:46.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:45 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:45.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:46.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:45 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:45.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:46.004 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:45 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:45.509+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:46.128 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:45 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:45.789+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:46.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:45 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:45.789+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:46.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:45 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:45.789+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:46.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:45 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:45.789+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:12.499176+0000 front 2024-09-19T09:31:12.499278+0000 (oldest deadline 2024-09-19T09:31:38.299520+0000) 2024-09-19T09:31:46.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:46 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:46.140+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:46.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:46 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:46.140+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:46.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:46 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:46.140+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:46.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:46 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:46.140+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:46.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:46 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:46.140+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:46.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:46 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:46.140+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:46.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:46 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:46.494+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:46.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:46 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:46.494+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:46.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:46 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:46.494+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:46.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:46 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:46.494+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:46.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:46 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:46.494+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:46.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:46 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:46.596+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:46.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:46 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:46.596+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:46.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:46 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:46.596+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:46.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:46 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:46.596+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:46.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:46 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:46.596+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:46.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:46 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:46.596+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:47.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:46 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:46.814+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.595476+0000 front 2024-09-19T09:30:51.595502+0000 (oldest deadline 2024-09-19T09:31:17.495538+0000) 2024-09-19T09:31:47.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:46 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:46.814+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:57.496051+0000 front 2024-09-19T09:30:57.496085+0000 (oldest deadline 2024-09-19T09:31:20.296757+0000) 2024-09-19T09:31:47.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:46 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:46.814+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:06.598763+0000 front 2024-09-19T09:31:06.598790+0000 (oldest deadline 2024-09-19T09:31:32.498663+0000) 2024-09-19T09:31:47.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:46 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:46.814+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:12.499176+0000 front 2024-09-19T09:31:12.499278+0000 (oldest deadline 2024-09-19T09:31:38.299520+0000) 2024-09-19T09:31:47.129 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:46 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8[53369]: 2024-09-19T09:31:46.814+0000 7fcad74b0640 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:21.200328+0000 front 2024-09-19T09:31:21.200374+0000 (oldest deadline 2024-09-19T09:31:45.900478+0000) 2024-09-19T09:31:47.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:47 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:47.174+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:47.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:47 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:47.174+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:47.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:47 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:47.174+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:47.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:47 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:47.174+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:47.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:47 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:47.174+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:47.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:47 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:47.174+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:47.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:47 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:47.447+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:47.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:47 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:47.447+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:47.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:47 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:47.447+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:47.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:47 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:47.447+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:47.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:47 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:47.447+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:47.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:47 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:47.586+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:47.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:47 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:47.586+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:47.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:47 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:47.586+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:47.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:47 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:47.586+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:47.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:47 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:47.586+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:47.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:47 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:47.586+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:48.021 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:47 smithi142 podman[68847]: 2024-09-19 09:31:47.740583563 +0000 UTC m=+5.572016294 container died a116edc2e79b4b52ba2a6c0b453e9cd2cd6d54acf7a1b09994d17ef918a22183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T09:31:48.379 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:48 smithi142 podman[68847]: 2024-09-19 09:31:48.118945427 +0000 UTC m=+5.950378134 container cleanup a116edc2e79b4b52ba2a6c0b453e9cd2cd6d54acf7a1b09994d17ef918a22183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T09:31:48.379 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:48 smithi142 bash[68847]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8 2024-09-19T09:31:48.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:48 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:48.216+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:48.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:48 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:48.216+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:48.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:48 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:48.216+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:48.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:48 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:48.216+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:48.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:48 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:48.216+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:48.379 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:48 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:48.216+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:48.736 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:48 smithi142 podman[68864]: 2024-09-19 09:31:48.389030991 +0000 UTC m=+0.648694148 container remove a116edc2e79b4b52ba2a6c0b453e9cd2cd6d54acf7a1b09994d17ef918a22183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T09:31:48.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:48 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:48.484+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:48.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:48 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:48.484+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:48.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:48 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:48.484+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:48.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:48 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:48.484+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:48.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:48 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:48.484+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:48.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:48 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:48.600+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:48.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:48 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:48.600+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:48.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:48 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:48.600+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:48.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:48 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:48.600+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:48.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:48 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:48.600+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:48.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:48 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:48.600+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:49.010 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:48 smithi142 podman[68952]: 2024-09-19 09:31:48.635603272 +0000 UTC m=+0.015653919 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:31:49.010 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:48 smithi142 podman[68952]: 2024-09-19 09:31:48.77347486 +0000 UTC m=+0.153525535 container create eb3621f0636e51dfe33db4ee57ed828487e810d67419c0904ba1e6d59d3f62a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T09:31:49.010 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:49 smithi142 podman[68952]: 2024-09-19 09:31:49.009036306 +0000 UTC m=+0.389086950 container init eb3621f0636e51dfe33db4ee57ed828487e810d67419c0904ba1e6d59d3f62a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T09:31:49.281 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:49 smithi142 podman[68952]: 2024-09-19 09:31:49.012706375 +0000 UTC m=+0.392757008 container start eb3621f0636e51dfe33db4ee57ed828487e810d67419c0904ba1e6d59d3f62a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T09:31:49.281 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:49 smithi142 podman[68952]: 2024-09-19 09:31:49.100663536 +0000 UTC m=+0.480714204 container attach eb3621f0636e51dfe33db4ee57ed828487e810d67419c0904ba1e6d59d3f62a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:31:49.281 INFO:journalctl@ceph.osd.8.smithi142.stdout:Sep 19 09:31:49 smithi142 podman[68952]: 2024-09-19 09:31:49.207247841 +0000 UTC m=+0.587298474 container died eb3621f0636e51dfe33db4ee57ed828487e810d67419c0904ba1e6d59d3f62a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-19T09:31:49.282 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:49 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:49.168+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:49.282 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:49 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:49.168+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:49.282 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:49 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:49.168+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:49.282 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:49 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:49.168+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:49.282 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:49 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:49.168+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:49.282 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:49 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:49.168+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:49.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:49 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:49.465+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:49.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:49 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:49.465+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:49.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:49 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:49.465+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:49.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:49 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:49.465+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:49.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:49 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:49.465+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:49.871 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T09:31:49.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:49 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:49.566+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:49.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:49 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:49.566+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:49.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:49 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:49.566+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:49.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:49 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:49.566+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:49.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:49 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:49.566+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:49.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:49 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:49.566+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:49.949 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:49.950 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T09:31:49.950 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T09:31:49.950 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.9 2024-09-19T09:31:50.253 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:50 smithi016 systemd[1]: Stopping Ceph osd.9 for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:31:50.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:50 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:50.164+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:50.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:50 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:50.164+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:50.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:50 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:50.164+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:50.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:50 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:50.164+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:50.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:50 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:50.164+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:50.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:50 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:50.164+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:50.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:50 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:50.264+0000 7f4a06045640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:50.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:50 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:50.264+0000 7f4a06045640 -1 osd.9 84 *** Got signal Terminated *** 2024-09-19T09:31:50.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:50 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:50.264+0000 7f4a06045640 -1 osd.9 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:31:50.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:50 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:50.500+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:50.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:50 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:50.500+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:50.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:50 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:50.500+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:50.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:50 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:50.500+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:50.754 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:50 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:50.500+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:50.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:50 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:50.579+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:50.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:50 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:50.579+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:50.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:50 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:50.579+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:50.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:50 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:50.579+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:50.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:50 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:50.579+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:50.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:50 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:50.579+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:51.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:51 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:51.207+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:51.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:51 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:51.207+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:51.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:51 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:51.207+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:51.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:51 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:51.207+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:51.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:51 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:51.207+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:51.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:51 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:51.207+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:51.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:51 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:51.452+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:51.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:51 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:51.452+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:51.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:51 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:51.452+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:51.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:51 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:51.452+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:51.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:51 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:51.452+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:51.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:51 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:51.608+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:51.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:51 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:51.608+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:51.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:51 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:51.608+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:51.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:51 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:51.608+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:51.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:51 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:51.608+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:51.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:51 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:51.608+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:52.628 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:52 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:52.247+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:52.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:52 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:52.247+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:52.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:52 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:52.247+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:52.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:52 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:52.247+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:52.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:52 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:52.247+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:52.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:52 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:52.247+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:52.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:52 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:52.454+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:52.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:52 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:52.454+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:52.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:52 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:52.454+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:52.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:52 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:52.454+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:52.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:52 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:52.454+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:53.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:52 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:52.649+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:53.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:52 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:52.649+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:53.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:52 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:52.649+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:53.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:52 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:52.649+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:53.137 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:52 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:52.649+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:53.137 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:52 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:52.649+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:53.137 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:52 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:52.649+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-19T09:31:29.584776+0000 front 2024-09-19T09:31:29.584848+0000 (oldest deadline 2024-09-19T09:31:52.484452+0000) 2024-09-19T09:31:53.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:53 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:53.294+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:53.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:53 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:53.294+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:53.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:53 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:53.294+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:53.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:53 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:53.294+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:53.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:53 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:53.294+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:53.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:53 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:53.294+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:53.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:53 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:53.489+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:53.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:53 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:53.489+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:53.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:53 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:53.489+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:53.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:53 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:53.489+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:53.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:53 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:53.489+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:53.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:53 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:53.603+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:53.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:53 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:53.603+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:53.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:53 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:53.603+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:53.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:53 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:53.603+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:53.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:53 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:53.603+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:53.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:53 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:53.603+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:53.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:53 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:53.603+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-19T09:31:29.584776+0000 front 2024-09-19T09:31:29.584848+0000 (oldest deadline 2024-09-19T09:31:52.484452+0000) 2024-09-19T09:31:54.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:54 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:54.295+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:54.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:54 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:54.295+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:54.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:54 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:54.295+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:54.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:54 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:54.295+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:54.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:54 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:54.295+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:54.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:54 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:54.295+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:54.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:54 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:54.581+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:54.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:54 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:54.581+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:54.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:54 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:54.581+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:54.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:54 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:54.581+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:54.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:54 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:54.581+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:54.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:54 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:54.581+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:54.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:54 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:54.581+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-19T09:31:29.584776+0000 front 2024-09-19T09:31:29.584848+0000 (oldest deadline 2024-09-19T09:31:52.484452+0000) 2024-09-19T09:31:55.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:54 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:54.502+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:43.351575+0000 front 2024-09-19T09:30:43.351623+0000 (oldest deadline 2024-09-19T09:31:08.051789+0000) 2024-09-19T09:31:55.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:54 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:54.502+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.553108+0000 front 2024-09-19T09:30:51.553182+0000 (oldest deadline 2024-09-19T09:31:16.853047+0000) 2024-09-19T09:31:55.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:54 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:54.502+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.554250+0000 front 2024-09-19T09:30:58.554225+0000 (oldest deadline 2024-09-19T09:31:20.853993+0000) 2024-09-19T09:31:55.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:54 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:54.502+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:04.855607+0000 front 2024-09-19T09:31:04.855530+0000 (oldest deadline 2024-09-19T09:31:28.355733+0000) 2024-09-19T09:31:55.003 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:54 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9[62566]: 2024-09-19T09:31:54.502+0000 7f49fe642640 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:15.357375+0000 front 2024-09-19T09:31:15.357529+0000 (oldest deadline 2024-09-19T09:31:36.457359+0000) 2024-09-19T09:31:55.530 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:55 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:55.269+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:55.530 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:55 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:55.269+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:55.530 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:55 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:55.269+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:55.530 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:55 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:55.269+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:55.531 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:55 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:55.269+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:55.531 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:55 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:55.269+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:55.867 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:55 smithi016 podman[92500]: 2024-09-19 09:31:55.60729293 +0000 UTC m=+5.448552940 container died 4429da0321e4999ee4279a7d99a9ea78d0458f75ea84db6aadc870ad93986a30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T09:31:55.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:55 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:55.610+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:55.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:55 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:55.610+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:55.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:55 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:55.610+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:55.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:55 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:55.610+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:55.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:55 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:55.610+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:55.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:55 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:55.610+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:55.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:55 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:55.610+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-19T09:31:29.584776+0000 front 2024-09-19T09:31:29.584848+0000 (oldest deadline 2024-09-19T09:31:52.484452+0000) 2024-09-19T09:31:56.253 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:56 smithi016 podman[92500]: 2024-09-19 09:31:56.04822569 +0000 UTC m=+5.889485688 container cleanup 4429da0321e4999ee4279a7d99a9ea78d0458f75ea84db6aadc870ad93986a30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T09:31:56.254 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:56 smithi016 bash[92500]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9 2024-09-19T09:31:56.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:56 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:56.251+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:56.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:56 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:56.251+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:56.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:56 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:56.251+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:56.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:56 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:56.251+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:56.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:56 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:56.251+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:56.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:56 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:56.251+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:56.753 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:56 smithi016 podman[92526]: 2024-09-19 09:31:56.358417482 +0000 UTC m=+0.744987557 container remove 4429da0321e4999ee4279a7d99a9ea78d0458f75ea84db6aadc870ad93986a30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2) 2024-09-19T09:31:56.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:56 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:56.560+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:56.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:56 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:56.560+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:56.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:56 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:56.560+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:56.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:56 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:56.560+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:56.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:56 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:56.560+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:56.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:56 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:56.560+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:56.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:56 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:56.560+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-19T09:31:29.584776+0000 front 2024-09-19T09:31:29.584848+0000 (oldest deadline 2024-09-19T09:31:52.484452+0000) 2024-09-19T09:31:57.087 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:56 smithi016 podman[92616]: 2024-09-19 09:31:56.728415473 +0000 UTC m=+0.016344117 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:31:57.088 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:56 smithi016 podman[92616]: 2024-09-19 09:31:56.851057576 +0000 UTC m=+0.138986245 container create a012dea699f465cd348d8650d29c4236a6c4d7cbccd77982c0b524a3a9bbd462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T09:31:57.088 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:57 smithi016 podman[92616]: 2024-09-19 09:31:57.08742715 +0000 UTC m=+0.375355788 container init a012dea699f465cd348d8650d29c4236a6c4d7cbccd77982c0b524a3a9bbd462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T09:31:57.503 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:57 smithi016 podman[92616]: 2024-09-19 09:31:57.09084407 +0000 UTC m=+0.378772693 container start a012dea699f465cd348d8650d29c4236a6c4d7cbccd77982c0b524a3a9bbd462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T09:31:57.503 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:57 smithi016 podman[92616]: 2024-09-19 09:31:57.178925077 +0000 UTC m=+0.466853714 container attach a012dea699f465cd348d8650d29c4236a6c4d7cbccd77982c0b524a3a9bbd462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:31:57.503 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 19 09:31:57 smithi016 podman[92616]: 2024-09-19 09:31:57.283228454 +0000 UTC m=+0.571157092 container died a012dea699f465cd348d8650d29c4236a6c4d7cbccd77982c0b524a3a9bbd462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2) 2024-09-19T09:31:57.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:57 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:57.274+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:57.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:57 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:57.274+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:57.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:57 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:57.274+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:57.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:57 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:57.274+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:57.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:57 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:57.274+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:57.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:57 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:57.274+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:57.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:57 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:57.553+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:57.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:57 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:57.553+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:57.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:57 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:57.553+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:57.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:57 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:57.553+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:57.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:57 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:57.553+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:57.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:57 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:57.553+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:57.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:57 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:57.553+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-19T09:31:29.584776+0000 front 2024-09-19T09:31:29.584848+0000 (oldest deadline 2024-09-19T09:31:52.484452+0000) 2024-09-19T09:31:57.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T09:31:57.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:31:57.979 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T09:31:57.979 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T09:31:57.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.10 2024-09-19T09:31:58.287 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:58 smithi097 systemd[1]: Stopping Ceph osd.10 for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:31:58.287 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:58 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:58.284+0000 7fd75c36a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:31:58.287 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:58 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:58.284+0000 7fd75c36a640 -1 osd.10 84 *** Got signal Terminated *** 2024-09-19T09:31:58.288 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:58 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:58.284+0000 7fd75c36a640 -1 osd.10 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:31:58.628 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:58 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:58.238+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:58.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:58 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:58.238+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:58.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:58 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:58.238+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:58.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:58 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:58.238+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:58.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:58 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:58.238+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:58.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:58 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:58.238+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:58.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:58 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:58.599+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:58.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:58 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:58.599+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:58.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:58 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:58.599+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:58.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:58 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:58.599+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:58.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:58 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:58.599+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:58.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:58 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:58.599+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:58.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:58 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:58.599+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-19T09:31:29.584776+0000 front 2024-09-19T09:31:29.584848+0000 (oldest deadline 2024-09-19T09:31:52.484452+0000) 2024-09-19T09:31:59.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:59 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:59.279+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:31:59.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:59 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:59.279+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:31:59.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:59 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:59.279+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:31:59.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:59 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:59.279+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:31:59.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:59 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:59.279+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:31:59.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:31:59 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:31:59.279+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:31:59.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:59 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:59.556+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:31:59.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:59 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:59.556+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:31:59.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:59 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:59.556+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:31:59.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:59 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:59.556+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:31:59.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:59 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:59.556+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:31:59.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:59 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:59.556+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:31:59.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:31:59 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:31:59.556+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-19T09:31:29.584776+0000 front 2024-09-19T09:31:29.584848+0000 (oldest deadline 2024-09-19T09:31:52.484452+0000) 2024-09-19T09:32:00.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:00 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:00.292+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:32:00.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:00 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:00.292+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:32:00.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:00 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:00.292+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:32:00.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:00 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:00.292+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:32:00.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:00 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:00.292+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:32:00.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:00 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:00.292+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:32:00.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:00 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:00.600+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:32:00.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:00 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:00.600+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:32:00.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:00 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:00.600+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:32:00.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:00 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:00.600+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:32:00.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:00 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:00.600+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:32:00.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:00 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:00.600+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:32:00.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:00 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:00.600+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-19T09:31:29.584776+0000 front 2024-09-19T09:31:29.584848+0000 (oldest deadline 2024-09-19T09:31:52.484452+0000) 2024-09-19T09:32:01.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:01 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:01.292+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:32:01.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:01 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:01.292+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:32:01.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:01 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:01.292+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:32:01.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:01 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:01.292+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:32:01.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:01 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:01.292+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:32:01.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:01 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:01.292+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:32:01.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:01 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:01.292+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-09-19T09:31:39.390721+0000 front 2024-09-19T09:31:39.390657+0000 (oldest deadline 2024-09-19T09:32:01.090491+0000) 2024-09-19T09:32:01.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:01 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:01.611+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:32:01.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:01 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:01.611+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:32:01.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:01 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:01.611+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:32:01.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:01 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:01.611+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:32:01.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:01 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:01.611+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:32:01.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:01 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:01.611+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:32:01.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:01 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:01.611+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-19T09:31:29.584776+0000 front 2024-09-19T09:31:29.584848+0000 (oldest deadline 2024-09-19T09:31:52.484452+0000) 2024-09-19T09:32:02.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:02 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:02.340+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:32:02.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:02 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:02.340+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:32:02.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:02 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:02.340+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:32:02.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:02 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:02.340+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:32:02.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:02 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:02.340+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:32:02.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:02 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:02.340+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:32:02.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:02 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:02.340+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-09-19T09:31:39.390721+0000 front 2024-09-19T09:31:39.390657+0000 (oldest deadline 2024-09-19T09:32:01.090491+0000) 2024-09-19T09:32:02.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:02 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:02.592+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:39.978128+0000 front 2024-09-19T09:30:39.978140+0000 (oldest deadline 2024-09-19T09:31:05.278048+0000) 2024-09-19T09:32:02.886 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:02 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:02.592+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:50.579011+0000 front 2024-09-19T09:30:50.579065+0000 (oldest deadline 2024-09-19T09:31:13.479139+0000) 2024-09-19T09:32:02.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:02 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:02.592+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:58.780262+0000 front 2024-09-19T09:30:58.780286+0000 (oldest deadline 2024-09-19T09:31:21.079940+0000) 2024-09-19T09:32:02.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:02 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:02.592+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:03.981034+0000 front 2024-09-19T09:31:03.981072+0000 (oldest deadline 2024-09-19T09:31:28.680955+0000) 2024-09-19T09:32:02.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:02 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:02.592+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.381740+0000 front 2024-09-19T09:31:13.381785+0000 (oldest deadline 2024-09-19T09:31:38.681898+0000) 2024-09-19T09:32:02.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:02 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:02.592+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.783031+0000 front 2024-09-19T09:31:22.783127+0000 (oldest deadline 2024-09-19T09:31:45.082906+0000) 2024-09-19T09:32:02.887 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:02 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10[59579]: 2024-09-19T09:32:02.592+0000 7fd75896f640 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-19T09:31:29.584776+0000 front 2024-09-19T09:31:29.584848+0000 (oldest deadline 2024-09-19T09:31:52.484452+0000) 2024-09-19T09:32:03.862 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:03 smithi097 podman[68411]: 2024-09-19 09:32:03.597653366 +0000 UTC m=+5.432023590 container died bc16396fdd2bfaf74b8f0441cae938dd8b3388ae40c120091d283bd0461e4e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T09:32:03.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:03 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:03.384+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:32:03.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:03 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:03.384+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:32:03.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:03 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:03.384+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:32:03.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:03 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:03.384+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:32:03.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:03 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:03.384+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:32:03.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:03 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:03.384+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:32:03.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:03 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:03.384+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-09-19T09:31:39.390721+0000 front 2024-09-19T09:31:39.390657+0000 (oldest deadline 2024-09-19T09:32:01.090491+0000) 2024-09-19T09:32:04.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:03 smithi097 podman[68411]: 2024-09-19 09:32:03.935269192 +0000 UTC m=+5.769639416 container cleanup bc16396fdd2bfaf74b8f0441cae938dd8b3388ae40c120091d283bd0461e4e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T09:32:04.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:03 smithi097 bash[68411]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10 2024-09-19T09:32:04.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:04 smithi097 podman[68426]: 2024-09-19 09:32:04.111460697 +0000 UTC m=+0.510602669 container remove bc16396fdd2bfaf74b8f0441cae938dd8b3388ae40c120091d283bd0461e4e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T09:32:04.840 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:04 smithi097 podman[68513]: 2024-09-19 09:32:04.545594893 +0000 UTC m=+0.109601937 container create e1555ba33d7a74e09da0d2c2fe9854e8781a5f32bf7645adcb58701302c6c2da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T09:32:04.841 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:04 smithi097 podman[68513]: 2024-09-19 09:32:04.45142442 +0000 UTC m=+0.015431427 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:32:04.841 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:04 smithi097 podman[68513]: 2024-09-19 09:32:04.764865353 +0000 UTC m=+0.328872351 container init e1555ba33d7a74e09da0d2c2fe9854e8781a5f32bf7645adcb58701302c6c2da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T09:32:04.841 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:04 smithi097 podman[68513]: 2024-09-19 09:32:04.768130598 +0000 UTC m=+0.332137604 container start e1555ba33d7a74e09da0d2c2fe9854e8781a5f32bf7645adcb58701302c6c2da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T09:32:04.841 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:04 smithi097 podman[68513]: 2024-09-19 09:32:04.839632329 +0000 UTC m=+0.403639346 container attach e1555ba33d7a74e09da0d2c2fe9854e8781a5f32bf7645adcb58701302c6c2da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-19T09:32:04.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:04 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:04.392+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:32:04.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:04 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:04.392+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:32:04.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:04 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:04.392+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:32:04.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:04 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:04.392+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:32:04.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:04 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:04.392+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:32:04.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:04 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:04.392+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:32:04.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:04 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:04.392+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-09-19T09:31:39.390721+0000 front 2024-09-19T09:31:39.390657+0000 (oldest deadline 2024-09-19T09:32:01.090491+0000) 2024-09-19T09:32:05.136 INFO:journalctl@ceph.osd.10.smithi097.stdout:Sep 19 09:32:04 smithi097 podman[68536]: 2024-09-19 09:32:04.977349717 +0000 UTC m=+0.016605385 container died e1555ba33d7a74e09da0d2c2fe9854e8781a5f32bf7645adcb58701302c6c2da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T09:32:05.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T09:32:05.611 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:32:05.611 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T09:32:05.611 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T09:32:05.611 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.11 2024-09-19T09:32:05.660 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:05 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:05.387+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:32:05.661 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:05 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:05.387+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:32:05.661 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:05 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:05.387+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:32:05.661 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:05 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:05.387+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:32:05.661 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:05 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:05.387+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:32:05.661 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:05 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:05.387+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:32:05.661 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:05 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:05.387+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-09-19T09:31:39.390721+0000 front 2024-09-19T09:31:39.390657+0000 (oldest deadline 2024-09-19T09:32:01.090491+0000) 2024-09-19T09:32:06.129 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:05 smithi142 systemd[1]: Stopping Ceph osd.11 for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:32:06.129 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:05 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:05.904+0000 7ff14aafb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:32:06.129 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:05 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:05.904+0000 7ff14aafb640 -1 osd.11 84 *** Got signal Terminated *** 2024-09-19T09:32:06.129 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:05 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:05.904+0000 7ff14aafb640 -1 osd.11 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T09:32:06.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:06 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:06.393+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:32:06.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:06 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:06.393+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:32:06.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:06 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:06.393+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:32:06.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:06 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:06.393+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:32:06.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:06 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:06.393+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:32:06.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:06 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:06.393+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:32:06.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:06 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:06.393+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-09-19T09:31:39.390721+0000 front 2024-09-19T09:31:39.390657+0000 (oldest deadline 2024-09-19T09:32:01.090491+0000) 2024-09-19T09:32:07.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:07 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:07.373+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:32:07.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:07 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:07.373+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:32:07.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:07 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:07.373+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:32:07.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:07 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:07.373+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:32:07.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:07 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:07.373+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:32:07.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:07 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:07.373+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:32:07.629 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:07 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:07.373+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-09-19T09:31:39.390721+0000 front 2024-09-19T09:31:39.390657+0000 (oldest deadline 2024-09-19T09:32:01.090491+0000) 2024-09-19T09:32:08.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:08 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:08.406+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:32:08.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:08 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:08.406+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:32:08.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:08 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:08.406+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:32:08.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:08 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:08.406+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:32:08.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:08 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:08.406+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:32:08.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:08 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:08.406+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:32:08.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:08 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:08.406+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-09-19T09:31:39.390721+0000 front 2024-09-19T09:31:39.390657+0000 (oldest deadline 2024-09-19T09:32:01.090491+0000) 2024-09-19T09:32:09.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:09 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:09.398+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:32:09.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:09 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:09.398+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:32:09.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:09 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:09.398+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:32:09.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:09 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:09.398+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:32:09.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:09 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:09.398+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:32:09.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:09 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:09.398+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:32:09.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:09 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:09.398+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-09-19T09:31:39.390721+0000 front 2024-09-19T09:31:39.390657+0000 (oldest deadline 2024-09-19T09:32:01.090491+0000) 2024-09-19T09:32:09.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:09 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:09.398+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6820 osd.8 since back 2024-09-19T09:31:46.391421+0000 front 2024-09-19T09:31:46.391447+0000 (oldest deadline 2024-09-19T09:32:08.691420+0000) 2024-09-19T09:32:10.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:10 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:10.439+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-19T09:30:40.581144+0000 front 2024-09-19T09:30:40.581099+0000 (oldest deadline 2024-09-19T09:31:05.781575+0000) 2024-09-19T09:32:10.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:10 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:10.439+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-09-19T09:30:51.082670+0000 front 2024-09-19T09:30:51.082605+0000 (oldest deadline 2024-09-19T09:31:16.382618+0000) 2024-09-19T09:32:10.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:10 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:10.439+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-09-19T09:30:56.382982+0000 front 2024-09-19T09:30:56.383025+0000 (oldest deadline 2024-09-19T09:31:21.083046+0000) 2024-09-19T09:32:10.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:10 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:10.439+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-19T09:31:07.385159+0000 front 2024-09-19T09:31:07.385172+0000 (oldest deadline 2024-09-19T09:31:29.085164+0000) 2024-09-19T09:32:10.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:10 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:10.439+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-09-19T09:31:13.786030+0000 front 2024-09-19T09:31:13.786003+0000 (oldest deadline 2024-09-19T09:31:37.885965+0000) 2024-09-19T09:32:10.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:10 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:10.439+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-09-19T09:31:22.487782+0000 front 2024-09-19T09:31:22.487758+0000 (oldest deadline 2024-09-19T09:31:45.387821+0000) 2024-09-19T09:32:10.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:10 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:10.439+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-09-19T09:31:39.390721+0000 front 2024-09-19T09:31:39.390657+0000 (oldest deadline 2024-09-19T09:32:01.090491+0000) 2024-09-19T09:32:10.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:10 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11[60154]: 2024-09-19T09:32:10.439+0000 7ff147100640 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.142:6820 osd.8 since back 2024-09-19T09:31:46.391421+0000 front 2024-09-19T09:31:46.391447+0000 (oldest deadline 2024-09-19T09:32:08.691420+0000) 2024-09-19T09:32:11.540 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:11 smithi142 podman[69066]: 2024-09-19 09:32:11.280440325 +0000 UTC m=+5.477006357 container died 04b1da0a587eca544fceaf1e9ba334bd97e2040953401e68df2e86929caedba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T09:32:11.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:11 smithi142 podman[69066]: 2024-09-19 09:32:11.646808823 +0000 UTC m=+5.843374851 container cleanup 04b1da0a587eca544fceaf1e9ba334bd97e2040953401e68df2e86929caedba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T09:32:11.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:11 smithi142 bash[69066]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11 2024-09-19T09:32:12.261 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:11 smithi142 podman[69081]: 2024-09-19 09:32:11.897806841 +0000 UTC m=+0.620224976 container remove 04b1da0a587eca544fceaf1e9ba334bd97e2040953401e68df2e86929caedba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T09:32:12.526 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:12 smithi142 podman[69172]: 2024-09-19 09:32:12.163106996 +0000 UTC m=+0.015756869 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T09:32:12.526 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:12 smithi142 podman[69172]: 2024-09-19 09:32:12.281770524 +0000 UTC m=+0.134420383 container create 04186812acc50ddf02f66cd2e3e3f5618f9ae154c77784b1f85ebea84507e9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-19T09:32:12.526 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:12 smithi142 podman[69172]: 2024-09-19 09:32:12.525826412 +0000 UTC m=+0.378476272 container init 04186812acc50ddf02f66cd2e3e3f5618f9ae154c77784b1f85ebea84507e9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-19T09:32:12.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:12 smithi142 podman[69172]: 2024-09-19 09:32:12.529097169 +0000 UTC m=+0.381747032 container start 04186812acc50ddf02f66cd2e3e3f5618f9ae154c77784b1f85ebea84507e9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11-deactivate, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T09:32:12.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:12 smithi142 podman[69172]: 2024-09-19 09:32:12.600958304 +0000 UTC m=+0.453608171 container attach 04186812acc50ddf02f66cd2e3e3f5618f9ae154c77784b1f85ebea84507e9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:32:12.879 INFO:journalctl@ceph.osd.11.smithi142.stdout:Sep 19 09:32:12 smithi142 podman[69172]: 2024-09-19 09:32:12.719024766 +0000 UTC m=+0.571674627 container died 04186812acc50ddf02f66cd2e3e3f5618f9ae154c77784b1f85ebea84507e9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T09:32:13.364 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T09:32:13.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:32:13.407 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T09:32:13.407 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T09:32:13.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.a 2024-09-19T09:32:13.720 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:32:13 smithi016 systemd[1]: Stopping Ceph mds.a for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:32:14.003 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:32:13 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-a[69766]: 2024-09-19T09:32:13.718+0000 7f3112552640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:32:14.003 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:32:13 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-a[69766]: 2024-09-19T09:32:13.718+0000 7f3112552640 -1 mds.a *** got signal Terminated *** 2024-09-19T09:32:16.452 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:32:16 smithi016 podman[92720]: 2024-09-19 09:32:16.175250795 +0000 UTC m=+2.572258328 container died b920f858065034c9d7b04dccd4efa5a4f32d9bd46dd6495a4737a31f25f61b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T09:32:16.753 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:32:16 smithi016 podman[92720]: 2024-09-19 09:32:16.609999594 +0000 UTC m=+3.007007126 container cleanup b920f858065034c9d7b04dccd4efa5a4f32d9bd46dd6495a4737a31f25f61b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T09:32:16.753 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:32:16 smithi016 bash[92720]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-a 2024-09-19T09:32:17.049 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 19 09:32:16 smithi016 podman[92732]: 2024-09-19 09:32:16.79477635 +0000 UTC m=+0.620172570 container remove b920f858065034c9d7b04dccd4efa5a4f32d9bd46dd6495a4737a31f25f61b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T09:32:17.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T09:32:17.089 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:32:17.090 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T09:32:17.090 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T09:32:17.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.d 2024-09-19T09:32:17.423 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 19 09:32:17 smithi016 systemd[1]: Stopping Ceph mds.d for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:32:17.753 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 19 09:32:17 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-d[68578]: 2024-09-19T09:32:17.421+0000 7fc7c6a5d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:32:17.753 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 19 09:32:17 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-d[68578]: 2024-09-19T09:32:17.421+0000 7fc7c6a5d640 -1 mds.d *** got signal Terminated *** 2024-09-19T09:32:18.957 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 19 09:32:18 smithi016 podman[92842]: 2024-09-19 09:32:18.674700483 +0000 UTC m=+1.355990300 container died acd1cdb9e6fc073ccd029cb830547c290dd9b19b02dc3598a9e20100efbb0b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T09:32:19.253 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 19 09:32:19 smithi016 podman[92842]: 2024-09-19 09:32:19.064008252 +0000 UTC m=+1.745298066 container cleanup acd1cdb9e6fc073ccd029cb830547c290dd9b19b02dc3598a9e20100efbb0b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T09:32:19.254 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 19 09:32:19 smithi016 bash[92842]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-d 2024-09-19T09:32:19.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T09:32:19.485 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:32:19.485 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T09:32:19.486 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T09:32:19.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.g 2024-09-19T09:32:20.003 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:32:19 smithi016 systemd[1]: Stopping Ceph mds.g for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:32:20.004 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:32:19 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-g[69351]: 2024-09-19T09:32:19.809+0000 7f45174f4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:32:20.004 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:32:19 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-g[69351]: 2024-09-19T09:32:19.809+0000 7f45174f4640 -1 mds.g *** got signal Terminated *** 2024-09-19T09:32:24.102 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:32:23 smithi016 podman[92965]: 2024-09-19 09:32:23.824096788 +0000 UTC m=+4.117380918 container died 2f96fd20c7fcddb008e1209143faccb8e2da30ab51ef6a3b47dd3fbd11fdc9d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:32:24.503 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:32:24 smithi016 podman[92965]: 2024-09-19 09:32:24.3016319 +0000 UTC m=+4.594916028 container cleanup 2f96fd20c7fcddb008e1209143faccb8e2da30ab51ef6a3b47dd3fbd11fdc9d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:32:24.503 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 19 09:32:24 smithi016 bash[92965]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-g 2024-09-19T09:32:24.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T09:32:24.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:32:24.756 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T09:32:24.756 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T09:32:24.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.j 2024-09-19T09:32:25.165 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:32:24 smithi016 systemd[1]: Stopping Ceph mds.j for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:32:25.503 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:32:25 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-j[68168]: 2024-09-19T09:32:25.163+0000 7fa786975640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:32:25.503 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:32:25 smithi016 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-j[68168]: 2024-09-19T09:32:25.163+0000 7fa786975640 -1 mds.j *** got signal Terminated *** 2024-09-19T09:32:27.524 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:32:27 smithi016 podman[93087]: 2024-09-19 09:32:27.264141883 +0000 UTC m=+2.288959148 container died f9a7443f2ede2962d5683a7fb03ac444daedf9b3a6d2e52f7fe302cb0409fe69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-j, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T09:32:27.892 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:32:27 smithi016 podman[93087]: 2024-09-19 09:32:27.680849464 +0000 UTC m=+2.705666758 container cleanup f9a7443f2ede2962d5683a7fb03ac444daedf9b3a6d2e52f7fe302cb0409fe69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T09:32:27.892 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:32:27 smithi016 bash[93087]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-j 2024-09-19T09:32:28.130 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T09:32:28.152 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:32:27 smithi016 podman[93099]: 2024-09-19 09:32:27.890347318 +0000 UTC m=+0.622344549 container remove f9a7443f2ede2962d5683a7fb03ac444daedf9b3a6d2e52f7fe302cb0409fe69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T09:32:28.152 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:32:28 smithi016 systemd[1]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.j.service: Deactivated successfully. 2024-09-19T09:32:28.152 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 19 09:32:28 smithi016 systemd[1]: Stopped Ceph mds.j for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:32:28.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:32:28.163 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T09:32:28.163 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T09:32:28.163 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.b 2024-09-19T09:32:28.470 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:32:28 smithi097 systemd[1]: Stopping Ceph mds.b for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:32:28.886 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:32:28 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b[64788]: 2024-09-19T09:32:28.468+0000 7fea44d6d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:32:28.886 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:32:28 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b[64788]: 2024-09-19T09:32:28.468+0000 7fea44d6d640 -1 mds.b *** got signal Terminated *** 2024-09-19T09:32:32.479 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:32:32 smithi097 podman[68617]: 2024-09-19 09:32:32.17906221 +0000 UTC m=+3.808636302 container died 582e7ba753cc229799fca4ce6491f95b427d6adcdcb6ba6a80dd7272cb4976a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T09:32:32.828 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:32:32 smithi097 podman[68617]: 2024-09-19 09:32:32.576683205 +0000 UTC m=+4.206257306 container cleanup 582e7ba753cc229799fca4ce6491f95b427d6adcdcb6ba6a80dd7272cb4976a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T09:32:32.828 INFO:journalctl@ceph.mds.b.smithi097.stdout:Sep 19 09:32:32 smithi097 bash[68617]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-b 2024-09-19T09:32:32.958 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T09:32:32.994 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:32:32.994 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T09:32:32.994 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T09:32:32.994 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.e 2024-09-19T09:32:33.386 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:32:33 smithi097 systemd[1]: Stopping Ceph mds.e for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:32:33.386 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:32:33 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-e[64185]: 2024-09-19T09:32:33.321+0000 7f5a7d6ac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:32:33.386 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:32:33 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-e[64185]: 2024-09-19T09:32:33.321+0000 7f5a7d6ac640 -1 mds.e *** got signal Terminated *** 2024-09-19T09:32:36.636 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:32:36 smithi097 podman[68753]: 2024-09-19 09:32:36.169760522 +0000 UTC m=+2.951927197 container died d0bae33a229bff2a68d7421a5409592b9aa65b5b5934367888f0dbfa41a04f98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T09:32:36.990 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:32:36 smithi097 podman[68753]: 2024-09-19 09:32:36.738594719 +0000 UTC m=+3.520761402 container cleanup d0bae33a229bff2a68d7421a5409592b9aa65b5b5934367888f0dbfa41a04f98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T09:32:36.990 INFO:journalctl@ceph.mds.e.smithi097.stdout:Sep 19 09:32:36 smithi097 bash[68753]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-e 2024-09-19T09:32:37.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T09:32:37.202 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:32:37.202 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T09:32:37.202 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T09:32:37.202 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.h 2024-09-19T09:32:37.518 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:32:37 smithi097 systemd[1]: Stopping Ceph mds.h for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:32:37.886 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:32:37 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h[64488]: 2024-09-19T09:32:37.517+0000 7fe96e6d6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:32:37.886 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:32:37 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h[64488]: 2024-09-19T09:32:37.517+0000 7fe96e6d6640 -1 mds.h *** got signal Terminated *** 2024-09-19T09:32:40.452 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:32:40 smithi097 podman[68874]: 2024-09-19 09:32:40.169540767 +0000 UTC m=+2.752438681 container died 08718aa33904b0abd3e5c1dbaf45c8888471181097dabf92e0fc95fb31770dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T09:32:40.785 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:32:40 smithi097 podman[68874]: 2024-09-19 09:32:40.550482007 +0000 UTC m=+3.133379910 container cleanup 08718aa33904b0abd3e5c1dbaf45c8888471181097dabf92e0fc95fb31770dff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T09:32:40.786 INFO:journalctl@ceph.mds.h.smithi097.stdout:Sep 19 09:32:40 smithi097 bash[68874]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-h 2024-09-19T09:32:40.933 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T09:32:40.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:32:40.966 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T09:32:40.966 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T09:32:40.966 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.k 2024-09-19T09:32:41.388 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:32:41 smithi097 systemd[1]: Stopping Ceph mds.k for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:32:41.388 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:32:41 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-k[63870]: 2024-09-19T09:32:41.295+0000 7eff14823640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:32:41.388 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:32:41 smithi097 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-k[63870]: 2024-09-19T09:32:41.295+0000 7eff14823640 -1 mds.k *** got signal Terminated *** 2024-09-19T09:32:44.539 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:32:44 smithi097 podman[68997]: 2024-09-19 09:32:44.183968095 +0000 UTC m=+2.999687438 container died baea3117dccf740f217320a3c96dc82b12648c011903e48f1982988dfeecc670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:32:44.886 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:32:44 smithi097 podman[68997]: 2024-09-19 09:32:44.637477932 +0000 UTC m=+3.453197276 container cleanup baea3117dccf740f217320a3c96dc82b12648c011903e48f1982988dfeecc670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-k, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T09:32:44.887 INFO:journalctl@ceph.mds.k.smithi097.stdout:Sep 19 09:32:44 smithi097 bash[68997]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-k 2024-09-19T09:32:45.068 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T09:32:45.100 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:32:45.101 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T09:32:45.101 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T09:32:45.101 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.c 2024-09-19T09:32:45.379 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:32:45 smithi142 systemd[1]: Stopping Ceph mds.c for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:32:45.878 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:32:45 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-c[65010]: 2024-09-19T09:32:45.413+0000 7fbc0349e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:32:45.879 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:32:45 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-c[65010]: 2024-09-19T09:32:45.413+0000 7fbc0349e640 -1 mds.c *** got signal Terminated *** 2024-09-19T09:32:48.487 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:32:48 smithi142 podman[69274]: 2024-09-19 09:32:48.177764001 +0000 UTC m=+2.884229193 container died e18c3349f4b65171044f3ffcfd197b6469ab5bd7c698c228effdc94290679c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-c, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T09:32:48.796 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:32:48 smithi142 podman[69274]: 2024-09-19 09:32:48.594132086 +0000 UTC m=+3.300597283 container cleanup e18c3349f4b65171044f3ffcfd197b6469ab5bd7c698c228effdc94290679c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T09:32:48.796 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:32:48 smithi142 bash[69274]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-c 2024-09-19T09:32:49.024 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T09:32:49.046 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:32:48 smithi142 podman[69298]: 2024-09-19 09:32:48.795447288 +0000 UTC m=+0.618434086 container remove e18c3349f4b65171044f3ffcfd197b6469ab5bd7c698c228effdc94290679c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-c, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T09:32:49.046 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:32:49 smithi142 systemd[1]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.c.service: Deactivated successfully. 2024-09-19T09:32:49.046 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:32:49 smithi142 systemd[1]: Stopped Ceph mds.c for ee1ab676-7667-11ef-bceb-c7b262605968. 2024-09-19T09:32:49.046 INFO:journalctl@ceph.mds.c.smithi142.stdout:Sep 19 09:32:49 smithi142 systemd[1]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.c.service: Consumed 2.232s CPU time. 2024-09-19T09:32:49.060 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:32:49.060 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T09:32:49.060 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T09:32:49.060 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.f 2024-09-19T09:32:49.379 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:32:49 smithi142 systemd[1]: Stopping Ceph mds.f for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:32:49.878 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:32:49 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-f[64706]: 2024-09-19T09:32:49.380+0000 7fcf24903640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:32:49.879 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:32:49 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-f[64706]: 2024-09-19T09:32:49.380+0000 7fcf24903640 -1 mds.f *** got signal Terminated *** 2024-09-19T09:32:52.582 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:32:52 smithi142 podman[69403]: 2024-09-19 09:32:52.185669027 +0000 UTC m=+2.911931125 container died 0317d9ea27f71e73ed21cbc1a93872cda640b35af7b09beb67ef01bc28253f8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-f, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T09:32:52.878 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:32:52 smithi142 podman[69403]: 2024-09-19 09:32:52.655716145 +0000 UTC m=+3.381978245 container cleanup 0317d9ea27f71e73ed21cbc1a93872cda640b35af7b09beb67ef01bc28253f8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-19T09:32:52.879 INFO:journalctl@ceph.mds.f.smithi142.stdout:Sep 19 09:32:52 smithi142 bash[69403]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-f 2024-09-19T09:32:53.124 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T09:32:53.157 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:32:53.157 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T09:32:53.157 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T09:32:53.157 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.i 2024-09-19T09:32:53.494 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:32:53 smithi142 systemd[1]: Stopping Ceph mds.i for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:32:53.878 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:32:53 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-i[65309]: 2024-09-19T09:32:53.492+0000 7f058052c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:32:53.879 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:32:53 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-i[65309]: 2024-09-19T09:32:53.492+0000 7f058052c640 -1 mds.i *** got signal Terminated *** 2024-09-19T09:32:56.460 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:32:56 smithi142 podman[69523]: 2024-09-19 09:32:56.187738915 +0000 UTC m=+2.809809271 container died bcc57a0f2b71fab51245af68cdc31302e5f4f43f898f6468637324441fd12b28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T09:32:56.819 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:32:56 smithi142 podman[69523]: 2024-09-19 09:32:56.550802432 +0000 UTC m=+3.172872786 container cleanup bcc57a0f2b71fab51245af68cdc31302e5f4f43f898f6468637324441fd12b28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-i, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T09:32:56.819 INFO:journalctl@ceph.mds.i.smithi142.stdout:Sep 19 09:32:56 smithi142 bash[69523]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-i 2024-09-19T09:32:56.985 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T09:32:57.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:32:57.017 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T09:32:57.017 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T09:32:57.017 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.l 2024-09-19T09:32:57.347 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:32:57 smithi142 systemd[1]: Stopping Ceph mds.l for ee1ab676-7667-11ef-bceb-c7b262605968... 2024-09-19T09:32:57.629 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:32:57 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-l[64407]: 2024-09-19T09:32:57.345+0000 7f858311f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T09:32:57.629 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:32:57 smithi142 ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-l[64407]: 2024-09-19T09:32:57.345+0000 7f858311f640 -1 mds.l *** got signal Terminated *** 2024-09-19T09:33:00.473 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:33:00 smithi142 podman[69643]: 2024-09-19 09:33:00.175968692 +0000 UTC m=+2.936324204 container died bda487c244015e5f98a8a0eed424f1ae9f756f9f3fdf5b9b10ad342ab74d6ac9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-l, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T09:33:00.789 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:33:00 smithi142 podman[69643]: 2024-09-19 09:33:00.546068326 +0000 UTC m=+3.306423763 container cleanup bda487c244015e5f98a8a0eed424f1ae9f756f9f3fdf5b9b10ad342ab74d6ac9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T09:33:00.789 INFO:journalctl@ceph.mds.l.smithi142.stdout:Sep 19 09:33:00 smithi142 bash[69643]: ceph-ee1ab676-7667-11ef-bceb-c7b262605968-mds-l 2024-09-19T09:33:00.909 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ee1ab676-7667-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T09:33:00.940 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T09:33:00.940 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T09:33:00.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ee1ab676-7667-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T09:33:01.151 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:33:03.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ee1ab676-7667-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T09:33:03.841 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:33:06.343 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ee1ab676-7667-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T09:33:06.545 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:33:08.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:33:08.968 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:33:09.000 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T09:33:09.027 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T09:33:09.029 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911905/remote/smithi016/crash 2024-09-19T09:33:09.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/crash -- . 2024-09-19T09:33:09.071 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T09:33:09.071 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-09-19T09:33:09.073 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911905/remote/smithi097/crash 2024-09-19T09:33:09.074 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/crash -- . 2024-09-19T09:33:09.121 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T09:33:09.121 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2024-09-19T09:33:09.124 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911905/remote/smithi142/crash 2024-09-19T09:33:09.125 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/crash -- . 2024-09-19T09:33:09.157 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T09:33:09.157 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2024-09-19T09:33:09.158 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T09:33:09.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T09:33:09.199 INFO:tasks.cephadm:Compressing logs... 2024-09-19T09:33:09.199 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T09:33:09.241 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T09:33:09.244 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T09:33:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-19T09:33:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T09:33:09.267 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T09:33:09.268 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T09:33:09.268 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.0.log: /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T09:33:09.268 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T09:33:09.268 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T09:33:09.271 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T09:33:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T09:33:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mon.a.log: 82.6% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T09:33:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T09:33:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.log 2024-09-19T09:33:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.g.log: /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T09:33:09.272 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T09:33:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T09:33:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:: No such file or directory 2024-09-19T09:33:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T09:33:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T09:33:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.10.log: /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T09:33:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T09:33:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T09:33:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T09:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T09:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T09:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T09:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T09:33:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T09:33:09.278 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.log: 86.1% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T09:33:09.278 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T09:33:09.278 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T09:33:09.278 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T09:33:09.280 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T09:33:09.280 INFO:teuthology.orchestra.run.smithi142.stderr:: No such file or directory 2024-09-19T09:33:09.280 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T09:33:09.280 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T09:33:09.280 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T09:33:09.280 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T09:33:09.280 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T09:33:09.280 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T09:33:09.280 INFO:teuthology.orchestra.run.smithi142.stderr: 82.5% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T09:33:09.280 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T09:33:09.280 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.log 2024-09-19T09:33:09.281 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T09:33:09.281 INFO:teuthology.orchestra.run.smithi142.stderr: 90.5% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T09:33:09.281 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.log: 86.1% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T09:33:09.281 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T09:33:09.281 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T09:33:09.282 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.log 2024-09-19T09:33:09.282 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mgr.y.log: 92.1% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T09:33:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mgr.x.log: 90.8% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T09:33:09.283 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T09:33:09.284 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.8.log: 91.0% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T09:33:09.284 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T09:33:09.284 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T09:33:09.284 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.log: 91.0% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T09:33:09.284 INFO:teuthology.orchestra.run.smithi097.stderr: 86.0% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T09:33:09.286 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T09:33:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T09:33:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T09:33:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T09:33:09.288 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mgr.z.log: 92.2% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T09:33:09.289 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T09:33:09.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T09:33:09.299 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T09:33:09.306 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 93.9% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T09:33:09.308 INFO:teuthology.orchestra.run.smithi016.stderr: 94.0% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T09:33:09.313 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T09:33:09.315 INFO:teuthology.orchestra.run.smithi016.stderr: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T09:33:09.330 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T09:33:09.333 INFO:teuthology.orchestra.run.smithi142.stderr: 93.5% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T09:33:09.337 INFO:teuthology.orchestra.run.smithi142.stderr: 93.4% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T09:33:09.338 INFO:teuthology.orchestra.run.smithi097.stderr: 93.3% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T09:33:09.349 INFO:teuthology.orchestra.run.smithi097.stderr: 94.0% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T09:33:09.355 INFO:teuthology.orchestra.run.smithi097.stderr: 93.3% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T09:33:09.355 INFO:teuthology.orchestra.run.smithi142.stderr: 93.9% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T09:33:09.485 INFO:teuthology.orchestra.run.smithi016.stderr: 89.8% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T09:33:09.486 INFO:teuthology.orchestra.run.smithi142.stderr: 91.8% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T09:33:09.589 INFO:teuthology.orchestra.run.smithi097.stderr: 91.8% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T09:33:09.590 INFO:teuthology.orchestra.run.smithi142.stderr: 95.8% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T09:33:09.723 INFO:teuthology.orchestra.run.smithi016.stderr: 93.1% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T09:33:09.790 INFO:teuthology.orchestra.run.smithi142.stderr: 93.7% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T09:33:09.823 INFO:teuthology.orchestra.run.smithi097.stderr: 93.4% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T09:33:09.824 INFO:teuthology.orchestra.run.smithi016.stderr: 92.9% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T09:33:09.833 INFO:teuthology.orchestra.run.smithi016.stderr: 93.8% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T09:33:09.842 INFO:teuthology.orchestra.run.smithi097.stderr: 93.2% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T09:33:09.860 INFO:teuthology.orchestra.run.smithi097.stderr: 93.4% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T09:33:09.875 INFO:teuthology.orchestra.run.smithi016.stderr: 90.6% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T09:33:09.912 INFO:teuthology.orchestra.run.smithi142.stderr: 93.6% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T09:33:09.921 INFO:teuthology.orchestra.run.smithi016.stderr: 93.1% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T09:33:09.923 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-19T09:33:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m0.668s 2024-09-19T09:33:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m3.037s 2024-09-19T09:33:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m0.124s 2024-09-19T09:33:09.950 INFO:teuthology.orchestra.run.smithi142.stderr: 93.7% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T09:33:10.038 INFO:teuthology.orchestra.run.smithi097.stderr: 93.7% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T09:33:12.031 INFO:teuthology.orchestra.run.smithi142.stderr: 96.5% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T09:33:12.033 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-09-19T09:33:12.033 INFO:teuthology.orchestra.run.smithi142.stderr:real 0m2.775s 2024-09-19T09:33:12.033 INFO:teuthology.orchestra.run.smithi142.stderr:user 0m5.103s 2024-09-19T09:33:12.033 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m0.203s 2024-09-19T09:33:12.867 INFO:teuthology.orchestra.run.smithi097.stderr: 93.4% -- replaced with /var/log/ceph/ee1ab676-7667-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T09:33:12.869 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-19T09:33:12.869 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m3.611s 2024-09-19T09:33:12.869 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m6.334s 2024-09-19T09:33:12.869 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m0.240s 2024-09-19T09:33:12.870 INFO:tasks.cephadm:Archiving logs... 2024-09-19T09:33:12.870 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911905/remote/smithi016/log 2024-09-19T09:33:12.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T09:33:13.286 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911905/remote/smithi097/log 2024-09-19T09:33:13.287 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T09:33:14.191 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911905/remote/smithi142/log 2024-09-19T09:33:14.197 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T09:33:14.819 INFO:tasks.cephadm:Removing cluster... 2024-09-19T09:33:14.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ee1ab676-7667-11ef-bceb-c7b262605968 --force 2024-09-19T09:33:15.033 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:33:15.301 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ee1ab676-7667-11ef-bceb-c7b262605968 --force 2024-09-19T09:33:15.490 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:33:15.762 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ee1ab676-7667-11ef-bceb-c7b262605968 --force 2024-09-19T09:33:15.970 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: ee1ab676-7667-11ef-bceb-c7b262605968 2024-09-19T09:33:16.250 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T09:33:16.250 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T09:33:16.270 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T09:33:16.286 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T09:33:16.302 INFO:tasks.cephadm:Teardown complete 2024-09-19T09:33:16.303 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T09:33:16.316 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T09:33:16.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T09:33:16.319 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T09:33:16.329 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T09:33:16.438 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T09:33:16.448 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T09:33:16.448 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:33:16.451 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:33:16.453 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T09:33:16.466 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:33:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:33:16.468 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-09-19T09:33:16.470 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:33:16.470 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-09-19T09:33:16.470 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 34 +847us[ +847us] +/- 38ms 2024-09-19T09:33:16.471 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv02.front.sepia.ceph.com 3 7 377 102 -2200us[-2051us] +/- 55ms 2024-09-19T09:33:16.471 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +1122us[+1122us] +/- 61ms 2024-09-19T09:33:16.471 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:33:16.471 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:33:16.471 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-19T09:33:16.471 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv01.front.sepia.ceph.com 3 7 377 99 +838us[ +849us] +/- 37ms 2024-09-19T09:33:16.471 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -2171us[-2157us] +/- 56ms 2024-09-19T09:33:16.472 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +1138us[+1138us] +/- 61ms 2024-09-19T09:33:16.472 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:33:16.473 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T09:33:16.473 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-09-19T09:33:16.473 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 37 +832us[ +837us] +/- 38ms 2024-09-19T09:33:16.473 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -2170us[-2165us] +/- 56ms 2024-09-19T09:33:16.473 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +1142us[+1142us] +/- 61ms 2024-09-19T09:33:16.473 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T09:33:16.474 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T09:33:16.483 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T09:33:16.483 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T09:33:16.493 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T09:33:16.596 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T09:33:16.663 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T09:33:16.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:33:16.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:33:16.764 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:33:16.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:33:16.792 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T09:33:16.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:33:16.819 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T09:33:16.829 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T09:33:16.886 INFO:teuthology.task.internal:Duration was 1316.457209 seconds 2024-09-19T09:33:16.887 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T09:33:16.897 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T09:33:16.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T09:33:16.900 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T09:33:16.902 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T09:33:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:33:16.940 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:33:16.942 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T09:33:17.424 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T09:33:17.424 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-09-19T09:33:17.424 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T09:33:17.448 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-09-19T09:33:17.448 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T09:33:17.494 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-09-19T09:33:17.494 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T09:33:17.518 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T09:33:17.518 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T09:33:17.521 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T09:33:17.536 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T09:33:17.585 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T09:33:17.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T09:33:17.588 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T09:33:17.603 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T09:33:18.332 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T09:33:18.343 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T09:33:18.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T09:33:18.369 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T09:33:18.396 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T09:33:18.423 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T09:33:18.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:33:18.436 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:33:18.438 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:33:18.458 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-09-19T09:33:18.461 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-09-19T09:33:18.488 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-09-19T09:33:18.507 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:33:18.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:33:18.526 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:33:18.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:33:18.542 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T09:33:18.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T09:33:18.562 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T09:33:18.576 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T09:33:18.577 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911905/remote/smithi016 2024-09-19T09:33:18.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T09:33:18.631 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911905/remote/smithi097 2024-09-19T09:33:18.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T09:33:18.668 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911905/remote/smithi142 2024-09-19T09:33:18.668 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T09:33:18.706 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T09:33:18.706 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T09:33:18.709 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T09:33:18.711 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T09:33:18.756 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T09:33:18.768 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T09:33:18.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T09:33:18.780 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T09:33:18.780 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T09:33:18.782 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T09:33:18.785 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T09:33:18.795 INFO:teuthology.orchestra.run.smithi016.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 09:33 /home/ubuntu/cephtest 2024-09-19T09:33:18.797 INFO:teuthology.orchestra.run.smithi097.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 09:33 /home/ubuntu/cephtest 2024-09-19T09:33:18.812 INFO:teuthology.orchestra.run.smithi142.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 09:33 /home/ubuntu/cephtest 2024-09-19T09:33:18.813 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T09:33:18.824 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T09:33:18.850 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/direct_io}} duration: 1316.4572093486786 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T09:33:18.850 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T09:33:18.949 INFO:teuthology.run:pass